| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Hybrid automata monitors's clock variables have two different
representations:
- The invariant representation, which is the timestamp when the invariant
expires
- The guard representation, which is the timestamp when the clock is last
reset
This dual representation makes the logic quite difficult to follow (well,
at least for me). It also complicates the monitors and the generation tool,
as it requires conversion back and forth between the representation.
Simplify by using the clock variables for a single purpose: storing the
time stamp since the clock is last reset.
This also allows simplifying rvgen, which will be done in a follow-up
commit.
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Nam Cao <namcao@linutronix.de>
Link: https://lore.kernel.org/r/c0f600dcbf3d8b487c944406851a39146f4d91fa.1781847583.git.namcao@linutronix.de
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
|
|
EXPORT_SYMBOL_GPL(msi_domain_free_irqs_all) was mistakenly placed after
msi_domain_free_irqs_range() instead of after its intended function
msi_domain_free_irqs_all().
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Radu Rendec <radu@rendec.net>
Link: https://patch.msgid.link/20260717143157.1718-1-lirongqing@baidu.com
|
|
Two unrelated things go by "cgroup" in the cid form. Sub-schedulers attach
to cgroups, and the cgroup_*() ops deliver cpu controller events. While the
ops names suggest cgroup2 hierarchy, they actually operate on the cpu
controller.
Rename them to cpuctl_* in struct sched_ext_ops_cid, which has no users
outside scx_qmap yet. The cpu form is deployed ABI and keeps the old names.
The layout is unchanged and the kernel keeps calling through the cpu-form
union view.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Sub-schedulers don't get cgroups yet: every task_group is inited on the root
sched and the routing added by the previous patches always resolves to it.
Add the handover: an enabling sub-scheduler takes over the cgroups in its
subtree and a disabling one returns them to its parent.
scx_cgroup_claim_subtree() runs while the sub enables, after the subtree's
cgrp->scx_sched's are set and before any task is claimed. It inits each
subtree task_group on the sub, exits it from the parent and updates
tg->scx.sched. A failed ops.cgroup_init() unwinds the sub-side inits and
aborts the enable with the parent untouched.
Disabling reverses it with scx_cgroup_return_subtree(): exit each cgroup
from the sub, then re-init it on the parent with the current tg->scx.*
values, resyncing weight and bandwidth changes made while the sub had it.
When a re-init fails, the parent is failed and the remaining task_groups
still transfer uninited and get no cgroup ops - the same punting done for
tasks. The dying parent's own disable moves them onward.
The handover walks include dying but not yet offlined task_groups, the same
as root's bulk walks: a removed cgroup keeps hosting scheduling events until
its dying tasks finish their final context switches, and its
ops.cgroup_exit() must follow the last of them. tg on/offlining is excluded
through cgroup_lock(), so either ordering against an rmdir of a subtree
cgroup delivers balanced init/exit pairs.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
With sub-schedulers claiming cgroup subtrees, cgroup ops must be delivered
to each task_group's sched rather than always to root. Add tg->scx.sched to
track which sched initialized the task_group. It is set and cleared together
with SCX_TG_INITED.
Deliver the ops accordingly:
- ops.cgroup_exit() goes to the sched whose ops.cgroup_init() it pairs with.
- ops.cgroup_prep_move/move/cancel_move() go to the task's sched, and only
for moves that don't re-home the task. A re-homing move is reported
through the ops.exit_task/init_task() pair instead. The cgroups passed to
the move ops can be outside the sched's inited set as the cpu controller
can be coarser than the sub-scheduler topology.
- Knobs of a cgroup belong to the parent, so ops.set_weight/idle/bandwidth()
go to the parent task_group's sched.
All task_groups currently resolve to the root sched, so no behavior changes
until sub-schedulers start claiming cgroups.
While at it, scx_cgroup_init() is restructured so both paths share the
recording.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A task's sched (p->scx.sched) must match its cgroup's owner
(cgrp->scx_sched). cgroup migration breaks the invariant:
scx_cgroup_move_task() only fires root's ops.cgroup_move() and never
re-homes the task, leading to wrong-sched scheduling and, once the stale
sched is freed, a use-after-free.
Hook into the new cgroup task migration events and re-home each task whose
destination cgroup is owned by a different sched. The events map naturally
to the transfer: MIGRATING runs the fallible init for the destination sched,
letting it reject the migration the same way ops.cgroup_prep_move() can,
MIGRATED does the re-home, which can't fail, and CANCELED undoes the init
when the migration falls through.
Pre-commit, the task's task_group still reflects the source, so
__scx_init_task() grows an explicit cgroup argument for the migration path
to hand ops.init_task() the destination cgroup.
Signed-off-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/r/alnxrsexEe_nQwqL@gpd4
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_cgroup_enabled is in the CONFIG_EXT_GROUP_SCHED block. The upcoming
cgroup migration re-homing needs the gate outside the block. Move the
definition and flag flips outside CONFIG_EXT_GROUP_SCHED. No functional
changes.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Factor out scx_rehome_task() and scx_punt_task() from the sub-disable
re-home loop and scx_fail_parent(). The upcoming cgroup migration re-homing
also needs scx_rehome_task(). No functional changes.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A subsystem can attach to the cgroup hierarchy itself, independent of which
controllers are enabled where - BPF hooks already behave this way and
sched_ext sub-schedulers do too. Controller callbacks can't track task
migrations for them: sched_ext must re-home a task whose migration crosses a
sub-scheduler boundary, but the cpu controller's attach callbacks fire only
when the task_group changes and miss moves whenever the controller topology
is coarser than the sub-scheduler topology.
Add cgroup_task_notifier with per-task migration events mirroring the
can_attach/attach/cancel_attach phases so that a consumer which prepares
per-task state can also veto a migration: CGROUP_TASK_MIGRATING fires
pre-commit, CGROUP_TASK_MIGRATED post-commit and
CGROUP_TASK_MIGRATE_CANCELED unwinds a failed migration. Only migrations
that change a task's dfl cgroup are reported.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The BPF_MOV64_PERCPU_REG insn requires JIT to emit native code to for
'dst_reg = src_reg + <percpu_base_off>'.
However, the interpreter ignores the 'off' at its ALU64_MOV_X label.
The 'off' indicates the insn is BPF_MOV64_PERCPU_REG insn. Then, when
the interpreter loads memory from the register, it will hit a page
fault.
[ 2.545572] BUG: unable to handle page fault for address: ffffffffacaaf034
[ 2.546485] #PF: supervisor read access in kernel mode
[ 2.547167] #PF: error_code(0x0000) - not-present page
[ 2.547850] PGD 134e63067 P4D 134e63067 PUD 134e64063 PMD 10021c063 PTE 800ffffeca550062
[ 2.548912] Oops: Oops: 0000 [#1] SMP PTI
Set jit_required as true in order to disallow interpreter fallback in
core.c::__bpf_prog_select_runtime(), if any BPF_ADDR_PERCPU insn is
patched to the prog.
BTW, rename the helper bpf_map_supports_cpu_flags() to
bpf_map_is_percpu_map().
Fixes: 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-4-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The interpreter does not recognize the BPF_JMP|BPF_JA|BPF_X insn, which
is used for insn_array map. Thereafter, it would hit the BUG_ON() in
___bpf_prog_run() at run time.
[ 2.563726] BPF interpreter: unknown opcode 0d (imm: 0x0)
[ 2.564557] ------------[ cut here ]------------
[ 2.565206] kernel BUG at kernel/bpf/core.c:2349!
[ 2.565882] Oops: invalid opcode: 0000 [#1] SMP PTI
Set jit_required as true when insn_array map is used in the prog in
order to disallow interpreter fallback for gotox insn in
core.c::__bpf_prog_select_runtime().
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-3-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Since the interpreter does not support the arena-related insns,
interpreter fallback should not be allowed for these insns in
core.c::__bpf_prog_select_runtime().
Currently, when the interpreter executes the arena ST/LDX/STX insns,
it would hit the BUG_ON() in ___bpf_prog_run() at run time.
[ 2.579196] BPF interpreter: unknown opcode a2 (imm: 0x0)
[ 2.579998] ------------[ cut here ]------------
[ 2.580652] kernel BUG at kernel/bpf/core.c:2349!
[ 2.581314] Oops: invalid opcode: 0000 [#1] SMP PTI
Set jit_required as true when arena map is used in the prog to disallow
interpreter fallback for arena-related insns.
Fixes: 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier.")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_arena_free_pages() accepts scalar arena addresses. The runtime
masks the address to the low 32 bits and reconstructs a full user
address from the arena base before returning the range to the arena
free tree.
When the scalar value is below the low 32 bits of the arena base,
full_uaddr falls below user_vm_start. The existing upper-end clipping
then turns this into an out-of-range free-tree offset. A later
allocation can reuse that offset and return an address below the arena
mapping.
Reject such frees before computing the clipped range.
Fixes: 317460317a02a ("bpf: Introduce bpf_arena.")
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717-c10-031-public-bpf-next-v2-b4-v2-1-54b555443a7c@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
arena is the only map type whose map_mem_usage() still returns 0, so
"bpftool map show" and fdinfo always showed 0 memlock for an arena no
matter how many pages it had.
Count the pages that are actually mapped into the arena: bump a counter in
apply_range_set_cb() when a page goes in and drop it in
apply_range_clear_cb() when a page goes out, both under the arena spinlock.
map_mem_usage() then just returns nr_pages << PAGE_SHIFT.
Only real data pages are counted, not the scratch page.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-3-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Replace the scratch_page field in the pte-callback data with the arena
pointer; later patches use other arena fields from these callbacks. No
functional change.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-2-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Pull to receive:
477869bfafea ("sched_ext: Reject setting disallow from init_task outside the enable path")
5f8b69642d18 ("sched_ext: Take cgroup_lock() first in scx_cgroup_lock()")
8c13364db9c9 ("sched_ext: Skip sub-disable teardown for never-linked sub-schedulers")
5cdc92859809 ("sched_ext: Don't enable non-ext tasks in the sub-sched task loops")
as dependencies for the upcoming cgroup migration patchset and to
resolve the conflicts with the ext.c/sub.c split on for-7.3.
5f8b69642d18 comments scx_cgroup_lock() which for-7.3 exported for
sub.c. Resolved by keeping the exported version with the comment.
8c13364db9c9 and 5cdc92859809 patch the pre-split sub-sched enable and
disable paths in ext.c which for-7.3 moved to sub.c. Resolved by
applying the never-linked teardown skip and the class gates to sub.c.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Root enable and scx_post_fork() enable a task only if it's on the ext class.
Tasks on other classes, possible under an SCX_OPS_SWITCH_PARTIAL root, are
left READY and enabled by switching_to_scx() when they switch over. The sub
enable-commit pass and the sub-disable re-home loop enable unconditionally,
so a fair-class READY task in the subtree becomes ENABLED while not on
sched_ext. A later switch to SCHED_EXT then trips the task state validation
WARN (ENABLED with the previous state not READY) and calls ops.enable() a
second time.
Gate scx_enable_task() on the task's class in both loops.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A sub-scheduler enable can fail before scx_link_sched() links the sched into
the hierarchy, e.g. when the parent is already being disabled, and cleanup
still runs the full scx_sub_disable().
That is racy against root disable: drain_descendants() is the only ordering
between a sub's disable-time task walk and root disable's all-task teardown,
and an unlinked sub is invisible to it. Root's teardown can thus run between
the never-linked sub's drain and its walk, exiting every task to no
scheduler.
The walk then trips the membership WARN and re-homes the exited tasks onto
the dying hierarchy, a use-after-free.
Skip the cgroup ownership reset and the task walk if @sch was never linked,
indicated by the empty ->sibling as unlinking only happens later in the same
function. The membership WARN remains valid: a linked sub is always waited
on by an ancestor's drain.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_cgroup_lock() write-locks scx_cgroup_ops_rwsem and then takes
cgroup_lock(), which can deadlock through kernfs:
scx enable/disable cgroup rmdir cpu.weight write
------------------ ------------ ----------------
cgroup_lock()
percpu_down_write(rwsem)
cgroup_lock()
kernfs_get_active()
percpu_down_read(rwsem)
kernfs_drain()
The enable path waits for the rmdir to release cgroup_mutex. The rmdir,
deactivating the cpu controller's files, waits in kernfs_drain() for the
write's active reference. The write, in scx_group_set_weight(), waits for
the rwsem behind the pending writer.
Take cgroup_lock() first. The set_* paths take no cgroup locks inside the
read side, so a pending write-lock then only waits for read sections that
always run to completion, and no dependency from the rwsem back to
cgroup_mutex remains.
Fixes: a5bd6ba30b33 ("sched_ext: Use cgroup_lock/unlock() to synchronize against cgroup operations")
Cc: stable@vger.kernel.org # v6.18+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The p->scx.disallow revert assumes the root enable path, where the switching
loop reads the reverted policy right afterwards and leaves the task off SCX.
The sub-scheduler disable path also reaches it when re-initializing the
returned tasks on a root parent. Nothing reads the policy there: the task is
enabled on root anyway and keeps running on the ext class with a silently
rewritten policy.
Kill the sched instead, matching the fork and non-root branches, and update
the disallow documentation, which equated !fork with the load path and
pointed at a stale debugfs path for nr_rejected.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Cross-merge BPF and other fixes after downstream PR.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
check_kfunc_call() reads meta.func_name when bpf_fetch_kfunc_arg_meta()
returns -EACCES, but that error can come from fetch_kfunc_meta() (e.g.
fd_array_get_btf() rejecting BTF binding for a signed program) before
meta is memset(), leaving it uninitialized and risking a garbage deref
in verbose().
Move the memset() to the start of bpf_fetch_kfunc_arg_meta() so meta is
zeroed on every error return. The intended "not allowed" -EACCES path
still sets func_name first, so its message is unchanged.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-3-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
For a bloom filter, the value argument of bpf_map_peek_elem() is always
an input. Therefore, the verifier should not allow passing uninitialized
stack memory to it to avoid information leak.
bpf_map_peek_elem() tags its value argument ARG_PTR_TO_MAP_VALUE |
MEM_UNINIT, telling the verifier the callee fills the buffer. This holds
for queue/stack maps, but not for a bloom filter, which reads the buffer
as an input to test set membership and never writes it.
As a result, a program can pass an uninitialized stack buffer to
bpf_map_peek_elem() on a bloom filter. The verifier accepts it and marks
the buffer initialized on return, letting the program read back leftover
kernel stack memory. Bloom maps require CAP_BPF to create, so this is a
CAP_BPF-gated stack infoleak that bypasses the boundary CAP_BPF is meant
to enforce (arbitrary kernel reads are gated behind CAP_PERFMON).
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-2-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Pull bpf fixes from Kumar Kartikeya Dwivedi:
- Fix a UAF in socket clone early bailout paths (Matt Bobrowski)
- Reject unhashed UDP sockets on sockmap update to prevent refcount
leaks (Michal Luczaj)
- Account for receive queue data in FIONREAD on sockmap sockets without
a verdict program (Mattia Meleleo)
- Reject negative constant offsets for verifier buffer pointers (Sun
Jian)
- Fix for tracing of kfuncs with implicit arguments (Ihor Solodrai)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
selftests/bpf: Cover tracing implicit kfunc args
bpf: Fix tracing of kfuncs with implicit args
selftests/bpf: Cover negative buffer pointer offsets
bpf: Reject negative const offsets for buffer pointers
selftests/bpf: Test FIONREAD on a sockmap socket without a verdict program
bpf, sockmap: Account for receive queue in FIONREAD without a verdict program
selftests/bpf: Fail unbound UDP on sockmap update
selftests/bpf: Adapt sockmap update error handling
bpf, sockmap: Reject unhashed UDP sockets on sockmap update
selftests/bpf: Ensure UDP sockets are bound
bpf: Fix UAF in sock clone early bailouts
|
|
The function try_enable_preferred_console() currently has the
non-obvious side effect of returning success for consoles that are
already pre-enabled. This obscures the logic flow during console
registration.
Move the check for pre-enabled consoles directly into the top-level
register_console(). This change makes the handling of pre-enabled
consoles explicit and easier to follow.
Furthermore, this separation lays the groundwork for future cleanups
where try_enable_preferred_console() can be restricted to cases where
an entry actually exists in the console_cmdline[] array.
Also it fixes a possible out-of-bound access when the console_cmdline[]
array is full and no console matched. In fact, the check of
c->user_specified did not make much sense. The new console either
matched and was handled in the for-cycle. Or it did not match
and then *c pointed to an unused entry.
Possible behavior change:
try_enable_preferred_console() will newly be called also with
@user_specified parameter set to "false" when it failed with the "true"
variant. But it looks like the right way to do. It will allow to call
newcon->setup() when the console was preferred by some platform
specific code.
Reported-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com> # out-of-bound access
Closes: https://lore.kernel.org/r/7sq4tr2nmlz32tvkf6vpsghv6exvqfghsrlvywjcqihzsqqbf7@bspclmti5xg4
Reviewed-by: John Ogness <john.ogness@linutronix.de>
Link: https://patch.msgid.link/20260604101459.393162-2-pmladek@suse.com
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
|
The SWIOTLB bounce buffer pool size is hardcoded at 64 MB via
IO_TLB_DEFAULT_SIZE with no compile-time knob to adjust it. On
memory-constrained embedded or mobile platforms equipped with a
hardware IOMMU (e.g., ARM SMMU) covering most DMA-capable devices,
reserving 64 MB at boot is unnecessarily wasteful — the SWIOTLB is
only exercised for devices that bypass the IOMMU or have restricted
DMA address ranges.
Introduce CONFIG_SWIOTLB_DEFAULT_SIZE_MB, an integer Kconfig option
(range 1–64 MB, default 64) that allows platforms to set a smaller
compile-time default. IO_TLB_DEFAULT_SIZE is updated to derive from
this value when CONFIG_SWIOTLB is enabled, preserving the existing
64 MB default when the option is not configured.
The runtime "swiotlb=<nslabs>" kernel parameter override remains
fully supported and takes precedence over the compile-time default.
Signed-off-by: Jagadeesh Pagadala <jpagadal@qti.qualcomm.com>
Signed-off-by: Bibek Kumar Patro <bibek.patro@oss.qualcomm.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Link: https://lore.kernel.org/r/20260702-swiotlb-v2-1-9205f3ba5408@oss.qualcomm.com
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Some JIT compilers, such as x86_64, rely on a register to pass the TCC.
When subprograms of synchronous callback invoke tailcall, C helpers
invoking bpf callback clobber this register, and the corrupted TCC may
bypass the TCC limit, leading to infinite tailcall.
Fix this by rejecting tailcall inside all subprogs of sync callback.
This also cleanly consolidates the existing async and exception callback
checks into a single unified `is_cb` check.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Reported-by: Björn Töpel <bjorn@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260716120157.835937-3-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Currently in check_max_stack_depth_subprog, when the verifier enters a
new callee branch, the local tail_call_reachable is not properly
synchronized with the callee's state.
Consider a main prog branching into multiple subprogs:
subprog0 -> tailcall
main <
subprog1 -> subprog2
When the verifier finishes checking subprog0 and backtracks to main
prog, the local tail_call_reachable state is left as true. As it
proceeds to subprog1, this uncleared state leaks into the new branch,
falsely marking subprog1 and subprog2 as tailcall reachable.
Fix this by explicitly syncing tail_call_reachable with the callee's
has_tail_call state on entry. The caller's state is safely preserved and
restored via the existing backtracking logic.
Fixes: ebf7d1f508a7 ("bpf, x64: rework pro/epilogue and tailcall handling in JIT")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260716120157.835937-2-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
A kfunc marked with KF_IMPLICIT_ARGS flag takes implicit arguments
(such as bpf_prog_aux) that the verifier injects at load time.
resolve_btfids strips those from the kfunc's BTF-visible prototype and
keeps the real kernel ABI in a counterpart _impl prototype [1].
fentry/fexit/fmod_ret/fsession programs may attach to the BPF kernel
functions, including those with implicit args. However
bpf_check_attach_target() and bpf_check_attach_btf_id_multi() extract
the struct btf_func_model from the wrong BTF prototype of the
kfunc. The btf_func_model is later read to construct the trampoline,
which then causes the injected implicit argument to be clobbered and
the kfunc dereferencing garbage.
Add btf_attach_func_proto() to resolve the real ABI prototype of the
kfunc the way the call site does: by looking up the _impl prototype
for a KF_IMPLICIT_ARGS kfunc. Use it at both attach-target model
construction sites.
To enable this, make two supporting changes:
* pass bpf_verifier_log instead of bpf_verifier_env to
find_kfunc_impl_proto(), so it can be reused from the attach path
* add btf_kfunc_check_flag() to test a flag across all of a kfunc's
hook sets, because a program attaching to a kfunc is not in the
kfunc's call-set
KF_IMPLICIT_ARGS must be consistent across the sets, so
btf_kfunc_check_flag() returns -EINVAL on inconsistency.
btf_kfunc_check_flag() reads the kfunc's flags from the target's
kfunc_set_tab. For a module BTF that table is stable only after the
module is live, so take a module reference around the read, mirroring
how the kfunc call path gates the same lookup with btf_try_get_module().
The remaining call sites of btf_distill_func_proto() are safe as
is. The BPF_TRACE_ITER case distills a registered iterator's
prototype, and bpf_struct_ops_desc_init() distills the
function-pointer members of a struct_ops type. Neither is a kfunc, and
so can't have implicit arguments.
[1] https://lore.kernel.org/all/20260120222638.3976562-1-ihor.solodrai@linux.dev/
Fixes: 64e1360524b9 ("bpf: Verifier support for KF_IMPLICIT_ARGS")
Reported-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://github.com/sched-ext/scx/issues/3687#issuecomment-4906694106
Link: https://patch.msgid.link/20260713235223.1639022-2-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
With cgroup v2 migration of a multithreaded process having threads
in different cgroups of a threaded subtree, it is possible that
cpuset_can_attach() can be called with tasks that are not migrating with
respect to cpuset if cpuset controller is not enabled in some of the
subtree cgroups. IOW, the old cpuset can be the same as the new one. This
can cause problem when we need to track the set of old cpusets and the
new cpusets in singly linked lists as a cpuset cannot be in both lists.
As reported by Tejun, the following is an example threaded subtree with
partial cpuset delegation that can cause this issue to show up.
P (+cpuset)
|- R (cpuset) <- destination
| `- C (no cpuset) -> effective cpuset == R
`- W (cpuset)
Group leader in R, thread_a in C, thread_b in W; migrate the whole
process into R (echo $PID > R/cgroup.procs). thread_a moves C->R:
its cgroup changes so compare_css_sets() keeps it in the taskset, but
its cpuset css is unchanged (C inherits R's), so task_cs() == cs ==
R. cpuset is in ss_mask because thread_b (W->R) changed. can_attach()
then tags R as a source (thread_a) and the destination (thread_b):
Handle this special case by skipping tasks that are not migrating in
cpuset_can_attach() and avoid calling cpuset_can_attach_check() in this
case. By doing so, the destination cpuset will not be put into source
cpuset linked list.
As the source cpuset cannot be easily determined in cpuset_attach(),
unnecessary work can be performed if a task is not actually
migrating. However, no harm will be done except wasting some
CPU cycles. If it happens that none of the tasks is migrating,
attach_ctx.old_cs will be NULL and task iteration won't be needed.
Reported-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/lkml/e254af713b5345aec3d086771ecf1e71@kernel.org
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The only case where the cgroup_taskset structure requires task migration
to multiple cpusets is when enabling a cpuset controller in cgroup v2
where the newly created child cpusets inherits the same effective CPUs
and memory nodes from the parent. In that case, task migration can happen
directly with no update to tasks' CPU and memory nodes assignment and no
further work needed from the cpuset side except updating nr_deadline_tasks
when DL tasks are involved and setting old_mems_allowed in the child
cpusets.
Do that by tracking all the destination cpusets with a new dst_cs_head
singly linked list. The reset_migrate_dl_data() function is integrated
into clear_attach_data() so that it can be used for both source and
destination cpusets.
A warning will be printed if there are multiple destination cpusets but
it is not on default hierarchy or when the CPUs or memory nodes change.
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The smp_call*() functions handle their required preemption and CPU
pinning internally. The explicit preempt_disable() in
scftorture_invoke_one() is therefore no longer required for correctness.
Keeping the outer preempt_disable() would also prevent scftorture from
exercising the narrowed internal preemption-disabled regions during IPI
dispatch.
Removing the outer preemption protection can expose a CPU hotplug race in
the test validation when use_cpus_read_lock is false. For multicast
operations, SCF_PRIM_MANY or SCF_PRIM_ALL, if only one CPU is online,
smp_call_function_many() correctly skips sending IPIs and leaves scfc_out
false. Without preemption disabled, a CPU hotplug thread can preempt the
test thread, bring a second CPU online and increment num_online_cpus().
When the test thread resumes, the validation check can observe
num_online_cpus() > 1 and falsely trigger the memory-ordering warning,
leaking the scfcp structure.
Remove the preempt_disable() and preempt_enable() pairs around the
smp_call*() invocations in scftorture_invoke_one(). Restrict the
num_online_cpus() > 1 validation to the use_cpus_read_lock=true case,
where the CPU count is stable during the evaluation.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-10-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() handles the preemption and CPU pinning
requirements internally. on_each_cpu_cond_mask() only builds the call
flags and forwards the request to that helper.
Remove the outer preempt_disable() and preempt_enable() pair from
on_each_cpu_cond_mask().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-9-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() handles the preemption and CPU pinning
requirements internally. smp_call_function() only forwards the request to
that helper for cpu_online_mask and does not access CPU-local state on
its own.
Remove the outer preempt_disable() and preempt_enable() pair from
smp_call_function().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-8-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() still has to keep the caller pinned to the
current CPU while the remote IPI request is built and dispatched. This
protects the queueing state and CPU-hotplug boundary that are required
before the synchronous wait starts:
- It protects the current CPU's per-CPU scratch cpumask,
cfd->cpumask_ipi. Another task running on the same CPU could otherwise
enter smp_call_function_many_cond() and reuse that scratch cpumask
before the current caller has finished building and sending the IPI
request.
- It provides the CPU-hotplug exclusion required by the CSD queueing
side. New CSDs must not be queued after smpcfd_dying_cpu() has flushed
the outgoing CPU's callback queue. Keeping preemption disabled until
all required CSDs have been queued and the corresponding IPIs have
been sent prevents CPU offline from crossing that boundary in the
middle of the queueing operation.
The CSD acquisition side also relies on that caller-side CPU pinning.
csd_lock() waits for CSD_FLAG_LOCK to clear and then marks the CSD busy
with a regular store, so another task on the same CPU must not be
allowed to acquire and reinitialize the same per-CPU CSD concurrently.
After the callbacks have been queued and the IPIs have been sent, the
caller only performs the final csd_lock_wait() completion wait. If it is
preempted there, another task running on the original CPU may enter
smp_call_function_many_cond(), but any attempt to reuse the same per-CPU
CSD will block in csd_lock() until the previous callback clears
CSD_FLAG_LOCK. The final csd_lock_wait() does not acquire or reinitialize
the CSD, so it does not need the same caller-side preemption-disabled
protection.
The wait mask is task-local, so it cannot be overwritten by another task
on the original CPU. The per-CPU CSD storage also remains allocated
across CPU offline, so csd_lock_wait() can safely dereference it even if
the target CPU is offlined after the caller is unpinned.
With those requirements satisfied, enable preemption before the
synchronous csd_lock_wait() loop. This makes the potentially long wait
preemptible and migratable while keeping the CPU-pinned section around
the remote CPU selection and IPI dispatch.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-7-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() uses per-CPU CSD objects when queueing
callbacks to remote CPUs, and the wait path later dereferences those CSDs
from csd_lock_wait().
Making the wait path preemptible allows the initiating task to be
preempted or migrated before it waits for completion. A target CPU can be
offlined in that window. If smpcfd_dead_cpu() frees the target CPU's
per-CPU CSD storage, csd_lock_wait() can later dereference freed memory.
One way to protect the CSD storage is to free it via RCU or after a
synchronization step in the CPU offline path, but that would add
unnecessary complexity and can delay CPU shutdown.
Allocate the per-CPU CSD storage the first time a CPU comes up and keep
it allocated when the CPU is offlined. This allows csd_lock_wait() to
access the CSD even when the target CPU is offlined after preemption is
re-enabled and before the wait is invoked.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: Muchun Song <muchun.song@linux.dev>
Link: https://patch.msgid.link/20260709122933.4021501-6-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() uses the per-CPU cfd->cpumask as the list
of remote CPUs to wait for. That is safe while the caller remains pinned
to the current CPU for the whole operation, because another task cannot
run on the same CPU and reuse the per-CPU mask.
The synchronous wait is the long-latency part of the operation. To make
that wait preemptible, the mask iterated by csd_lock_wait() must remain
stable even if the task is preempted or migrates. If the wait used the
per-CPU cfd->cpumask after dropping CPU pinning, another task scheduled
on the original CPU could enter smp_call_function_many_cond() and
overwrite the mask while the first task is still iterating it.
Give each task private IPI cpumask storage and use it as the wait mask in
smp_call_function_many_cond(). Other cpumask storage choices do not fit
this use case:
- Per-CPU storage is the state that becomes unsafe once the wait is
made preemptible. After the caller drops CPU pinning, another task
scheduled on the original CPU can enter smp_call_function_many_cond()
and reuse the same per-CPU mask.
- Stack storage is not suitable for large NR_CPUS or
CONFIG_CPUMASK_OFFSTACK=y configurations. The wait mask needs to
scale with cpumask_size(), and putting that storage on the stack is
not acceptable on large systems.
- Allocating the mask inside smp_call_function_many_cond() would put an
allocation and a failure path in the generic IPI path. A sleeping
allocation is not suitable because callers have historically only
provided a preempt-disabled context, not a sleepable one. GFP_ATOMIC
would avoid sleeping, but a failure fallback would make the latency
improvement opportunistic instead of guaranteed.
The users are not limited to a small, pre-identifiable class of tasks. On
x86, ordinary tasks can reach this path through TLB flushes during exit,
unmap and reclaim, so allocating the mask only for a known subset of
tasks is not straightforward.
The memory cost is explicit: one word is added to task_struct. When
cpumask_size() fits in that word, the mask is stored inline and no
separate allocation is needed. Larger systems allocate cpumask_size() per
task; on x86-64 NR_CPUS=8192 this is 1 KiB per task. For context,
x86 already carries several KiB of per-task architecture and FPU state,
depending on the enabled features and configuration. That does not make
the extra cpumask free, but it puts the large-NR_CPUS case in
perspective.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-5-zhouchuyi@bytedance.com
|
|
smp_call_function_any() disables preemption across the entire operation:
selecting a target CPU, enqueueing the IPI, and synchronously waiting for
the remote CPU. smp_call_function_single() already re-enables preemption
before the synchronous csd_lock_wait(), so callers of
smp_call_function_any() should benefit from the same shorter
preemption-disabled section.
Simply removing get_cpu() and put_cpu() from smp_call_function_any()
would leave the preemption disablement entirely to
smp_call_function_single(). That opens a preemption window between
selecting the remote CPU, for example via sched_numa_find_nth_cpu(), and
dispatching the IPI in smp_call_function_single(). If the selected CPU is
fully offlined in that window, smp_call_function_single() fails its
cpu_online() check and returns -ENXIO to the caller, violating the
guarantee that smp_call_function_any() executes on any online CPU in the
mask.
Move the remote CPU selection into a common
__smp_call_function_single() helper. Keep the target CPU selection and
IPI dispatch within the same preemption-disabled region, while still
allowing the wait path to use the shorter preemption-disabled section
provided by smp_call_function_single().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-4-zhouchuyi@bytedance.com
|
|
smp_call_function_single() disables preemption while it validates the
target CPU, prepares the call single data, queues the callback and sends
the IPI.
For the !wait case, preemption protects the per-CPU csd_data from
concurrent modification by another task on the same CPU. For the wait
case, the CSD is stack allocated and no other task can reuse it. CPU
pinning is still required until the callback has been queued and the IPI
has been sent, to ensure that the target CPU cannot be offlined after the
online check but before dispatch.
After generic_exec_single() has queued the callback, the synchronous
csd_lock_wait() invocation at the end of the execution does not require
the caller to remain pinned to the current CPU.
Enable preemption before csd_lock_wait() to shorten the
preemption-disabled section.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-3-zhouchuyi@bytedance.com
|
|
The CSD lock wait debugging code in __csd_lock_wait() must run with
preemption disabled. The smp function call mechanisms which invoke it
currently keep preemption disabled across the wait, so the debugging code
inherits that guarantee from its callers.
Keeping preemption disabled across the whole smp function call operation
can induce large scheduling latencies. Shortening the caller-side
preemption-disabled region would invoke __csd_lock_wait() with preemption
enabled.
Prepare for that by disabling preemption explicitly around the CSD lock
wait debugging code in __csd_lock_wait().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: Muchun Song <muchun.song@linux.dev>
Link: https://patch.msgid.link/20260709122933.4021501-2-zhouchuyi@bytedance.com
|
|
LIVEUPDATE_SESSION_GET_NAME was developed in the liveupdate/next branch
while the session type validation change was carried in liveupdate-fixes.
When the conflict between the two branches was resolved, the GET_NAME
operation descriptor picked up the structure and last member from
RETRIEVE_FD.
This makes both its known size and minimum size 16 bytes rather than 72.
A zero-initialized request still succeeds because luo_session_get_name()
writes the full name before luo_ucmd_respond() copies the full GET_NAME
response to userspace. However, copy_struct_from_user() treats the
output-only name field as unknown trailing data and rejects the request
with -E2BIG if any byte in that field is nonzero.
Use the GET_NAME structure and its name field in the descriptor.
Link: https://lore.kernel.org/all/ahWlYXNjGUbkKoHy@sirena.org.uk/
Assisted-by: Codex:gpt-5.6-sol
Reviewed-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
Link: https://patch.msgid.link/20260716012607.22020-1-liu.yun@linux.dev
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
With CONFIG_EXT_SUB_SCHED=y but no sub-scheduler attached - the common case
- hot paths still pay for sub-sched bookkeeping. Gate it behind
__scx_has_subs, a static key counting live sub-schedulers, so that a
root-only system stops paying.
Most conversions are simple skip-if-no-sub tests. scx_idle_notify() is
special - it's a hierarchy walk, so give it a fast path which notifies the
root directly using the same tests as the walk. A pending
SCX_RQ_SUB_IDLE_RENOTIFY can be ignored as no sub can be owed one and the
caller clears the flag either way.
Suggested-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
rq->scx.sub_dispatch_prev is sub-sched-only but was left unconditional. Move
it into the CONFIG_EXT_SUB_SCHED block next to ecaps_to_sync and gate its
updates.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_dispatch_sched() is common dispatch machinery and looks out of place in
sub.h, but it needs scx_cpu_arg() from cid.h and can't move into internal.h
without creating a circular include. Add inlines.h on top of internal.h and
cid.h, and move the function there. The function was sub.h's only cid.h
user, so drop that include. Pure code move, no functional change.
v2: Host the function in a new inlines.h instead of at internal.h's tail,
which formed a circular include with cid.h. Drop sub.h's now-unused
cid.h include. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_discard_ecaps_to_sync() waited for balance_one() to consume a dying
sched's queued ecaps sync, polling with resched_cpu() + msleep(). The wait
is unbounded - the ext dl_server forces picks through sustained fair or RT
load only while ext tasks are queued, so an ext-idle cpu monopolized by a
higher class can stall the teardown indefinitely.
Remove the node directly instead: take all queued nodes, drop the dying
sched's and resplice the rest. Consumption runs under the rq lock and batch
nodes read as on-list throughout, so the producer-side dedup stays correct.
A node that an in-flight scx_process_sync_ecaps() batch holds across a
dispatch-induced rq unlock still needs a wait, but one bounded by that batch
completing rather than by a future balance.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
This is needed by rust/helpers/srcu.c which now adds
rust_helper_srcu_readers_active() as a wrapper around the SRCU helper
for Rust callers.
To achive this:
1- Move the srcu_readers_active() implementation from
"kernel/rcu/srcutree.c" to "include/linux/srcutree.h".
2- Implement a matching srcu_readers_active() in
"include/linux/srcutiny.h" and use it on the existing open-coded
WARN_ON() check in cleanup_srcu_struct().
Signed-off-by: Onur Özkan <work@onurozkan.dev>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Restructure the SRCU initialization functions so it always follows
one direction:
init_srcu_struct() -> __init_srcu_struct() -> lockdep or generic
This uses the same wrapper style as mutex. It avoids the old confusing
style where init_srcu_struct() and __init_srcu_struct() called each
other in different configs. It also helps Rust side to have simpler
helper for SRCU initialization.
Signed-off-by: Onur Özkan <work@onurozkan.dev>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
This commit abstracts the open-coded dumping of reader segments in the
rcu_torture_cleanup() function into a new rcu_torture_dump_read_segs()
function. This abstraction will allow reader segments to be dumped for
other purposes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Now that Tasks Trace RCU is implemented in terms of SRCU, it no longer
has any particular need for the IRQ_WORK Kconfig option. This commit
therefore removes the "select IRQ_WORK" from the TASKS_TRACE_RCU Kconfig
option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The kerneldoc of sugov_iowait_apply() says the IO boost value is increased
in sugov_iowait_apply() and, in the same sentence, that it is decreased by
the same function. That is self-contradictory, and the first part is wrong:
sugov_iowait_apply() only decreases the boost.
The boost is actually increased in sugov_iowait_boost(). Fix the comment to
name sugov_iowait_boost() as the place where the boost is increased, so it
matches the code.
No functional change.
Fixes: fd7d5287fd65 ("cpufreq: schedutil: Cleanup and document iowait boost")
Signed-off-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com>
Reviewed-by: Christian Loehle <christian.loehle@arm.com>
Link: https://patch.msgid.link/20260703092433.4080165-1-zhongqiu.han@oss.qualcomm.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
|
Nam Cao
Li RongQing
Tejun Heo
Leon Hwang
Yiyang Chen
Jiayuan Chen
Kumar Kartikeya Dwivedi
Amery Hung
Linus Torvalds
Petr Mladek
Jagadeesh Pagadala
Pu Lehui
Ihor Solodrai
Waiman Long
Chuyi Zhou
Jackie Liu
Onur Özkan
Paul E. McKenney
Zhongqiu Han