| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
send_signal_locked() unconditionally copies siginfo before the namespace
translation to avoid corrupting a shared siginfo.
Not that I think this can actually hurt performance-wise, just it doesn't
look clean to me; the copy is only needed in the unlikely case when the
translation will actually change something.
Defer it to the two cases where si_pid/si_uid are rewritten, and while at
it add #ifdef's just for completeness.
Link: https://lore.kernel.org/akkaAgNfUby5_3nM@redhat.com
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Eric Biederman <ebiederm@xmission.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
prepare_kill_siginfo(PIDTYPE_TGID) fills si_code = SI_USER and sets
si_pid/si_uid in the sender's namespace. Then send_signal_locked()
translates si_pid/si_uid to the target's namespace.
SEND_SIG_NOINFO exists precisely for the case when si_code == SI_USER
and si_pid/si_uid are the sender's ids; this is exactly what sys_kill()
does via prepare_kill_siginfo(PIDTYPE_TGID). Change sys_kill() to use
it directly.
SEND_SIG_NOINFO produces the same result: si_code = SI_USER, and
__send_signal_locked() computes si_pid/si_uid directly in the target's
namespace. The force computation is also the same: both check if the
sender is visible in the target's pid namespace.
This is just a cleanup and microoptimization (especially with [1]),
this skips the has_si_pid_and_uid() block in send_signal_locked() and
offloads the namespace translation logic to
__send_signal_locked(SEND_SIG_NOINFO) which uses the simpler
computations.
NOTE: As a "side effect" this also fixes the kill(pid < 0, sig) case
where send_signal_locked() rewrites si_pid/si_uid in the shared
siginfo, corrupting it for subsequent recipients. But for other group
senders like __kill_pgrp_info() we still need the fix from Bradley
Morgan [1] who found this problem.
TODO: kill prepare_kill_siginfo() and change other users to use
SEND_SIG_NOINFO too. This needs trivial changes in
__send_signal_locked() and TP_STORE_SIGINFO().
Link: https://lore.kernel.org/aj6btAZqYuv59a8w@redhat.com
Link: https://lore.kernel.org/all/20260622164029.11474-1-include@grrlz.net/ [1]
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Cc: Eric Biederman <ebiederm@xmission.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
send_signal_locked() rewrites sender ids for the target namespace. Group
sends reuse the same siginfo, so one recipient can affect the next.
Copy the siginfo before changing it.
Link: https://lore.kernel.org/86a8857d58d43ee26a8b365b837fd24830343494.1782159692.git.include@grrlz.net
Fixes: 7a0cf094944e ("signal: Correct namespace fixups of si_pid and si_uid")
Signed-off-by: Bradley Morgan <include@grrlz.net>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Adrian Huang <adrianhuang0701@gmail.com>
Cc: Aleksandr Nogikh <nogikh@google.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Marco Elver <elver@google.com>
Cc: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Fix misspellings of "softlockup" in the watchdog enabled bit definitions
and related comments. Also fix a nearby "successful" typo.
No functional change.
Link: https://lore.kernel.org/20260615174557.1836562-1-edcr1790@gmail.com
Signed-off-by: Matthew Chen <edcr1790@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Inside parse_one(), the core parameter-parsing engine prints the address
of the parameter-set callback function using %p: pr_debug("handling %s
with %p\n", param, params[i].ops->set);
Since the string value of the parameter being parsed (val) is already
available, print the parameter name and its value instead, and avoid
tainting the kernel by exposing a kernel-ptr.
Link: https://lore.kernel.org/20260618190715.3563047-1-jim.cromie@gmail.com
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The "resource sanity check" warning need not mean that there is a
functional issue and in the cases when there isn't one, it is just useless
and confusing noise.
For this reason, downgrade the log level of it from "warn" to "debug".
This reduces log clutter while keeping the diagnostic information
available for debugging purposes if needed.
Link: https://lore.kernel.org/20260523000131.7086-1-sakunix@yahoo.com
Signed-off-by: Manuel Quintero Fonseca <sakunix@yahoo.com>
Cc: Andriy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Hans de Goede <hansg@kernel.org>
Cc: Mika Westeberg <mika.westerberg@linux.intel.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
An 'F' entry, and every interpreter a 'B' entry binds, holds a file open
from registration until the entry goes away, pinning the file, its inode,
the mount it came from and that mount's superblock. Nothing bounds how
many of those a user namespace can hold. An entry binds at most
BINFMT_MISC_INTERP_MAX interpreters, but nothing caps the entries.
Charge each binding to the user namespace and uid that makes it against a
new UCOUNT_BINFMT_MISC_INTERPRETERS. Going over budget causes -ENOSPC.
A per-instance cap would suck. Instances are keyed on the user
namespace. So any constant is multiplied by the number of namespaces the
caller creates. Creating those is virtually free. A ucount charges the
namespace and every one of its ancestors. And a namespace can raise only
its own limit. So nesting buys nothing.
The knob is /proc/sys/user/max_binfmt_misc_interpreters. Leave it at the
max_threads/2 default fork_init() gives a new type. No existing
configuration comes close to that.
binfmt_misc is tristate, which makes it the first ucount user that can be
built as a module. Export inc_ucount() and dec_ucount(); without them
CONFIG_BINFMT_MISC=m fails to link. Export them to binfmt_misc alone:
charging a ucount type is not something a module has any business doing
in general, and the list is trivial to extend if a second user shows up.
init_user_ns and init_binfmt_misc are already exported for the same
module.
Link: https://patch.msgid.link/20260803-work-binfmt_misc-interplimit-v1-1-4a2435500bd9@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
When rescue demand on a cpu persistently exceeds the configured bandwidth,
tasks age on that cpu's rescue DSQ until the stall watchdog fires. The
watchdog blames the waiting task's owner, but the misbehaving party is
whoever floods the queue, not whoever happens to time out.
Track each sched's recent rescue consumption per cpu as a decaying average.
Once the oldest waiter on a cpu's rescue DSQ has been queued past a
threshold derived from the rescue knobs (4s at the defaults), the rescue
timer ejects the sub with the highest recent consumption on that cpu with
SCX_EXIT_ERROR_RESCUE. With no recent consumer there is no victim and
nothing is ejected - the generic stall watchdog eventually blames the
waiter's owner instead. Ejections on a cpu are spaced one threshold apart so
the freed bandwidth can drain the backlog before another sub is judged.
The overload check only wins the race against the stall watchdog when the
watchdog timeout clears the threshold, and a single in-budget wait must not
cross the trigger on its own. Warn on a scheduler whose timeout doesn't fit
and on knobs whose funding period exceeds half the threshold.
v2: - Track kill_at in jiffies_64 - on 32-bit, the time_before() grace check
wraps 2^31 ticks after the last ejection and suppresses ejections.
(sashiko AI)
- Track rescue_avg_at in jiffies_64 likewise - the unsigned long decay
delta truncates mod 2^32 on 32-bit and can revive a weeks-old usage
average in the victim pick.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A local DSQ insert lacking the needed caps is diverted to the reject DSQ and
bounced back through ops.enqueue() so the scheduler can re-decide. That
recovery assumes the scheduler has somewhere legal to send the task. When it
doesn't, e.g. when the task's affinity is restricted to cids delegated away,
the task starves until the stall watchdog ejects the scheduler. An exiting
task is worse - it skips ops.enqueue() and the rejection becomes a
self-requeuing cycle that burns the CPU until the watchdog fires.
Add SCX_ENQ_RESCUE, a fallback modifier on local DSQ inserts. When the
insert would be rejected for missing caps, the kernel takes over and runs
the task on the target CPU without consulting the owning scheduler. The
kernel sets the flag itself when enqueueing an exiting task.
Rescue is a last-resort forward-progress backstop with a persistent
disadvantage, not a way around cap enforcement. A per-CPU token bucket
accrues rescue_bandwidth_ppt (default 2%) of CPU time and rescues run one at
a time in arrival order. Each is granted a slice of the rescue_quantum_us
(default 5ms) quantum divided across the waiters, waits at the tail of the
local DSQ claiming no priority, and rejoins its scheduler as a fresh arrival
once the slice is served.
The schedulers keep their normal control over an admitted rescuee and may
preempt or reslice it. Service is measured on CPU time actually received, so
neither shortens the rescue. Prolonged denial escalates - the remaining
slice turns into protected execution (SCX_TASK_PROTECTED) and the rescuee
preempts the current task. Escalation is paced by the same bucket, and
delivered service converges on the configured bandwidth no matter how
aggressively the schedulers dispatch.
Both knobs are root-only and SCX_RESCUE_DISABLE turns rescue off, making
SCX_ENQ_RESCUE inserts reject as usual.
v2: - Add SCX_OPS_OPEN() fix-ups for the new ops fields so cpu-form
schedulers setting them still load on older kernels. (Andrea)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A BPF scheduler can displace any of its tasks at will - cut a running one's
slice with an SCX_ENQ_PREEMPT dispatch, an SCX_KICK_PREEMPT kick or a direct
shortening, and jump a queued one with HEAD insertions. Sometimes the kernel
needs a slice and a DSQ position to stick regardless.
Add SCX_TASK_PROTECTED, guarding both:
- The slice becomes immutable. Every scheduler-reachable write is refused
and counted as SCX_EV_SLICE_DENIED. Higher scheduling classes are
unaffected. PREEMPT|IMMED can't preempt a running protected task and gets
reenqueued.
- A protected task that reached the head of its DSQ keeps it - HEAD
insertions land behind the leading run of protected tasks and reenqueue
sweeps skip them. Only rq-owned DSQs can hold protected tasks, so the walk
runs only for them.
The bit lives in p->scx.flags so that both the refusal and the head walk
read it under the rq lock that protects it.
Protection ends when the slice is consumed, when the task leaves the rq
except for a save/restore on the running task, on a yield, when the
scheduler enters bypass, and when the task leaves scx. The flag is
kernel-internal and not used yet.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
p->scx.slice and p->scx.dsq_vtime writes have no synchronization rules. The
dsq insert kfuncs write both fields synchronously from whatever context
they're called in - a direct dispatch from ops.select_cpu() writes with only
pi_lock held - and, as the kfuncs are safe to call spuriously with the
invalid dispatch discarded later, a scheduler can modify any task's slice by
spuriously calling them. The latter stands in the way of an upcoming patch
which adds kernel-granted slices that the schedulers must not be able to
modify.
Give both fields explicit rules. While the task is running, sleeping or
queued on an rq-owned DSQ, the rq lock protects them - these are the states
where the kernel consumes the slice. While queued on a user DSQ or on the
BPF side, the kernel neither consumes nor decides on the fields and every
writer acts for the BPF scheduler - synchronizing the writers is the
scheduler's responsibility and whichever write lands last wins.
To conform, an insert kfunc no longer writes the fields when called. The
values travel with the dispatch and take effect when the task is inserted. A
discarded dispatch has no side effects. The rq lock rule is asserted at the
slice store.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
SCX_ENQ_IGNORE_CAPS is kernel-internal and marks a placement the kernel
forces. scx_caps_for_enq() waives the enqueue cap for it, but a PREEMPT
insert still picks up the preemption cap requirement from
scx_caps_for_preempt(). Update scx_caps_for_preempt() to take enq_flags and
require nothing when SCX_ENQ_IGNORE_CAPS is set.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The dsq insert kfuncs reject __SCX_ENQ_INTERNAL_MASK bits in
scx_dsq_insert_preamble() instead of scx_vet_enq_flags(). A scheduler can
smuggle internal flags such as SCX_ENQ_CLEAR_OPSS through the dsq move
kfuncs and corrupt the dispatch protocol. Move the rejection into
scx_vet_enq_flags(). The vtime move wrapper OR'd the internal
SCX_ENQ_DSQ_PRIQ bit into enq_flags before the vet; the bit now goes in
inside scx_dsq_move() after the vet.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_bpf_now() couples the valid-or-fresh rq clock read to the current rq.
The read is useful for kernel-internal timing against a specific rq,
including a remotely locked one. Factor it out into __scx_bpf_now().
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
set_task_slice(), task_unlink_from_dsq(), move_local_task_to_local_dsq(),
init_dsq() and dump_line() will be used outside ext.c. Add the scx_ prefix
and declare them in internal.h. The scx_sched_all list will also be used
outside ext.c, drop its static. No functional changes.
v2: Declare scx_sched_all outside the CONFIG_EXT_SUB_SCHED block - the
definition is unconditional. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The following rescue execution addition gives the function a third possible
destination, making a name that enumerates the outcomes a poor fit. Rename
to the destination-neutral scx_resolve_local_dsq(). No functional changes.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The KHO radix tree was designed to track preserved pages. So it does not
provide the capability to track any 64-bit key. Instead, it limits the
key width to how much it needs for tracking PFNs and their orders.
Limiting the width reduces the number of levels in the tree.
KHO is not expected to be the only user of the radix tree. With the API
generalized to allow other users, now it is possible to add any key to
the tree.
Check the key width at kho_radix_add_key(), and error out if it exceeds
what the tree can handle. Do this instead of increasing the tree depth
since right now there are no users that need to use wider keys, so this
avoids memory overhead and ABI breakage.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-4-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
The KHO radix tree constants are somewhat hard to understand. The tree
depth essentially comes from the max key width. The max key width comes
from the need to store a 52-bit PFN plus one more bit for the order.
All this is very obscure with the corrent code. The PFN width is defined
as KHO_ORDER_0_LOG2, which makes very little sense to a new reader not
already familiar with what the value means. Then the fact that an extra
bit is needed is hidden in the KHO_TREE_MAX_DEPTH calculation.
Simplify this by removing KHO_ORDER_0_LOG2 and replace it with
KHO_RADIX_KEY_WIDTH. Update the comment to explain why this value is
used. This moves the +1 from KHO_TREE_MAX_DEPTH to KHO_RADIX_KEY_WIDTH,
making things clearer.
Update kho_{encode,decode}_radix_key() to not use KHO_ORDER_0_LOG2.
Instead, refactor the code and comments to make it clearer how the
encoding and decoding is done.
In kho_encode_radix_key(), add a new variable for the shift for physical
address. Use that in calculating where the order bit goes and in
calculating the shifted PFN. Update comments to make this clearer.
In kho_radix_decode_key(), turn order_bit to 0-indexed to simplify the
eventual calculation for order. Touch up comments to make the
computation clearer.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-3-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
The KHO radix tree is a data structure that can track the presence or
absence of an arbitrary key, with nothing inherently tied to KHO memory
preservation tracking. This was one of the design goals of the radix
tree. This was done to enable it to be re-used by other users of KHO.
Despite that, the radix tree APIs are very closely tied to KHO memory
preservation tracking. Adding a key is done by kho_radix_add_page(),
which encodes it as a page tracking operation and takes in PFN and
order. kho_radix_del_page() does the same. These functions encode the
key internally that goes into the radix tree. kho_radix_walk_tree() does
the same by baking the PFN and order into the callback arguments.
Generalize the APIs by taking the key directly and doing the encoding at
the callers. Rename the functions to kho_radix_add_key() and
kho_radix_del_key(). In practice, this removes a line each from the
functions and moves the encoding function call to the callers.
Similarly, update kho_radix_tree_walk_callback_t to take the key
directly.
Now that key encoding is no longer an inherent part of the radix tree
and can be decided by the user, rename kho_radix_{encode,decode}_key()
to kho_{encode,decode}_radix_key(). This moves them out of the
"kho_radix_" name space into the "kho_" namespace. This emphasizes that
this is KHO's way of encoding the key for its radix tree.
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-2-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Pull liveupdate fix from Mike Rapoport:
- fix a regression caused by allowing coexistence of KHO with deferred
initialization of the memory map
* tag 'liveupdate-fixes-2026-08-03' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux:
kho: align kho_scratch to MAX_ORDER_NR_PAGES pages
|
|
The built-in idle masks are reset with all online CPUs marked idle
before sched_ext is enabled. Busy CPUs can therefore be incorrectly
advertised as idle until their next idle transition.
Initialize the masks empty so that the initial state is conservative.
When bypass is lifted, every CPU is rescheduled and idle-to-idle
re-picks populate the masks with CPUs that are actually idle. Later
idle transitions keep the masks up to date.
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Pull sched_ext fixes from Tejun Heo:
- More lifecycle fixes for the new sub-scheduler support: a failed
enable could tear down a never-linked sub-scheduler in a way that
races the root scheduler's disable and leads to a use-after-free,
tasks that were not on the ext class could still get the enable
callback, and a policy-rejection path silently rewrote a running
task's scheduling policy instead of aborting the scheduler.
- Scheduler enable/disable could deadlock with cgroup removal and a
concurrent cgroup weight write through kernfs. Fixed by reordering
lock acquisition.
- Sync wakeups could leave the waker CPU incorrectly marked idle in the
built-in idle-CPU tracking.
- A selftest fix for sleeping tasks whose CPU affinity changes before
wakeup.
* tag 'sched_ext-for-7.2-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
selftests/sched_ext: Handle sleeping task affinity changes in numa test
sched_ext: Mark waker CPU busy when selected in WAKE_SYNC case
sched_ext: Don't enable non-ext tasks in the sub-sched task loops
sched_ext: Skip sub-disable teardown for never-linked sub-schedulers
sched_ext: Take cgroup_lock() first in scx_cgroup_lock()
sched_ext: Reject setting disallow from init_task outside the enable path
|
|
Pull cgroup fixes from Tejun Heo:
- A pressure trigger's poll timer could be re-armed while the last
trigger was being torn down and then fire after the cgroup was freed.
Tie the timer to the cgroup's lifetime and shut it down when the
cgroup is freed.
- Writing to a pressure file forked a worker kthread while holding the
cgroup mutex, creating lock dependencies from the mutex to the whole
fork path. A pressure write racing a sched_ext scheduler enable,
which blocks forks before grabbing the mutex, deadlocked.
Fork the worker with the mutex dropped.
- Documentation fix for io.latency behavior on non-rotational devices.
* tag 'cgroup-for-7.2-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
Docs/admin-guide/cgroup-v2: document io.latency rotational vs non-rotational behavior
sched/psi: Shut down rtpoll_timer in psi_cgroup_free()
sched/psi: Create the psimon kthread outside of cgroup_mutex
|
|
Commit 814cba835ef6 ("bpf, x86: Fix trampoline stack size for 128-bit
arguments") changed the x86 trampoline to compute the number of
registers from arg_size for every argument, which removed the last user
of BTF_FMODEL_STRUCT_ARG. No other architecture or verifier code looks
at the flag, so remove the macro and the code in __get_type_fmodel_flags()
which sets it.
Keep BTF_FMODEL_SIGNED_ARG at BIT(1) rather than renumbering it to
BIT(0), so BIT(0) is available for a future flag.
No functional change.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260803052726.2821447-1-yonghong.song@linux.dev
|
|
uretprobes_srcu currently uses normal SRCU, which issues
two smp_mb() per read lock/unlock pair. This overhead is
paid on every uretprobe hit.
Switch to SRCU-fast-updown, which eliminates the per-reader
memory barriers by moving the ordering cost to the
grace-period side (synchronize_rcu() instead of smp_mb()).
This is acceptable because grace periods (uprobe
unregistration) are infrequent compared to reader-side
uretprobe hits.
The updown flavor is required because the SRCU read lock is
taken in prepare_uretprobe() when a return instance is
created and is held until that return instance is finalized.
The traced thread returns to user space in between, so the
lock is inherently released in a different context from
where it was acquired: on the normal return path via
uprobe_handle_trampoline() -> hprobe_finalize(), or from
ri_timer() (expiry) or dup_utask() (fork) via
hprobe_expire(). srcu_down_read_fast() / srcu_up_read_fast()
are designed for this acquire-here / release-elsewhere
pattern and, unlike the same-context srcu_read_lock_fast()
variant, do not carry the lockdep read-side tracking that
would warn on it.
The short, same-context SRCU sections in ri_timer() and
dup_utask() (which guard the uprobe against reuse across the
hprobe_expire() cmpxchg) instead use guard(srcu_fast_updown)
for proper lockdep coverage.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://patch.msgid.link/20260706172744.3920417-3-puranjay@kernel.org
|
|
Once binfmt_misc is loaded load_misc_binary() runs for every execve()
on the system since binfmt_misc registers at the head of the formats
list. Every exec therefore performs read_lock() and read_unlock() on
the entries_lock of the relevant binfmt_misc instance, i.e., two
atomic read-modify-writes on a shared cacheline. User namespaces
without their own binfmt_misc mount fall back to an ancestor's
instance so on container-heavy systems every exec on the machine
typically ends up hammering the cacheline of init_binfmt_misc. On
PREEMPT_RT the rwlock additionally turns the handler lookup into a
sleeping lock on the exec fast path.
The lock protects very little. Entries are immutable after publication
except for the Enabled bit which is already toggled locklessly via
set_bit()/clear_bit() and entry lifetime is already handled by the
users refcount via get_binfmt_handler()/put_binfmt_handler(). The read
lock's only remaining job is to make "the entry is still linked" and
"take a reference" atomic with respect to the unlink sites.
Switch the lookup to an RCU walk:
* Lookup walks the entry list under rcu_read_lock() and acquires a
reference via refcount_inc_not_zero(). The refcount can only drop to
zero after an entry has been unlinked so a failed increment means
the walk raced with an unlink. Restarting the search is bounded
because an unlinked entry cannot be found again.
* The unlink sites use hlist_del_init_rcu() which keeps the forward
pointer intact for concurrent walkers and preserves hlist_unhashed()
as the protection against double removal.
* The final put frees the entry via kfree_rcu() as a concurrent walker
may still dereference its flags, magic, mask, and inline strings.
They all live in the entry allocation itself and thus stay valid
until a grace period has elapsed. Closing the interpreter file stays
synchronous. It is only used with a reference already held and all
final puts run in process context.
* Writers remain serialized by the inode lock of the root dentry with
one exception. bm_evict_inode() called from generic_shutdown_super()
during umount unlinks entries without holding it. Keep a spinlock
around the unlink sites instead of relying on superblock lifetime
rules to make that exclusion implicit.
Handler removal semantics are unchanged. An exec that acquired a
reference just before its handler was unregistered already completes
with the removed handler today. The read lock never protected against
that, it only made the window smaller.
With this an exec that matches no binfmt_misc entry, the common case,
no longer writes to any shared cacheline at all.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-5-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The upcoming conversion of the handler lookup to RCU walks cannot use
list_del_init(): reinitializing the forward pointer of a removed entry
would make a concurrent lockless walker standing on that entry loop
back onto it indefinitely. The removal paths do rely on
reinitialization though because bm_{entry,status}_write() and
bm_evict_inode() need to detect whether an entry has already been
unlinked.
hlists support exactly this pattern: hlist_del_init_rcu() keeps the
forward pointer of the removed entry intact for concurrent walkers and
only zeroes ->pprev with hlist_unhashed() serving as the linked test.
Convert the entry list to an hlist now while keeping the rwlock so the
subsequent RCU conversion is a pure locking change. hlist_add_head()
inserts at the head just as list_add() did so lookup precedence
between registered handlers is unchanged.
Link: https://patch.msgid.link/20260710-work-binfmt_misc-locking-v3-4-a162f7cb58d6@kernel.org
Reviewed-by: Jori Koolstra <jkoolstra@xs4all.nl>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Reported by the Sashiko AI review.
arena_vm_fault() returns VM_FAULT_RETRY when it can't take
arena->spinlock, but it never took mmap_lock. The fault path assumes a
VM_FAULT_RETRY handler already dropped mmap_lock and re-takes it on the
retry, so mmap_lock gets taken twice and can deadlock:
do_user_addr_fault()
{
fault = handle_mm_fault(...); // calls arena_vm_fault()
if (fault & VM_FAULT_RETRY)
goto retry; // re-locks mmap_lock
mmap_read_unlock(mm);
}
Return VM_FAULT_SIGBUS instead, for two reasons:
1. We could keep VM_FAULT_RETRY, but then we'd have to drop the fault
lock first and cap the retry ourselves, the way __folio_lock_or_retry()
does.
2. A failed raw_res_spin_lock_irqsave() already means a possible deadlock
was detected, so retrying just hits the same lock again.
So returning VM_FAULT_RETRY here is overkill.
Fixes: b8467290edab ("bpf: arena: make arena kfuncs any context safe")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260728060517.95183-1-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Kfunc argument checking re-derives each argument's kfunc_ptr_arg_type
from BTF on every verification of a call in check_kfunc_args(). Now that
get_kfunc_arg_type() is a function of the kfunc's BTF alone, it no
longer inspects register state. The classification can be computed once
when the call is added and cached. This is a step toward describing
kfuncs with a bpf_func_proto and sharing the helper argument-checking
path.
Generate the classification at bpf_add_kfunc_call() time:
- Extend struct bpf_func_proto to be able to describe a kfunc: widen
arg_type[] and the arg_btf_id[]/arg_size[] union from 5 to
MAX_BPF_FUNC_ARGS, since a kfunc may take up to 12 arguments (5 in
registers, 7 on the stack).
- Embed a bpf_func_proto in struct bpf_kfunc_desc, populated by
gen_kfunc_arg_proto() which runs get_kfunc_arg_type() for each
argument and stores the result in proto.arg_type[]. Grow the
descriptor table's descs[] as a flexible array to not waste memory.
- check_kfunc_args() reads the cached classification from meta->fn
The KF_ARG_PTR_TO_CTX classification depends on the resolved program type,
and for BPF_PROG_TYPE_EXT that is the target program's type, which
resolve_prog_type() reads from prog->aux->saved_dst_prog_type. That field
is normally recorded later during verification in check_attach_btf_id(),
after bpf_add_kfunc_call() has run.
Record saved_dst_prog_type and saved_dst_attach_type from dst_prog at
program load time in bpf_prog_load() so the resolved type is available
at add-call time without reordering check_attach_btf_id(). This keeps
e.g. an freplace of an XDP program calling bpf_xdp_metadata_rx_hash()
classifying its struct xdp_md * argument as context.
The classification result is unchanged; it is only computed earlier and
cached.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-19-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Add kfunc scalar argument types, classify them in get_kfunc_arg_type()
along side with pointer arguments and move scalar type verification
into the main switch in check_kfunc_args(). This keeps BTF-based
classification separate from register validation for every argument,
paving the way for generating the kfunc argument prototype at add-call
time. No functional change intended.
KF_ARG_MEM_SIZE and KF_ARG_CONST_MEM_SIZE now are reachable. Therefore,
remove the fallthrough from KF_ARG_PTR_TO_MEM case and adjust the
register indexing.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-18-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Now that get_kfunc_ptr_arg_type() classifies a kfunc pointer argument
from its BTF alone, express a nullable argument by OR-ing PTR_MAYBE_NULL
into the classified type, and resolve a NULL register after
classification instead of before it.
Previously check_kfunc_args() short-circuited a nullable argument passed
a NULL register with a continue placed before get_kfunc_ptr_arg_type(),
so the NULL never reached classification. That kept a register-state
decision (bpf_register_is_null()) ahead of the BTF-based classification.
This mirrors how helper arguments carry PTR_MAYBE_NULL in their
bpf_arg_type and is a step toward describing kfuncs with a bpf_func_proto:
the nullability now travels with the per-argument classification, so it is
captured when the prototype is generated at add-call time.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-17-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
get_kfunc_ptr_arg_type() decided part of a kfunc pointer argument's type
from the caller's register: a PTR_TO_BTF_ID (or reg2btf_ids) register made
the argument KF_ARG_PTR_TO_BTF_ID, otherwise it fell through to a memory
buffer. Folding register state into argument classification prevents
describing a kfunc's arguments from its BTF alone, which is a prerequisite
for generating a helper-like prototype and eventually sharing the argument
checking (check_func_arg()) between helpers and kfuncs.
Classify pointer arguments from BTF only, and resolve them against the
register in check_kfunc_args():
- A pointer to a struct that is not paired with a __sz/__szk size
argument is classified KF_ARG_PTR_TO_BTF_ID and then checked against
the register. A register carrying a BTF ID (PTR_TO_BTF_ID or a
reg2btf_ids type) must be referenced or trusted and is matched against
the expected type. The only relaxation is when the struct is composed
of scalars, the register may be verified as a fixed-size memory buffer
sized from the BTF type; anything else is rejected.
- A pointer paired with a size argument is always a memory buffer and is
never classified as BTF_ID, so the __sz/__szk case no longer detours
through BTF_ID.
The new design now accepts one previously rejected case: passing
PTR_TO_BTF_ID to a pointer to scalar w/o a following __sz/__szk. The
argument will be classified as KF_ARG_PTR_TO_MEM | MEM_FIXED_SIZE. The
PTR_TO_BTF_ID register will go through check_mem_reg() ->
check_helper_mem_access() -> check_ptr_to_btf_access(). For a pointer to
scalar arg, a kernel btf id will be rejected unless explicitly granted
by btf_struct_access(); a program allocated btf id will be allowed.
The referenced-or-trusted check thus moves into the KF_ARG_PTR_TO_BTF_ID
resolution, alongside the type match.
get_kfunc_ptr_arg_type() no longer needs the register, so drop its regs
and reg parameters; it is now a pure function of the kfunc's BTF.
When a register cannot satisfy a BTF_ID argument, report the register type
passed and, when the expected struct has a reg2btf_ids mapping, the
register type that would be accepted, instead of a confusing "socket".
Update the affected selftest messages accordingly.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-16-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
A kfunc memory-pointer argument comes in two flavors: a fixed-size buffer
whose access size is derived from the pointed-to BTF type, and a
variable-size buffer paired with a following __sz/__szk size argument.
Both were represented by separate kfunc_ptr_arg_type values
(KF_ARG_PTR_TO_MEM vs KF_ARG_PTR_TO_MEM_SIZE) with the pointer classified
as the latter when a size argument followed.
Mirror how helpers describe the same distinction: classify both as
KF_ARG_PTR_TO_MEM and OR in MEM_FIXED_SIZE for the fixed-size case, just
as helpers use ARG_PTR_TO_MEM | MEM_FIXED_SIZE. The switches now key on
base_type(kf_arg_type) so the flag rides along, and the KF_ARG_PTR_TO_MEM
handler either resolves the size from BTF (MEM_FIXED_SIZE) or falls
through to the mem/size-pair check, which validates the buffer against the
following size register and skips it. No functional change.
Currently, KF_ARG_MEM_SIZE and KF_ARG_CONST_MEM_SIZE are only reachable
from ARG_PTR_TO_MEM fallthrough. A patch later will merge scalar
checking into the same switch and remove the fallthrough.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-15-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
get_kfunc_ptr_arg_type() returned KF_ARG_PTR_TO_NULL when a nullable
pointer argument was passed a NULL register. This folded a register-state
decision (bpf_register_is_null()) into what is otherwise BTF-based
argument classification, and it short-circuited before the BTF_ID/MEM
resolution.
Drop KF_ARG_PTR_TO_NULL and handle the NULL case in check_kfunc_args()
instead: a nullable argument that is actually NULL is skipped. Note that
it is okay to skip even when it is a mem+size pair because the size
argument check has been moved to the scalar section. The skip is done
before get_kfunc_ptr_arg_type() so that a NULL passed to a nullable
non-scalar-struct argument is not newly rejected by the BTF_ID/MEM
resolution.
No functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-14-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
check_kfunc_args() already makes sure a scalar value is passed to a
scalar kfunc argument. Drop the check in is_kfunc_arg_mem_size() and
is_kfunc_arg_const_mem_size() to further decouple
get_kfunc_ptr_arg_type() from register state (a prerequisite for
generating a helper-like prototype from kfunc's BTF).
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-13-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
To align helper and kfunc pointer to memory argument handling, move
kfunc constant memorry size argument handling to the kfunc scalar
section. In addition, factor out constant scalar argument handling.
The constant size argument (__szk) of a kfunc memory/size pair was
recorded into meta->arg_constant by a dedicated block in the
KF_ARG_PTR_TO_MEM_SIZE case, duplicating the "only one constant
argument" and "must be a known constant" checks already in the generic
scalar argument handling. That block also did an explicit i++ to skip
the size argument.
This also fixes a precision gap: the old dedicated block did not mark
the size register precise, relying on check_mem_size_reg() for that. But
check_mem_size_reg() is skipped when the buffer is a nullable arg passed
as NULL (e.g. bpf_dynptr_slice(_rdwr) with a NULL buffer), so in that
case the __szk value was recorded and used for regs[R0].mem_size without
marking it precise. Routing the size through the scalar path marks it
precise in all cases.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-11-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
ARG_CONST_SIZE does not require a constant: check_mem_size_reg() accepts
any bounded scalar and verifies the memory access against its maximum
(reg_umax). Rename ARG_CONST_SIZE and ARG_CONST_SIZE_OR_ZERO to
ARG_MEM_SIZE and ARG_MEM_SIZE_OR_ZERO to reflect that. ARG_CONST_ALLOC_
SIZE_OR_ZERO, which does require a constant, is left unchanged.
Pure rename, no functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-10-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Fixed-size memory arguments went through two paths: helpers called
check_helper_mem_access() directly, while kfuncs and global subprogs
used check_mem_reg(). Route the helper MEM_FIXED_SIZE case through
check_mem_reg() too so all three share the same check.
This also fixes a bug in the helper path. When passing a NULL to
PTR_MAYBE_NULL | ARG_PTR_TO_FIXED_SIZE_MEM argument, the program would
be falsely rejected by check_helper_mem_access(). This is not
triggerable since there is no such kind of helper. Also, note that
check_reg_type() still make sure NULL cannot be passed to an argument
not marked with PTR_MAYBE_NULL.
It also tightens the poisoned-stack-slot check. check_mem_reg() encoded
"a STACK_POISON slot may be read" as a negative access size for any
PTR_TO_STACK argument, but that is only sound for global subprogs, where
static stack liveness proved the callee body does not read those slots
(2cb27158adb3 ("bpf: poison dead stack slots")). Since check_mem_reg() is
also used for kfuncs, kfuncs accidentally inherited it and could read a
poisoned (dead, possibly uninitialized) stack slot. Restrict the negative
size to global subprogs (meta == NULL) so kfuncs, like helpers, require
the whole argument initialized.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-9-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Helper ARG_CONST_SIZE and kfunc KF_ARG_PTR_TO_MEM_SIZE memory arguments
already share check_mem_size_reg(), but the kfunc path reached it
through a thin wrapper, check_kfunc_mem_size_reg(). The wrapper existed
only to invoke check_mem_size_reg() twice. Once for BPF_READ and once for
BPF_WRITE because a kfunc mem argument may be both read and written,
whereas a helper argument carries a single access direction.
Let check_mem_size_reg() take a bitmask of access directions (widening
access_type to u32) and perform each requested access, then pass
BPF_READ | BPF_WRITE from the kfunc call site. This removes the
check_kfunc_mem_size_reg() wrapper so helper and kfunc mem+size arguments
run through exactly the same code.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-7-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_map_lookup_elem() is typed to return PTR_TO_MAP_VALUE for every
map, but for some map kinds the looked up value is actually a
different object: an inner map, a socket or an xsk socket.
Until now this reinterpretation happened once the pointer was
converted from its NULL-able form to a concrete value.
Such reinterpretation logic placement led to mark_ptr_not_null_reg()
being called for a temporary register copy in check_mem_reg() and
check_kfunc_mem_size_reg() (check_mem_size_reg() was buggy because of
not calling it). The temporary copy was necessary to pass
reinterpreted parameters as nullable helper and kfunc arguments.
Avoid this complication by refining map lookup result type right away.
The test case verifier_map_in_map/on_the_inner_map_pointer needs an
update because the verifier now prints a concrete NULL-able type for
the lookup.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-6-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
kfunc now shares the same bpf_call_arg_meta with helpers. Pass kfunc's
own meta to check_mem_reg() and check_kfunc_mem_size() instead of NULL
or a temporary meta on the stack.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-5-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Kfuncs used a single '__map' suffix (KF_ARG_PTR_TO_MAP) for two different
things: a verifier-known map matched by map_uid against a bound
timer/wq/task_work object (bpf_wq_init, bpf_task_work_schedule*), and an
opaque 'struct bpf_map *' used only at runtime (bpf_arena_*), which may be a
map fd or a PTR_TO_BTF_ID struct bpf_map (e.g. a bpf_map iterator's ctx->map).
That combined path only accepted the btf map form due to type confusion. The
'if (!reg->map_ptr)' check reads reg->map_ptr, which aliases reg->btf in the
bpf_reg_state union. A PTR_TO_BTF_ID register always has a non-NULL reg->btf,
so the guard silently passed and validation fell through to
process_kf_arg_ptr_to_btf_id(). It also recorded PTR_TO_BTF_ID info in
meta->map, which would be meaningless.
Split the annotation to avoid such type confusion and to align with
helper:
- '__const_map' -> KF_ARG_CONST_MAP_PTR: verifier-known map, handled by
process_map_ptr_arg() like helper ARG_CONST_MAP_PTR.
- '__map' -> KF_ARG_PTR_TO_BTF_ID: opaque struct bpf_map, validated by
process_kf_arg_ptr_to_btf_id(). A map fd still matches via
reg2btf_ids[CONST_PTR_TO_MAP], so bpf_arena_alloc_pages(&map) keeps
working.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-4-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Both the helper ARG_CONST_MAP_PTR and the kfunc KF_ARG_PTR_TO_MAP
recorded the map pointer in meta->map and, when a map was already
bound by a preceding timer/workqueue/task_work argument, rejected a
mismatching map.
Factor the logic into a single process_map_ptr_arg() used by both
paths. The bound-object name (timer, workqueue, or bpf_task_work) is
derived from the bound map's btf_record, and the register numbers in
the message are computed from the map argument position instead of
being hard-coded.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-3-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Drop process_timer_{helper,kfunc}() since bpf_call_arg_meta is now
shared by helper and kfunc. Call process_timer_func() directly.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-2-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
If the SCX_ENABLING -> SCX_ENABLED cmpxchg at the tail of
scx_root_enable_workfn() fails, the function jumps to err_disable
without setting ret. At that point ret still holds the return value
of the last successful __scx_init_task() call, which is 0, so the
err_disable fallback reports the meaningless message:
scx_root_enable() failed (0)
Set ret = -EBUSY, consistent with the other enable-state guards at
the top of the same function, so the fallback always reports a real
errno.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The kernel-doc comment for sched_ext_ops::sub_cgroup_id uses the old
@cgroup_id name, which no longer matches the struct member. This
produces two kernel-doc warnings:
Warning: struct member sub_cgroup_id not described in sched_ext_ops
Warning: Excess struct member cgroup_id description in sched_ext_ops
Update the @param name to match the actual member.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Pull scheduler fix from Ingo Molnar:
- Fix wakeups of deferred DL servers to be actually deferred (Gabriele
Monaco)
* tag 'sched-urgent-2026-08-02' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/deadline: Use revised wakeup rule only for running dl_server
|
|
Pull uprobes fix from Ingo Molnar:
- Fix uretprobes race that can crash the kernel (Breno Leitao)
* tag 'perf-urgent-2026-08-02' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
uprobes: Fix NULL pointer dereference in hprobe_expire()
|
|
The trailing semicolon belongs at the point of use, not in the macro
definition. All uses have been verified to have their own semicolons.
This was found using the following Coccinelle semantic patch:
@r@
identifier i : script:ocaml() { String.lowercase_ascii i = i };
expression e;
@@
*#define i(...) e;
Signed-off-by: Julia Lawall <Julia.Lawall@inria.fr>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
A robust futex unlock stores 0 over the whole futex value - wiping
FUTEX_WAITERS - and wakes a single waiter. That wakeup is a one-shot
notification: the protocol relies on its recipient to either acquire the
futex (and eventually unlock while aware of the remaining contention) or
re-arm FUTEX_WAITERS before sleeping again. If the woken waiter is killed
before it can do either, the kernel must jump in and wake the next task
down the line.
This is a known complication of the futex protocol with a previous
partial fix in commit ca16d5bee598 ("futex: Prevent robust futex exit
race"). Unfortunately, that fix is insufficient.
If a third task re-acquired the futex through the uncontended fast
path in the meantime, the notification is lost: robust exit processing
sees that it is owned by another task and does nothing, while the new
owner sees no FUTEX_WAITERS when it unlocks and wakes nobody.
The remaining waiters sleep forever behind a free futex:
A owns the futex, B and C sleep in FUTEX_WAIT
uval == A | FUTEX_WAITERS
A robust unlock: store 0, FUTEX_WAKE(1) wakes B
uval == 0
D fast path acquire: cmpxchg(0 -> D)
uval == D, no FUTEX_WAITERS
B killed before acting on the wakeup
B exit walk, pending op: owner D != B -> no action
D unlock: no FUTEX_WAITERS -> no wake
C sleeps forever
This is clearly a shortcoming in the implementation, which fails to keep
the FUTEX_WAITERS bit consistent.
Work around this by augmenting the robust list exit processing to also
perform the extra wakeup if the futex word is owned by another thread but
FUTEX_WAITERS is not set.
This does not fix the problem of a non-contended take over/release and free
sequence, which has been discussed for years and has been addressed by
commit 3ca9595d9fb6 ("futex: Add support for unlocking robust futexes") and
subsequent changes, but failed to take the problem described above into
account.
A more complete solution which is based on the in kernel unlock of
contended robust futexes has been discussed in the context of this change
and should show up in mainline sooner than later.
[ tglx: Amend change log slightly and fixup coding style ]
Fixes: ca16d5bee598 ("futex: Prevent robust futex exit race")
Signed-off-by: Keno Fischer <keno@juliahub.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Assisted-by: ClaudeCode:claude-fable-5 tla+
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260730194705.38981-1-keno@juliacomputing.com
|
Oleg Nesterov
Bradley Morgan
Matthew Chen
Jim Cromie
Manuel Quintero Fonseca
Christian Brauner
Tejun Heo
Pratyush Yadav (Google)
Linus Torvalds
Andrea Righi
Yonghong Song
Puranjay Mohan
Jiayuan Chen
Amery Hung
Eduard Zingerman
Liang Luo
Julia Lawall
Keno Fischer