linux-rng/kernel/sched/ext/internal.h, branch master

sched_ext: Merge branch 'for-7.3-arena-args' into for-7.3

2026-08-17T21:20:34Z

Pull to receive the __arena argument conversion: 67f1f4a48c24 ("sched_ext: Pass kernel arena pointers to ops_cid callbacks") a8dc810968af ("sched_ext: Convert sub-cap kfuncs to __arena cmask arguments") a05c5b5cb5cf ("sched_ext: Convert scx_bpf_cid_override() to __arena array arguments") along with the bpf-next branch carrying the __arena argument support they depend on. Conflict in kernel/sched/ext/ext.c between: c384ab8a0b13 ("sched_ext: Move the config-off sub-cap kfunc stubs into sub.c") and: a8dc810968af ("sched_ext: Convert sub-cap kfuncs to __arena cmask arguments") which updated the stubs in their old ext.c location. Resolved by keeping ext.c without the stubs and applying the prototype conversion to the relocated stubs in sub.c. Signed-off-by: Tejun Heo

sched_ext: Make core-sched task ordering hierarchy-aware

2026-08-16T02:07:49Z

With sub-schedulers, tasks of different schedulers routinely share rqs and SMT siblings, but scx_prio_less() consults ops.core_sched_before() only when both tasks belong to the same scheduler. Every pair spanning two schedulers falls back to the default ordering, so no scheduler can express ordering across a scheduler boundary, including a root over its sub-schedulers' tasks. Order a pair spanning schedulers by the nearest common ancestor that implements ops.core_sched_before(): both tasks are in its subtree, making this the one op where a scheduler is called on tasks it delegated to its sub-schedulers and may not be scheduling anymore. Same-scheduler pairs keep using the owning scheduler's op so a parent never orders inside a subtree it delegated. The op is skipped when the deciding scheduler is bypassing on either task's CPU. Update scx_qmap to fall back to the kernel's default ordering when handed a delegated task it has no task_ctx for. Signed-off-by: Tejun Heo

sched_ext: fix stale references in doc comments

2026-08-14T19:10:20Z

- inlines.h: scx_bpf_dispatch() doesn't exist; the comment means scx_bpf_sub_dispatch() - internal.h: name %SCX_DEQ_SCHED_CHANGE instead of the never-defined %SCX_DEQ_SAVE - internal.h: @name shows up in the ops file in the scheduler's sysfs directory, not a "kernel.sched_ext_ops" sysctl Signed-off-by: Tao Cui Signed-off-by: Tejun Heo

sched_ext: Pass kernel arena pointers to ops_cid callbacks

2026-08-12T19:55:33Z

The cid-form set_cmask() and sub_caps_updated() callbacks receive cmasks that the kernel builds in the arena, and the kernel converts the kernel addresses to the BPF arena pointer form by hand before each call. BPF now translates between BPF and kernel arena addresses for __arena arguments. Tag the arguments __arena in the cfi stubs and the ops_cid member declarations and pass the kernel arena addresses directly, dropping the manual scx_kaddr_to_arena() conversions and the now-unused helper. The delivered value is unchanged and existing BPF-side code works as before. The arena argument address translation is currently implemented only on x86-64. cid-form schedulers implementing these callbacks load only there for now. Signed-off-by: Tejun Heo

sched_ext: Eject the top rescue consumer on overload

2026-08-03T21:01:36Z

When rescue demand on a cpu persistently exceeds the configured bandwidth, tasks age on that cpu's rescue DSQ until the stall watchdog fires. The watchdog blames the waiting task's owner, but the misbehaving party is whoever floods the queue, not whoever happens to time out. Track each sched's recent rescue consumption per cpu as a decaying average. Once the oldest waiter on a cpu's rescue DSQ has been queued past a threshold derived from the rescue knobs (4s at the defaults), the rescue timer ejects the sub with the highest recent consumption on that cpu with SCX_EXIT_ERROR_RESCUE. With no recent consumer there is no victim and nothing is ejected - the generic stall watchdog eventually blames the waiter's owner instead. Ejections on a cpu are spaced one threshold apart so the freed bandwidth can drain the backlog before another sub is judged. The overload check only wins the race against the stall watchdog when the watchdog timeout clears the threshold, and a single in-budget wait must not cross the trigger on its own. Warn on a scheduler whose timeout doesn't fit and on knobs whose funding period exceeds half the threshold. v2: - Track kill_at in jiffies_64 - on 32-bit, the time_before() grace check wraps 2^31 ticks after the last ejection and suppresses ejections. (sashiko AI) - Track rescue_avg_at in jiffies_64 likewise - the unsigned long decay delta truncates mod 2^32 on 32-bit and can revive a weeks-old usage average in the victim pick. Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi

sched_ext: Add bandwidth-limited rescue execution for stranded tasks

2026-08-03T21:01:29Z

A local DSQ insert lacking the needed caps is diverted to the reject DSQ and bounced back through ops.enqueue() so the scheduler can re-decide. That recovery assumes the scheduler has somewhere legal to send the task. When it doesn't, e.g. when the task's affinity is restricted to cids delegated away, the task starves until the stall watchdog ejects the scheduler. An exiting task is worse - it skips ops.enqueue() and the rejection becomes a self-requeuing cycle that burns the CPU until the watchdog fires. Add SCX_ENQ_RESCUE, a fallback modifier on local DSQ inserts. When the insert would be rejected for missing caps, the kernel takes over and runs the task on the target CPU without consulting the owning scheduler. The kernel sets the flag itself when enqueueing an exiting task. Rescue is a last-resort forward-progress backstop with a persistent disadvantage, not a way around cap enforcement. A per-CPU token bucket accrues rescue_bandwidth_ppt (default 2%) of CPU time and rescues run one at a time in arrival order. Each is granted a slice of the rescue_quantum_us (default 5ms) quantum divided across the waiters, waits at the tail of the local DSQ claiming no priority, and rejoins its scheduler as a fresh arrival once the slice is served. The schedulers keep their normal control over an admitted rescuee and may preempt or reslice it. Service is measured on CPU time actually received, so neither shortens the rescue. Prolonged denial escalates - the remaining slice turns into protected execution (SCX_TASK_PROTECTED) and the rescuee preempts the current task. Escalation is paced by the same bucket, and delivered service converges on the configured bandwidth no matter how aggressively the schedulers dispatch. Both knobs are root-only and SCX_RESCUE_DISABLE turns rescue off, making SCX_ENQ_RESCUE inserts reject as usual. v2: - Add SCX_OPS_OPEN() fix-ups for the new ops fields so cpu-form schedulers setting them still load on older kernels. (Andrea) Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi

sched_ext: Add SCX_TASK_PROTECTED

2026-08-03T21:01:20Z

A BPF scheduler can displace any of its tasks at will - cut a running one's slice with an SCX_ENQ_PREEMPT dispatch, an SCX_KICK_PREEMPT kick or a direct shortening, and jump a queued one with HEAD insertions. Sometimes the kernel needs a slice and a DSQ position to stick regardless. Add SCX_TASK_PROTECTED, guarding both: - The slice becomes immutable. Every scheduler-reachable write is refused and counted as SCX_EV_SLICE_DENIED. Higher scheduling classes are unaffected. PREEMPT|IMMED can't preempt a running protected task and gets reenqueued. - A protected task that reached the head of its DSQ keeps it - HEAD insertions land behind the leading run of protected tasks and reenqueue sweeps skip them. Only rq-owned DSQs can hold protected tasks, so the walk runs only for them. The bit lives in p->scx.flags so that both the refusal and the head walk read it under the rq lock that protects it. Protection ends when the slice is consumed, when the task leaves the rq except for a save/restore on the running task, on a yield, when the scheduler enters bypass, and when the task leaves scx. The flag is kernel-internal and not used yet. Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi

sched_ext: Synchronize slice and dsq_vtime writes

2026-08-03T21:01:14Z

p->scx.slice and p->scx.dsq_vtime writes have no synchronization rules. The dsq insert kfuncs write both fields synchronously from whatever context they're called in - a direct dispatch from ops.select_cpu() writes with only pi_lock held - and, as the kfuncs are safe to call spuriously with the invalid dispatch discarded later, a scheduler can modify any task's slice by spuriously calling them. The latter stands in the way of an upcoming patch which adds kernel-granted slices that the schedulers must not be able to modify. Give both fields explicit rules. While the task is running, sleeping or queued on an rq-owned DSQ, the rq lock protects them - these are the states where the kernel consumes the slice. While queued on a user DSQ or on the BPF side, the kernel neither consumes nor decides on the fields and every writer acts for the BPF scheduler - synchronizing the writers is the scheduler's responsibility and whichever write lands last wins. To conform, an insert kfunc no longer writes the fields when called. The values travel with the dispatch and take effect when the task is inserted. A discarded dispatch has no side effects. The rq lock rule is asserted at the slice store. Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi

sched_ext: Factor out __scx_bpf_now()

2026-08-03T21:00:47Z

scx_bpf_now() couples the valid-or-fresh rq clock read to the current rq. The read is useful for kernel-internal timing against a specific rq, including a remotely locked one. Factor it out into __scx_bpf_now(). Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi

sched_ext: Make several ext.c helpers available outside ext.c

2026-08-03T21:00:39Z

set_task_slice(), task_unlink_from_dsq(), move_local_task_to_local_dsq(), init_dsq() and dump_line() will be used outside ext.c. Add the scx_ prefix and declare them in internal.h. The scx_sched_all list will also be used outside ext.c, drop its static. No functional changes. v2: Declare scx_sched_all outside the CONFIG_EXT_SUB_SCHED block - the definition is unconditional. (sashiko AI) Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi