laptop-kernel/kernel/bpf, branch master

bpf: Fix missing migration protection in __rhtab_map_lookup_and_delete_batch()

2026-10-02T11:34:15Z

bpf_mem_cache_free_rcu() uses this_cpu_ptr() which requires migration to be disabled. All callers of rhtab_delete_elem() disable migration except __rhtab_map_lookup_and_delete_batch(), which calls it under rcu_read_lock() only. On CONFIG_PREEMPT_RCU, rcu_read_lock() does not disable preemption or migration, so the task can migrate between CPUs during the delete loop, causing this_cpu_ptr() to trigger: BUG: using smp_processor_id() in preemptible [00000000] code Fix by wrapping the delete loop in migrate_disable()/migrate_enable() in __rhtab_map_lookup_and_delete_batch(), matching the migration protection that the other callers already provide. Fixes: 818e00848227 ("bpf: Implement iteration ops for resizable hashtab") Reported-by: syzbot+fd7e415d891073b83e1f@syzkaller.appspotmail.com Signed-off-by: Ömer Mete Kaya Signed-off-by: Alexei Starovoitov Acked-by: Mykyta Yatsenko Link: https://patch.msgid.link/20260929081609.557899-1-omermetekaya0@gmail.com Closes: https://syzkaller.appspot.com/bug?extid=fd7e415d891073b83e1f

bpf: Fix objects stuck in free_by_rcu_ttrace

2026-10-01T16:40:04Z

do_call_rcu_ttrace() returns early when call_rcu_ttrace_in_progress is set and leaves the objects in free_by_rcu_ttrace. __free_rcu() frees waiting_for_gp_ttrace only and clears the flag. Hence the objects that free_bulk() or __free_by_rcu() added while RCU tasks trace GP was in flight stay in free_by_rcu_ttrace until free_bulk() or alloc_bulk() is called for the same bpf_mem_cache again, which may never happen. The number of such objects is not bounded. Turn call_rcu_ttrace_in_progress into three states: 0 - idle 1 - __free_rcu() is queued 2 - __free_rcu() is queued and free_by_rcu_ttrace got more objects since do_call_rcu_ttrace() sets 2. __free_rcu() does cmpxchg(1 -> 0) and starts the next GP when it fails. It cannot clear the flag first and check free_by_rcu_ttrace later, since bpf_mem_alloc_destroy() frees bpf_mem_cache without waiting for RCU callbacks when the flag is zero. Now __free_rcu() queues itself, so the one that didn't see 'draining' may do call_rcu_tasks_trace() after rcu_barrier_tasks_trace() in free_mem_alloc(). Queue it under rcu_read_lock() and do synchronize_rcu() before the barriers. Calling rcu_barrier_tasks_trace() twice works too, but creating and destroying hash maps in a loop on many cpus slows down to one free_mem_alloc() per GP and kworkers pile up. Fixes: 8d5a8011b35d ("bpf: Batch call_rcu callbacks instead of SLAB_TYPESAFE_BY_RCU.") Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20260930095920.601738-3-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Factor out __do_call_rcu_ttrace()

2026-10-01T16:40:04Z

Move the part of do_call_rcu_ttrace() that runs after call_rcu_ttrace_in_progress is set into __do_call_rcu_ttrace(). The next patch will call it from __free_rcu(). No functional change. Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20260930095920.601738-2-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Fix packet range of pointers sharing an id

2026-10-01T16:39:27Z

Since commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers"), find_good_pkt_pointers() sets the range of all packet pointers sharing an id from the umax of the compared pointer, and check_packet_access() requires umax + off + size <= range. That assumes the umax of two such pointers differ by exactly their constant distance. reg_bounds_sync() breaks it when var_off tightens one umax and not the other: r4 &= 0x38 if r4 > 50 goto exit ; umax 50, var_off (0x0; 0x38) r5 = pkt + r4 ; umax 50 r6 = r5 r6 += 8 ; umax 56, not 58 Comparing r6 with pkt_end sets the range to 56, and the valid 8-byte load at r5 is rejected (50 + 8 > 56). Comparing r5 sets it to 50, and the out-of-bounds 1-byte load at r6 - 7, i.e. r5 + 1, is accepted (56 - 7 + 1 <= 50). Don't call reg_bounds_sync() on a packet pointer that keeps its id (a constant was added or subtracted) or its range (an unknown non-negative value was subtracted), so that var_off cannot tighten its umax. Only update the 32-bit bounds from var_off: reg_bounds_sanity_check() wants them constant when the lower half of var_off is, e.g. for pkt + 8. This relies on nothing else changing the 64-bit bounds of a packet pointer, which holds today. var_off of such a pointer is no longer narrowed by its bounds. Adjust three verifier_align expectations; the low bits, which the alignment checks use, don't change. veristat on the selftests shows no verdict changes and +0.8% insns in test_cls_redirect_subprogs. Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers") Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20261001145255.855630-1-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Skip detached progs in trampoline images that are still in use

2026-09-29T05:54:16Z

bpf_tramp_image_put() makes sure a trampoline image is not freed while a task may still be running in it, but nothing similar is done for the progs called by that image. Detach drops the last prog reference right away and the prog is freed after grace periods, on the basis that a task still in the traced function skips the fexit progs once the nop at ip_after_call is patched to a jump. That leaves out a task sleeping in a sleepable prog that runs before the detached one, which no grace period waits for: CPU 0 CPU 1 in image I, sleeping in prog S detach P from I's trampoline -> new image, bpf_tramp_image_put(I) bpf_prog_put(P), last ref grace periods, P freed back from S __bpf_prog_enter(P) call P->bpf_func If S and P are fexit progs the task is already past the patched jump, and fentry only images don't have one. On x86 this is an int3 in poisoned bpf_prog_pack memory: Oops: int3: 0000 [#1] SMP NOPTI CPU: 18 UID: 0 PID: 94573 Comm: x169 Not tainted 6.18.44 #1 PREEMPT(lazy) RIP: 0010:0xffffffffc0601d8d Call Trace: ? bpf_trampoline_6442515411+0x1a4/0x21b bpf_lsm_bprm_committed_creds+0x5/0x10 security_bprm_committed_creds+0x5f/0x70 begin_new_exec+0x2d6/0x410 ... We hit this in production when progs attached through trampolines got detached while their hooks were busy, and it was independently found with a fuzzer and KASAN. Have the JITs emit a patchable nop in front of each prog call sequence and record it in the image. When a prog is detached, patch its nop to a jump over the call sequence. Progs that stay attached keep running for the tasks that are in the image, and ip_after_call isn't needed anymore. The task can be in any image that isn't freed yet, not only in the current one. It sleeps in image I1 that calls S, P and Q, then P is detached and the trampoline moves to image I2, then Q is detached and its call is still in I1. So the trampoline keeps a list of its images until they are freed, and detaching a prog patches its nop in all of them. Images hold a reference on the trampoline for that long. On riscv and loongarch a jump of any range takes several instructions, and a task preempted in the middle of them could resume into half of the new sequence. The nop is a single instruction there, patched to a near branch through arch_bpf_trampoline_skip(). With the extra nops, BPF_MAX_TRAMP_LINKS progs no longer fit in a page on x86 and arm64 (and already didn't on powerpc), so lower the limit there like s390 does. Fixes: e21aa341785c ("bpf: Fix fexit trampoline.") Reported-by: Sechang Lim Suggested-by: Alexei Starovoitov Signed-off-by: Florent Revest (Anthropic) Signed-off-by: Alexei Starovoitov Link: https://patch.msgid.link/20260926135605.1217928-3-florent.revest@linux.dev Closes: https://lore.kernel.org/bpf/20260815071927.147049-1-zirajs7@gmail.com/

bpf: Wait for an RCU tasks grace period before freeing trampoline progs

2026-09-29T05:54:16Z

When a prog is detached from a trampoline, it is freed after an RCU grace period, or an RCU tasks trace one if it is sleepable. This covers the tasks that are running the prog, since the prog's enter helper takes the matching read lock before the prog is called. On a preemptible kernel, it doesn't cover a task that was preempted in the trampoline just before the enter helper. That task holds no lock yet, and it calls the prog after it was freed: BUG: KASAN: vmalloc-out-of-bounds in __bpf_prog_enter_recur+0x3a5/0x3f0 Read of size 8 at addr ffffc90000055040 by task candidate/110 CPU: 1 UID: 0 PID: 110 Comm: candidate Not tainted 7.3.0-rc2-00014-g15071f2a1263-dirty #2 PREEMPT(full) Call Trace: __bpf_prog_enter_recur+0x3a5/0x3f0 bpf_trampoline_6442509193+0x37/0xf1 __x64_sys_futex+0x9/0x410 do_syscall_64+0xb0/0x530 ... Wait for an RCU tasks grace period before the existing one when freeing a prog that was linked to a trampoline. An RCU tasks grace period only ends once the tasks that were preempted have run again, and bpf_tramp_image_put() already relies on it to free the image. It doesn't wait for tasks that sleep in the trampoline, the next commit takes care of those. Only progs that were linked to a trampoline can be called this way, so bpf_trampoline_add_prog() marks them and other progs are still freed as before. Fixes: e21aa341785c ("bpf: Fix fexit trampoline.") Reported-by: Junseo Lim Signed-off-by: Florent Revest (Anthropic) Signed-off-by: Alexei Starovoitov Link: https://patch.msgid.link/20260926135605.1217928-2-florent.revest@linux.dev Closes: https://lore.kernel.org/bpf/aqdrwVpanH3WGurX@omen-arch/

bpf: Hold map BTF for the memory allocator destructor record

2026-09-25T17:29:04Z

bpf_ma_set_dtor() duplicates map->record so that the bpf_mem_alloc destructor can release the special fields of hash and rhash map elements once the allocator frees them for good. btf_record_dup() only acquires references on kernel and module BTF. Fields whose types live in the map BTF keep pointing into it: a kptr to a local type refers to map->btf, and a list_head or rb_root field carries a value_rec owned by the struct meta table of map->btf. That borrowed state can outlive the map. When RCU callbacks are still in flight, bpf_mem_alloc_destroy() copies the allocator and defers the final drain, together with the destructor context, to a workqueue. bpf_map_free() then frees map->record and drops the map's BTF reference as soon as map_free() returns. Once the program that loaded the BTF is gone as well, the BTF is freed while the worker still uses the duplicate. The worker dereferences it through btf_is_kernel() and btf_find_struct_meta() in bpf_obj_free_fields() when destroying elements that were freed under RCU, and through btf_is_kernel() in btf_record_free() when releasing the context itself: BUG: KASAN: slab-use-after-free in btf_is_kernel+0x19/0x30 Read of size 1 at addr ffff88811505dce8 by task kworker/u16:2/50 Workqueue: events_unbound free_mem_alloc_deferred Call Trace: btf_is_kernel+0x19/0x30 btf_record_free+0xc7/0xf0 htab_dtor_ctx_free+0x1a/0x30 free_mem_alloc_no_barrier+0x39/0x230 free_mem_alloc_deferred+0x2a/0x40 process_scheduled_works+0x5ce/0x9d0 worker_thread+0x42a/0x5e0 kthread+0x1f5/0x230 ret_from_fork+0x1dd/0x390 ret_from_fork_asm+0x1a/0x30 Allocated by task 358: __kmalloc_cache_noprof+0x287/0x510 btf_new_fd+0xf7/0x3d0 __sys_bpf+0x487/0x840 Freed by task 0: kfree+0x186/0x560 rcu_core+0x6f6/0xd40 Freeing a map that still holds more elements than the allocator's high watermark is enough to get there, because the bulk free during teardown queues the RCU callback that makes bpf_mem_alloc_destroy() defer. Non-preallocated hash maps and resizable hash maps both register the destructor and are affected alike. Keep one reference on the map BTF in the destructor context. Every non-kernel pointer in the duplicated record is either map->btf or memory owned by it, so a single reference covers kptrs, list heads, and rb roots, and mirrors what bpf_map_meta_alloc() does for the duplicated record of an inner map. Release it only after the duplicated record has been freed: the worker is preemptible and not in an RCU read-side critical section, so dropping the last reference first would let the BTF be freed while btf_record_free() still reads its fields. Fixes: 1df97a7453ee ("bpf: Register dtor for freeing special fields") Reported-by: Yuan Chen Signed-off-by: Kumar Kartikeya Dwivedi Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20260901062845.1379760-3-chenyuan_fl@163.com Link: https://patch.msgid.link/20260925064652.2261885-1-memxor@gmail.com

bpf: Fix overflow of jump offset in constant blinding

2026-09-25T02:43:27Z

bpf_jit_blind_insn() rewrites 'if rX op imm goto off' into: AX = imm ^ rnd AX ^= rnd if rX op AX goto off and subtracts 2 from off when the jump is backward. off is s16, so -32767 and -32768 wrap and the jump goes 32767 insns forward, past the end of the prog. The jump is inside the patch, hence bpf_adj_branches() doesn't check it. The kernel crashes when such prog runs with bpf_jit_harden=2. Return -ERANGE in such case. bpf_jit_blind_constants() fails the same way when blinding of other insns moves a jump out of range. Fixes: 4f3446bb809f ("bpf: add generic constant blinding for use in jits") Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20260924182353.277516-1-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element

2026-09-24T14:24:51Z

pcpu_init_value() initializes the per-cpu area of a newly created [lru_]percpu_hash element. The area is recycled, so when the value comes from a BPF program (onallcpus == false) it writes the running CPU's slot and zeroes the rest. bpf_percpu_hash_update() passes onallcpus == true, which delegates to pcpu_copy_value(). pcpu_copy_value() writes only the CPU named in map_flags when BPF_F_CPU is set, so on the create path the other slots keep the recycled element's values: update(k1, 0xdeadc0de, BPF_F_ALL_CPUS) every CPU holds 0xdeadc0de delete(k1) element back on the freelist update(k2, 0xc0ffee, BPF_F_CPU | 0) creates, writes CPU 0 only lookup(k2) CPU 0 0xc0ffee, rest 0xdeadc0de Zero-fill the other CPUs on that arm too. Fixes: c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps") Signed-off-by: Donggeun Yoo Signed-off-by: Alexei Starovoitov Link: https://patch.msgid.link/20260924102321.2120434-2-donggeunyoo.kernel@gmail.com

bpf: Reject dev-bound-only programs on other devices

2026-09-23T02:19:02Z

__bpf_offload_dev_match() falls back to comparing offdev pointers after an exact netdev mismatch. Bound-only programs normally have NULL offdevs, so unrelated netdevs compare equal. A bound-only program on an offload-registered netdev can instead inherit a real offdev and match a sibling port. With CAP_BPF and CAP_NET_ADMIN, a caller can use bpf(BPF_LINK_CREATE) with a different target ifindex to run metadata kfuncs specialized for the bound driver on the target driver's xdp_buff. Running a veth-bound program on tun reads beyond tun's bare stack xdp_buff as a veth_xdp_buff. Oops: general protection fault, probably for non-canonical address KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017] RIP: 0010:veth_xdp_rx_timestamp (drivers/net/veth.c:1673) Call Trace: ... tun_build_skb (drivers/net/tun.c:1739) tun_get_user (drivers/net/tun.c:1856) tun_chr_write_iter (drivers/net/tun.c:2091) vfs_write (fs/read_write.c:595 fs/read_write.c:687) ksys_write (fs/read_write.c:739) do_syscall_64 (arch/x86/entry/syscall_64.c:84) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121) Kernel panic - not syncing: Fatal exception in interrupt Restrict non-offloaded programs to exact netdev matches and retain the shared-offdev fallback only for genuinely offloaded multi-port programs. Fixes: 2b3486bc2d23 ("bpf: Introduce device-bound XDP programs") Reported-by: Signed-off-by: Weiming Shi Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20260917161335.1020405-2-bestswngs@gmail.com/ Link: https://patch.msgid.link/20260920132303.4109240-3-bestswngs@gmail.com