| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Move NMI nesting tracking from the preempt_count bits to a separate
per-CPU counter (nmi_nesting). This is to free up the NMI bits in the
preempt_count, allowing those bits to be repurposed for other uses.
Reduce NMI_BITS from 4 to 1, using it only to detect if we're in an NMI.
The per-CPU counter currently caps nesting at 15.
[boqun: Address Steven Rostedt's comment on the BUG_ON() condition]
[boqun: Use preempt_count_set() in __nmi_exit() to avoid underflow]
Suggested-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260121223933.1568682-3-lyude@redhat.com
Link: https://patch.msgid.link/20260804161447.84806-2-boqun@kernel.org
|
|
Exercise fentry, fexit, and freplace programs that target a struct_ops
callback with an arena context argument. Verify each load is rejected with
-EOPNOTSUPP and the arena-specific verifier diagnostic.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260808003938.3486067-15-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Add a test_arena_stack member with eight leading scalar arguments so the
arena pointer is passed on the stack.
The callback validates the first and last scalar ctx slots before
dereferencing the pointer in ctx[8]. This exercises the indirect
trampoline stack layout and arena conversion together, and prevents a
regression where stack arguments are read one slot late.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Tested-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260808003938.3486067-13-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Add test_arena and test_arena_nullable members to bpf_testmod_ops3 with
arena-tagged stub arguments and kfuncs that forward a caller-provided
pointer to them. The kfuncs take arena-tagged arguments, so each round
trip exercises both conversion directions end to end: the kfunc receives
a kernel arena address and the trampoline converts it back to an arena
pointer for the callback.
The non-nullable callback dereferences its argument with no NULL branch
and captures the raw ctx value, which the trigger program compares
against the arena offset of the passed object, pinning the exact
(u32)(kaddr - kern_vm_start) conversion. The nullable callback verifies
that only a true kernel NULL arrives as NULL. Failure coverage: a
program with no arena is rejected when it loads. The tests run on x86-64
and skip elsewhere, as the programs fail verification where the JIT
lacks arena argument support.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260808003938.3486067-11-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Pin the exact rebase sequences the JITs emit for __arena and
__arena__nullable kfunc arguments with __jited assertions on x86-64: the
unconditional truncate-and-add, the nullable test-and-skip variant, and
all five argument registers in one call, which also covers the
REX-prefixed encoding of r8 on x86. The capture kfuncs take the argument
without dereferencing, so only the emitted code is under test. The
tests skip without LLVM disassembler support.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260808003938.3486067-10-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Add arena-argument kfuncs to bpf_testmod, which also exercises the
argument rebasing on module kfuncs, and tests covering the accepted
argument forms (arena pointer, low 32 bits as a scalar, full user
address as a scalar), the exact rebase semantics via capture kfuncs
returning the raw argument (zero low 32 bits arrive as the arena kernel
base under __arena and as NULL under __arena__nullable), five arena
arguments in one call, a mixed __arena plus __arena__nullable call
exercising both bitmasks on one call site, a kernel-side dereference of
an unpopulated page recovering through the scratch page, and the
rejections (no arena in the program, incompatible register type).
The tests run on x86-64 and skip elsewhere, as programs with
arena-tagged kfunc args fail verification where the JIT lacks support.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260808003938.3486067-9-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Includes the following test cases:
- a test showing that zero extension flags do not propagate through
state pruning in the unpatched kernel.
- a 32-bit subregister consumed by MOV32 and ALU32 operations
(never zext'ed);
- a 64-bit MOV (never zext'ed);
- a narrow (32-bit) BPF_LDX load whose result is read as 64-bit;
- 32-bit atomic fetch_add and cmpxchg whose result is read as 64-bit;
- a CFG case where a 32-bit definition's upper half is used only on one
of two branches;
- no zext for dead registers;
- LD_ABS defines only lower 32 bits, hence needs zext when the result
is used as 64-bits;
- helper, kfunc and subprogram parameters are considered to use full
64 bits;
- a 32-bit subregister consumed by JMP32 (X/K) operations;
- a 32-bit subregister consumed by JMP (X/K) operations;
- a 64-bit register consumed by both JMP and JMP32 operations
(never zext'ed);
- ALU64 and address space cast operations on arena pointers;
- memory loads using BPF_PROBE_MEM instructions.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260807-static-zext-v4-7-b6c270013c77@gmail.com
|
|
At the moment there are more callsites that want bpf_verbose_insn() to
not print a newline after the instruction, than callsites that want a
newline. Drop '\n' from disasm.c. Non-functional change.
The changes in bpftool are verified by writing a bpf program using a
variety of instructions and comparing `prog dump xlated` output in the
following modes: plain, opcodes, visual, visual opcodes. The output
before and after the changes is identical.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Quentin Monnet <qmo@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260807-static-zext-v4-1-b6c270013c77@gmail.com
|
|
Add a new devmem test case for binding the dmabuf with rx-page-size=16K.
The test sweeps RX payload sizes straddling the niov boundary to cover
the sub-niov, exact-niov, and multi-niov RX paths.
Silence pylint invalid-name (`with open() as f`) and too-many-arguments
(ncdevmem_rx grew to 6 args) at file scope.
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
Link: https://patch.msgid.link/20260805-tcpdm-large-niovs-v8-3-3e0225e2808c@meta.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add -b <bytes> to request a non-default niov size via
NETDEV_A_DMABUF_RX_PAGE_SIZE. When the value exceeds PAGE_SIZE,
udmabuf_alloc() switches to an MFD_HUGETLB-backed memfd so each 2 MB
hugepage produces one naturally-aligned sg entry.
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
Link: https://patch.msgid.link/20260805-tcpdm-large-niovs-v8-2-3e0225e2808c@meta.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Every devmem dmabuf binding today hands the page_pool PAGE_SIZE niovs.
This caps a single RX descriptor at PAGE_SIZE, burning CPU on buffer
churn for large flows.
Add a bind-time netlink attribute, NETDEV_A_DMABUF_RX_PAGE_SIZE, that
lets userspace request a larger niov size. The value must be a power of
two >= PAGE_SIZE.
The TX path is changed to always pass PAGE_SIZE.
Measurements:
Setup: kperf in devmem RX/TX cuda mode, 4 flows, 64 MB messages, 60s,
dctcp, num-rx-queues=4, dmabuf-rx/tx-size-mb=2048, 10 runs per niov
size, mlx5.
CPU Util:
niov net sirq % net idle % app sys % app idle %
----- ---------------- ---------------- ---------------- ----------------
4K 62.38 +/- 8.27 33.40 +/- 7.51 54.15 +/- 10.23 43.67 +/- 10.53
16K 58.91 +/- 5.35 35.23 +/- 5.88 41.05 +/- 8.87 56.42 +/- 9.24
32K 64.12 +/- 0.68 31.09 +/- 1.48 44.54 +/- 3.51 52.63 +/- 3.65
64K 54.69 +/- 5.54 39.67 +/- 5.81 35.47 +/- 3.11 61.97 +/- 3.27
RX app sys % drops ~19% from 4K to 64K.
Throughput:
niov RX dev Gbps RX flow avg Gbps
----- ---------------- -----------------
4K 300.63 +/- 53.21 75.16 +/- 13.30
16K 321.35 +/- 28.20 80.34 +/- 7.05
32K 347.63 +/- 2.20 86.91 +/- 0.55
64K 332.11 +/- 14.26 83.03 +/- 3.56
Throughput seems to increase, but the stdev is pretty wide so could just
be noise.
kperf support (not yet merged):
https://github.com/facebookexperimental/kperf/commit/8837577f920876bce6986ec18869ac04439ebcd2
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
Link: https://patch.msgid.link/20260805-tcpdm-large-niovs-v8-1-3e0225e2808c@meta.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
ovs_packet spec needs to fetch the right uAPI header,
like other ovs specs already do. Otherwise build breaks
on very old distros (Ubuntu 22.04).
Spec was added by commit b82bfddc46e2 ("netlink: specs: add OVS packet
family specification").
Reviewed-by: Hangbin Liu <liuhangbin@kylinos.cn>
Link: https://patch.msgid.link/20260807001918.61957-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add the IFF_BACKPRESSURE flag to the UAPI header and to its tools/ copy.
The flag has no effect yet, it is the opt-in switch for the qdisc
backpressure logic added by the following patches.
It is added to TUN_FEATURES only in the last patch of the series, once the
implementation is complete. Until then TUNSETIFF silently masks it off, as
it does for any flag outside TUN_FEATURES.
Keeping the flag and its users in separate patches would either leave a
window where backpressure is unconditional, or make the opt-in a later
add-on. Adding the flag first lets every following patch be a no-op
unless it is set.
Signed-off-by: Simon Schippers <simon.schippers@tu-dortmund.de>
Link: https://patch.msgid.link/20260803183641.96882-2-simon.schippers@tu-dortmund.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Exercise cleanup of nested nodes after deleting their last queue leaf. The
test builds a two-level node hierarchy and checks that removing the queue
also removes both now-empty node shapers.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-15-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add coverage for grouping leaves that currently belong to different parent
nodes. The test verifies that an implicit parent is rejected, an explicit
parent succeeds, and the old empty parent nodes are cleaned up.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-14-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Verify that invalid set requests fail without corrupting existing queue
shaper state. The test covers invalid node creation through set and invalid
queue identifiers, then confirms the original queue configuration remains
unchanged.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-13-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Exercise queue-scope scalar shaper attributes reported by the device,
including rate limits, burst, priority and weight. Build the set request
from advertised capabilities so devices are tested for the attributes they
claim rather than skipped for missing unrelated fields.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-12-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add reject_reparenting to verify that the group operation rejects attempts
to change an existing node's parent. The test creates two node shapers
under netdev and verifies that re-grouping the first node under the second
fails with EOPNOTSUPP. It also verifies that updating the node with the
same parent succeeds, and that updating the node without specifying a
parent keeps the queue leaves under the original node while updating their
weights.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-11-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add move_queue_between_nodes to verify that a queue can be moved
from one node to another via re-grouping. Creates N1 with Q1,Q2
and N2 with Q3, then re-groups N2 with Q1,Q3 to steal Q1 from
N1. Verifies Q1 moved to N2 and Q2 remains under N1.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-10-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add delete_child_reparent to verify that deleting a child node
reparents its queue leaves to the parent node. Creates a two-level
hierarchy (N1 with Q1,Q2 and child N2 with Q3), deletes N2, and
verifies Q3's parent becomes N1.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-9-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add nested_depth_limit to incrementally create deeper nesting
levels until the driver rejects. Reports the maximum supported
nesting depth on both pass and fail. A device advertising nesting
support must support at least depth 2, otherwise nesting is
meaningless.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-8-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add group_update_rate to test updating an existing node's rate
via the .group callback. Creates a node with bw_max=10000,
re-groups with bw_max=50000, and verifies the rate changed while
leaves remain under the same node.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-7-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add set_node_shaper to test updating a NODE scope shaper's rate
via the .set callback. Creates a node group with bw_max=10000,
updates to 20000 via .set, and verifies the change.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-6-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add a test that groups queues under the netdev parent with rate
limiting enabled. Extract the common group-under-netdev flow into
_group_under_netdev helper to share with basic_groups.
The test independently checks for netdev scope bw_max and metric
capabilities before proceeding, and verifies that the netdev
shaper persists after leaf deletion.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-5-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Decouple basic_groups from the set_nshapers test dependency. The
test was gated on cfg.netdev which is set by set_nshapers. Replace
with direct capability checks: netdev scope support (required for
grouping under netdev handle) and queue scope nesting + weight.
Remove bw-max and metric from the .group call so the test validates
pure queue grouping without rate limiting. The rate-limited variant is
restored in the following patch, which adds a dedicated
basic_groups_with_rate test.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-4-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
dup_leaves expects the kernel to reject a group request that lists the same
queue twice. When that rejection does not happen, ksft_raises only records
a failed check and leaves cm.exception as None, so the following errno
check raises AttributeError. Worse, the accepted group request leaves a
node shaper and queue 0 behind, which makes later tests fail for an
unrelated reason. Handle the negative test explicitly instead. If group
fails, verify that the errno is EINVAL and return. If group succeeds,
delete the node returned by the operation and queue 0 before reporting the
failure.
Give the duplicate leaves different weights so the request still contains
two distinct leaf entries while exercising duplicate handle validation.
This also introduces _delete_shaper(), cached _cap_get(), and
_require_caps() helpers as preparation for the following shaper group
tests. The follow-on tests need the same capability checks for node and
queue scope support. Keeping that logic in one place avoids repeating raw
EOPNOTSUPP handling in each test.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-3-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Commit 57bb59ab6fa3 ("selftests: net: bump default cmd() timeout to 20
seconds") raised the default cmd() timeout to 20 seconds, so the explicit
timeout=10 passed to the ethtool channel commands in queue_update() is
now redundant and, in fact, shorter than the default. Drop it and rely
on the default timeout.
Signed-off-by: Mohsin Bashir <hmohsin@meta.com>
Link: https://patch.msgid.link/20260805030936.1092907-2-mohsin.bashr@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Cross-merge BPF and other fixes after downstream PR.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
|
|
Merge updates related to system sleep handling and runtime PM, a power
capping update, and power management tools updates for 7.3-rc1:
- Rename the PM core module parameter prefix to "pm" and allow the PM
transition (DPM) watchdog to be disabled by default (Tzung-Bi Shih)
- Fix off-by-one in wakelocks number limit check in the system sleep
sysfs interface (Haowen Tu)
- Remove kernel-doc markings from helper descriptions in the core
hibernation code (Adi Nata)
- Use %pe to print error pointer values in the hibernation core (Ronan
Marchal)
- Fix memory leak in snapshot_write_next() error path (Malaya Kumar
Rout)
- Delay allocating and linking the next swap_map_page in the
hibernation image saving code until another image page actually
needs to be recorded (Haesung Kim)
- Stop setting runtime_error on runtime resume callback failures to
allow drivers to recover from resume issues (Praveen Talari)
- Handle PMU registration failure during probe in the intel_rapl_tpmi
driver (Sumeet Pawnikar)
- Avoid optional imports in intel_pstate_tracer unless they are really
needed (Yousef Alhouseen)
- Add generic CPPC performance display to the cpupower utility, build
and call CPPC information on non-AMD processors, make cpupower print
kernel and hardware frequency information, and add libm to cpupower
for generic CPPC view (Jeremy Linton)
- Remove conditional return with no effect from cpupower (Sang-Heon
Jeon)
* pm-sleep:
PM: sleep: Allow disabling DPM watchdog by default
PM: sleep: Rename module parameters prefix to "pm"
PM: hibernate: swap: defer linking the next map page
PM: hibernate: Fix memory leak in snapshot_write_next() error path
PM: hibernate: Use %pe to print error pointer values
PM: hibernate: Remove kernel-doc markings from helper descriptions
PM: sleep: Fix off-by-one in wakelocks number limit check
* pm-runtime:
PM: runtime: Only set runtime_error on suspend callback failures
* pm-powercap:
powercap: intel_rapl_tpmi: Handle PMU registration failure during probe
* pm-tools:
cpupower: remove conditional return with no effect
tools/power: intel_pstate_tracer: avoid optional imports for help
cpupower: Add libm to cpupower for generic CPPC view
cpupower: Print kernel and hardware frequency information
cpupower: Build and call CPPC information on non-AMD processors
cpupower: Add generic CPPC performance display
|
|
Merge updates of the thermal core, thermal drivers for Intel platforms
and the thermometer utility for 7.3-rc1:
- Use sysfs_emit_at() in trans_table_show() (Thorsten Blum)
- Remove hwmon class devices created for thermal zones when the thermal
zone devices holding them are removed (Rafael Wysocki)
- Add support for the Directed Package-level Thermal Interrupt to
the Intel thermal throttling driver to allow package-level thermal
interrupts to go to one specific CPU in a processor package instead
of going to all of the CPUs in it (Ricardo Neri)
- Clean up RFIM groups on DVFS failure and clean up ODVP on probe
failures in the int340x thermal driver (Pengpeng Hou)
- Remove redundant dev_err() from the int340x thermal driver and the
bxt_pmic driver (Pan Chuang)
- Simplify ptc_temperature_write() in the int340x thermal driver by
using kstrtou32_from_user() (Dmitry Antipov)
- Close fd on realloc() failure in the thermometer utility (Amarjeet)
* thermal-core:
thermal: sysfs: Use sysfs_emit_at() in trans_table_show()
* thermal-hwmon:
thermal: hwmon: Remove hwmon class device along with its parent
* thermal-intel:
thermal: intel: int340x: simplify ptc_temperature_write()
thermal: intel: bxt_pmic: Remove redundant dev_err()
thermal: intel: int340x: Remove redundant dev_err()
thermal: intel: int3400: clean up ODVP on probe failures
thermal: intel: int340x: clean up RFIM groups on DVFS failure
thermal: intel: Add a syscore shutdown callback for kexec reboot
thermal: intel: Add syscore callbacks for suspend and resume
thermal: intel: Enable the Directed Package-level Thermal Interrupt
thermal: intel: Add resources to handle directed package-level thermal interrupts
x86/thermal: Add bit definitions for Intel Directed Package Thermal Interrupt
* thermal-tools:
tools/thermal/thermometer: close fd on realloc() failure
|
|
Merge cpufreq updates for 7.3-rc1:
- Minor fixes and cleanups in assorted cpufreq drivers (Dan Carpenter,
Guru Das Srinagesh, Haoxiang Li, Karl Mehltretter, Sasha Finkelstein,
and Pan Chuang)
- Fix cpufreq table creation and bios_limits() callback in the Rust
bindings (Priya Bala Govindasamy)
- Add IPQ5210 support to qcom-nvmem driver (Varadarajan Narayanan)
- Adjust the .adjust_perf() cpufreq driver callback to allow the
maximum performance value to be passed to drivers and update the
intel_pstate driver to use it (Rafael Wysocki)
- Set policy->cur to the actual requested frequency in the intel_pstate
driver when the performance policy is used (Rafael Wysocki)
- Simplify HWP handling on Broadwell processors in intel_pstate (Rafael
Wysocki)
- Fix setting minimum P-state at init time in intel_pstate (Rafael
Wysocki)
- Consolidate frequency values computation in intel_pstate and clean up
code in that driver (Rafael Wysocki)
- Add missing kernel-doc desciptions for structure and union members in
the amd-pstate driver (David Vernet)
- Handle missing policy in dynamic EPP callbacks in the amd-pstate
driver (EDAMAMEX)
- Introduce EXPORT_SYMBOL_FOR_PSTATE_UT() to export amd-pstate driver
symbols to the amd-pstate-ut subdriver (K Prateek Nayak)
- Add dynamic EPP as an "energy_performance_preference" mode in
amd-pstate, remove the "amd_dynamic_epp" kernel command line option
and the "dynamic_epp" sysfs attribute, and update the dynamic_epp
documentation accordingly (K Prateek Nayak)
- Add unit tests for CPPC Performance Priority and the "dynamic" EPP
mode in the amd-pstate driver (K Prateek Nayak)
- Set min_limit_freq based on bios_min_perf in amd-pstate and remove
the defensive check for bios_min_perf from it (K Prateek Nayak)
- Fix EPP return type and handle errors in amd-pstate during
initialization, toggle auto_sel in active mode on shared memory
systems, and cache the firmware programmed EPP value (Marco Scardovi)
- Skip tests in amd-pstate-ut if the amd-pstate driver is not in active
use (Qianheng Peng)
- Replace sprintf() with sysfs_emit() in sysfs show in the cpufreq
schedutil governor and fix a self-contradictory comment in
sugov_iowait_apply() (Zhongqiu Han)
- Fix the usage example for the sampling_rate tunable of the ondemand
cpufreq governor in admin-guide (wangxiaodong)
* pm-cpufreq: (40 commits)
cpufreq: imx6q: fix out-of-bounds write when probed more than once
cpufreq: imx6q: fix devres accumulation across driver rebind
rust: cpufreq: Fix temporary write in Registration::bios_limit_callback
rust: cpufreq: Add CPUFREQ_TABLE_END as last table entry in TableBuilder::to_table
cpufreq: intel_pstate: Adjust policy->cur in active mode to policy
cpufreq/amd-pstate: Document missing kernel-doc members
cpufreq/amd-pstate-ut: Add unit test for CPPC Performance Priority
cpufreq/amd-pstate-ut: Add unit test for "dynamic" EPP mode
cpufreq/amd-pstate: Reduce the scope of exported symbols
Documentation/amd-pstate: Update dynamic_epp documentation with new behavior
cpufreq/amd-pstate: Remove "amd_dynamic_epp" cmdline and "dynamic_epp" sysfs
cpufreq/amd-pstate: Add dynamic EPP as an "energy_performance_preference" mode
cpufreq/amd-pstate: Extract platform profile to EPP conversion into a helper
cpufreq/amd-pstate: Remove the defensive check for bios_min_perf
cpufreq/amd-pstate: Set min_limit_freq based on bios_min_perf
cpufreq: apple-soc: Calculate frequency as a 64-bit value
kselftest: cpufreq: Backup and restore governor for sptests
selftests/cpufreq: Remove unnecessary sudo from quick_shuffle()
selftests/cpufreq: Remove unused local variables from switch_show_governor()
cpufreq/amd-pstate: handle missing policy in dynamic EPP callbacks
...
|
|
While 'perf sched latency' reports task runtime and delay statistics
(average and maximum delay), it does not provide a visual representation
of how task wait times are distributed across latency ranges between
snapshots (start and finish of the analysis window).
The --histogram option collects CPU wait latencies (time between when
a task becomes runnable and when it gets scheduled onto a CPU) into 22
latency buckets, displaying an ASCII bar chart distribution.
The --hist-mode option configures the bucketing scheme:
- log (default). Logarithmic latency buckets ranging from
sub-microsecond (< 1 us) up to >= 1.05 seconds
- linear. Equal-width linear latency buckets
(i.e., 100 us steps up to >= 2.1 ms)
The --time option allows filtering trace event processing to a
specific time interval [start,stop].
Example histogram output excerpt:
❯ sudo perf sched latency --histogram --CPU 0
CPU Wait Latency Distribution Histogram (between snapshots) (total samples: 36114)
-------------------------------------------------------------------
Latency Range | Count | Pct | Histogram Graph
-------------------------------------------------------------------
< 1 us | 17 | 0.0% | #
2 - 4 us | 673 | 1.9% | #
4 - 8 us | 6237 | 17.3% | ######
8 - 16 us | 3224 | 8.9% | ###
16 - 32 us | 1388 | 3.8% | #
32 - 64 us | 709 | 2.0% | #
64 - 128 us | 690 | 1.9% | #
128 - 256 us | 789 | 2.2% | #
256 - 512 us | 541 | 1.5% | #
512 - 1024 us | 2256 | 6.2% | ##
1 - 2 ms | 3577 | 9.9% | ###
2 - 4 ms | 13259 | 36.7% | ##############
4 - 8 ms | 2523 | 7.0% | ##
8 - 16 ms | 222 | 0.6% | #
16 - 32 ms | 10 | 0.0% | #
>= 1.05 s | 3 | 0.0% | #
-------------------------------------------------------------------
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Currently, 'perf sched latency' displays task runtime and delay values
exclusively in milliseconds (ms). This can be hard to read when
latencies are very small (in the microsecond or nanosecond range) or
unusually large (seconds).
Introduce auto-scaling for latency and runtime display columns. Values
are dynamically scaled and output with the most appropriate unit:
nanoseconds (ns), microseconds (us), milliseconds (ms), or seconds (s).
Additionally, rename column headers from "Runtime ms", "Avg delay ms",
and "Max delay ms" to "Runtime", "Avg delay", and "Max delay"
respectively, adjust spacing to maintain column alignment and stripe
redundant prefix strings from each row's format string to produce a
clean, tabular output.
For illustrative purposes, a comparison of the latency table header
before and after this change is shown below:
Before:
-------------------------------------------------------------------------------------------------------------------------------------------
Task | Runtime ms | Count | Avg delay ms | Max delay ms | Max delay start | Max delay end |
-------------------------------------------------------------------------------------------------------------------------------------------
kworker/2:2-mm_:154757 | 0.033 ms | 1 | avg: 0.829 ms | max: 0.829 ms | max start: 169486.543205 s | max end: 169486.544034 s
After:
------------------------------------------------------------------------------------------------------------------------------------------
Task | Runtime | Count | Avg delay | Max delay | Max delay start | Max delay end |
------------------------------------------------------------------------------------------------------------------------------------------
kworker/2:2-mm_:154757 | 32.873 us | 1 | 829.347 us | 829.347 us | 169486.543205 s | 169486.544034 s |
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
For pipe mode streams, event attributes are received dynamically during
event processing, meaning session->evlist is not populated prior to
perf_session__process_events(). To handle pipe input correctly:
- Register the missing .attr, .tracing_data, .build_id, and .feature
callbacks in cmd_sched()
- Promote the handlers array to file-scope (latency_handlers[]) and
dynamically assign matching tracepoint handlers
(or process_sched_ignore) inside
perf_sched__process_tracepoint_sample() when evsel->handler is NULL;
replace process_sched_wakeup_ignore() with process_sched_ignore()
- Perform the trace check post-processing when handling pipe data
Fixes: 27295592c22e ("perf session: Share the common trace sample_check routine as perf_session__has_traces")
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
When 'perf sched latency' is executed on a perf.data file that lacks
tracepoint samples (i.e., a file recorded without the -R flag or
containing only non-tracepoint events), perf_session__has_traces()
correctly outputs an error message. However, perf_sched__read_events()
subsequently falls through and returns 0 (success).
Consequently, caller functions such as perf_sched__lat() assume event
processing succeeded and proceed to render empty latency header tables
and total summary statistics.
Fix this behaviour by ensuring perf_sched__read_events() aborts early and
returns a suitable error code when perf_session__has_traces() evaluates
to false.
Additionally, validate thread__get_runtime() against NULL in
map_switch_event() to prevent potential null-pointer dereferences.
Fixes: 27295592c22e ("perf session: Share the common trace sample_check routine as perf_session__has_traces")
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Modify bounds and union member access in mmap2 build_id synthesis. Bound
max_filename_len against the minimum of filename array capacity and the
outer union stack layout minus sample ID trailers. This prevents both
-E2BIG overruns and _FORTIFY_SOURCE array bounds aborts on strlcpy even
if the enclosing union expands.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Clamp long DSO names to mmap/mmap2 filename boundaries accounting for
sample ID headers to prevent buffer overruns in
perf_event__synthesize_modules_maps_cb(). Explicitly clear misc flags and
union padding to prevent stale Build-ID state from leaking between module
synthesis events, and cast event buffer pointers to avoid _FORTIFY_SOURCE
array bounds aborts when zeroing padding trailers.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Fix a pre-existing stack buffer overflow bug in
perf_event__synthesize_cgroup() where an in-place null padding loop wrote
bytes past the end of the cgrp_root stack array buffer during cgroup tree
traversal. Eliminate in-place path mutation, use PERF_ALIGN for path_len,
clamp raw_path_len to prevent sample ID header trailer overruns, and use
strlcpy with combined zero padding for alignment and sample ID headers.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Fix critical logic and boundary bugs in read_proc_maps_line() and caller.
Ensure any mid-line hex/dec/char parsing failure invokes io__drain_line()
safely, using a do-while loop to read and discard remaining characters
until a newline or EOF is reached. Clamp pathname extraction size to
account for trailing sample ID headers, use standard '//toolong' fallback
literal for over-length pathnames, emit timeout flags for truncated entries
securely via goto out;, and cast event buffer pointers to avoid
_FORTIFY_SOURCE array bounds aborts across synthesis handlers.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Use getline() to dynamically allocate the required line buffer for maps
parsing, guaranteeing bounds safety and avoiding compiler warnings
by evaluating the return value in the loop condition directly.
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
skip() ignores do_read()'s return value and unconditionally
subtracts the requested chunk size from 'size' on every iteration.
This was previously bounded by size being 'int': a maliciously
large 64-bit value was truncated on assignment, capping the loop
early by accident.
Now that size is size_t, a crafted file supplying a very large
size causes skip() to keep requesting BUFSIZ-sized reads and
subtracting BUFSIZ from size regardless of whether do_read()
actually succeeds, spinning indefinitely even after EOF or a read
error.
Check do_read()'s return value and break out of the loop on
failure or EOF, so forward progress is only counted when a read
actually succeeds.
Signed-off-by: Tanushree Shah <tshah@linux.ibm.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Both functions read an attacker-controlled size directly from the
input file and pass size + 1 to malloc() before reading size bytes
into the result:
read_ftrace_printk(): size is an unsigned int from read4(). When
size == UINT_MAX, size + 1 overflows to 0, so malloc(0) returns a
minimal allocation while size itself remains UINT_MAX.
read_saved_cmdline(): size is an unsigned long long from read8().
When size == ULLONG_MAX, size + 1 overflows to 0 the same way.
In both cases, do_read(buf, size) then attempts to read the full,
unwrapped size into the tiny allocated buffer, a heap buffer
overflow.
This was previously masked by do_read()'s size parameter being
'int': passing these values truncated them, which the read()
syscall's own boundary checks rejected before any data was read.
Fixing that truncation (widening do_read() to size_t) is correct
on its own, but it removes this accidental protection and exposes
the pre-existing missing bounds check in both functions.
Reject the one value that causes the overflow before it's used, in
each function.
Signed-off-by: Tanushree Shah <tshah@linux.ibm.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
trace_event__cleanup() frees t->pevent but never clears the
pointer. It can be called twice on the same trace_event: once
from trace_report()'s error path, and again from
perf_session__delete() during session teardown, resulting in a
double free / use-after-free.
Separately, trace_event__init() overwrites t->pevent/t->plugin_list
without releasing any existing handle, leaking memory if it's
called more than once on the same struct. eg. via a perf.data
file with multiple PERF_RECORD_HEADER_TRACING_DATA headers.
Guard against re-entry by returning early if t->pevent is already
NULL, and clear it after cleanup so a repeat call is a safe no-op.
Call trace_event__cleanup() at the start of trace_event__init(),
so a repeated init releases any existing handle before allocating
a new one.
Signed-off-by: Tanushree Shah <tshah@linux.ibm.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
The do_read() and skip() functions use 'int' for size parameters,
truncating 64-bit sizes from callers. This causes two issues:
1. Uninitialized memory dump: do_read() reads fewer bytes than
allocated, leaving uninitialized heap memory that gets written
to output files.
2. Out-of-bounds read: Parsing functions process the full 64-bit
size while only partial data was read into the buffer.
Change do_read(), __do_read(), and skip() to use size_t for size
parameters and ssize_t for return values (where applicable), matching
read()/write() system calls.
Update callers to use ssize_t for storing return values.
Fixes: 4a31e56599d4 ("perf tools: Get rid of read_or_die() in trace-event-read.c")
Signed-off-by: Tanushree Shah <tshah@linux.ibm.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
|
|
Some ftracetest test cases parse command output by matching English
field names. Tools such as readelf may localize their output via
gettext, while ftracetest currently inherits the user locale from
the environment.
If a translated field name is printed, parsing can fail even though
the tested kernel behavior is unchanged. For example, add_remove_uprobe
can fail to find the ELF entry point and then write a uprobe event with
a missing offset.
Export LC_ALL=C in the top-level ftracetest runner so every test case
gets stable command output by default.
Link: https://lore.kernel.org/all/20260807081512.2974757-2-qirui.001@bytedance.com/
Signed-off-by: Rui Qi <qirui.001@bytedance.com>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
The add_remove_uprobe test uses readelf -h to obtain the ELF entry
point (e_entry) and passes it directly as the offset to uprobe_events.
However, uprobe_events expects a file offset, not a virtual address.
For PIE binaries, the virtual address happens to equal the file offset
because the first LOAD segment has p_vaddr == p_offset, so the test
works by coincidence. But for non-PIE executables, e_entry is an
absolute virtual address that can far exceed the file size. When the
probe is enabled, uprobe_register() checks offset > i_size_read(inode)
and rejects it with -EINVAL.
Fix this by converting the virtual address to a file offset using the
ELF program headers: scan readelf -lW output for the LOAD segment
containing the entry point, then compute file_offset = e_entry -
p_vaddr + p_offset. For PIE binaries the result is unchanged; for
non-PIE binaries the offset is correctly translated.
The conversion uses only POSIX shell primitives, with no dependency on
gawk or perl.
Link: https://lore.kernel.org/all/20260807081512.2974757-3-qirui.001@bytedance.com/
Fixes: dc4b165855f2 ("selftests/ftrace: Use readelf to find entry point in uprobe test")
Cc: stable@vger.kernel.org
Signed-off-by: Rui Qi <qirui.001@bytedance.com>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
queued_spin_lock_slowpath() and queued_spin_unlock() are dispatched
through pv_ops_lock via the paravirt-ops ALTERNATIVE machinery, which
picks the target (native inline store / hypervisor call) once at boot
and cannot change at runtime.
Convert both to static_call(). The site becomes a direct call patched in
place (one byte smaller), and on native the unlock still collapses to
the inline "movb $0, (%rdi)" store, so the fast path is unchanged.
Unlike the ALTERNATIVE mechanism, a static_call() target can also be
updated at runtime via static_call_update(). This is a prerequisite for
the contended_release tracepoint, which has to swap in a traced unlock
while the system is running.
[ ilvokhin: commit message; fix PARAVIRT_SPINLOCKS=n build; teach
__static_call_validate() about the inline unlock insn; make the
slowpath site module-safe: static_call_mod() +
EXPORT_STATIC_CALL_TRAMP(); pass @lock to the callee-save unlock,
fixing a boot hang under CALL_DEPTH_TRACKING. Boot tested native + KVM
PV guest. ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Co-developed-by: Dmitry Ilvokhin <d@ilvokhin.com>
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/all/20260603120811.GW3493090@noisy.programming.kicks-ass.net/
Link: https://patch.msgid.link/9a32ae399eb804a02a31af04dcabe7e7ee4f3fdf.1785778551.git.d@ilvokhin.com
|
|
The public libthermal header opens the C++ 'extern "C" {' block inside
the __LIBTHERMAL_H include guard, but places the closing brace after
the guard has already ended:
#endif /* __LIBTHERMAL_H */
#ifdef __cplusplus
}
#endif
On a single inclusion the braces still balance, so the problem is
invisible. On the second inclusion of the header in the same C++
translation unit the include guard skips the opening 'extern "C" {',
while the closing '}' lives outside the guard and is emitted anyway.
This leaves a stray '}' and breaks compilation for any C++ consumer
that includes the header more than once.
Move the closing block inside the include guard so both halves of the
'extern "C"' declaration are guarded consistently.
Signed-off-by: Andreas Haufler <andreas@haufler.info>
Signed-off-by: Daniel Lezcano <daniel.lezcano@kernel.org>
Link: https://patch.msgid.link/20260721083230.91246-1-andreas@haufler.info
|
|
Pull BPF fixes from Daniel Borkmann:
- Fix BPF verifier to preserve full pointer state for commuted
scalar += pointer arithmetic (Yiyang Chen, Eduard Zingerman)
- Fix a use-after-free of request sockets in the BPF TCP iterator
batching (Jose Fernandez)
- Fix a use-after-free of sk_redir in the BPF sockmap send verdict
path (Chengfeng Ye)
- Fix a netns reference imbalance in the BPF conntrack kfuncs
(Chengfeng Ye)
- Fix bpf_get_fsverity_digest() dynptr assumptions and silent
digest truncation (Eric Biggers)
- Fix bpf_tcp_{gen,check}_syncookie to check sk_state before
sk_protocol to make sure it is a full socket (Luxiao Xu)
- Fix rqspinlock to reset the tail when preserving the queue
on deadlock (Kumar Kartikeya Dwivedi)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
rqspinlock: Reset tail when preserving queue on deadlock
bpf: Check sk_state before sk_protocol in bpf_tcp_*_syncookie
fsverity: Fix silent truncation in bpf_get_fsverity_digest()
fsverity: Fix bpf_get_fsverity_digest() dynptr assumptions
bpf: tcp: Fix use-after-free in bpf_iter_tcp_established_batch()
bpf: Fix netns reference imbalance in conntrack kfuncs
bpf, sockmap: Fix sk_redir use-after-free in send verdict
selftests/bpf: Cover commuted pointer state propagation
bpf: Propagate untrusted pointer state in commuted arithmetic
bpf: Preserve pointer state for commuted arithmetic
bpf: Simplify sanitize_err() signature
|
|
Add a verifier test that a BPF_LOAD_ACQ from a rdonly_untrusted_mem pointer
(PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED, obtained via bpf_rdonly_cast())
is rejected. Such a source requires BPF_PROBE_MEM fault protection which
is not applied to atomic loads; without the verifier fix the load is accepted
and would crash the kernel on a fault.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_load_acquire
[...]
#621/1 verifier_load_acquire/load-acquire, 8-bit:OK
#621/2 verifier_load_acquire/load-acquire, 8-bit @unpriv:OK
#621/3 verifier_load_acquire/load-acquire, 16-bit:OK
#621/4 verifier_load_acquire/load-acquire, 16-bit @unpriv:OK
#621/5 verifier_load_acquire/load-acquire, 32-bit:OK
#621/6 verifier_load_acquire/load-acquire, 32-bit @unpriv:OK
#621/7 verifier_load_acquire/load-acquire, 64-bit:OK
#621/8 verifier_load_acquire/load-acquire, 64-bit @unpriv:OK
[...]
#621/19 verifier_load_acquire/load-acquire from rdonly_untrusted_mem pointer:OK
#621/20 verifier_load_acquire/load-acquire with invalid register R15:OK
#621/21 verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK
#621/22 verifier_load_acquire/load-acquire from pkt pointer:OK
#621/23 verifier_load_acquire/load-acquire from flow_keys pointer:OK
#621/24 verifier_load_acquire/load-acquire from sock pointer:OK
#621 verifier_load_acquire:OK
Summary: 1/24 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260806201047.333389-6-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
Joel Fernandes
Kumar Kartikeya Dwivedi
Tejun Heo
Eduard Zingerman
Bobby Eshleman
Jakub Kicinski
Simon Schippers
Mohsin Bashir
Daniel Borkmann
Rafael J. Wysocki
Aaron Tomlin
Ian Rogers
Tanushree Shah
Rui Qi
Peter Zijlstra
Andreas Haufler
Linus Torvalds