linux-dev/tools/bpf/bpftool/Makefile, branch master

bpftool: Complete libbfd feature detection

2022-08-10T13:44:01Z

Commit 6e8ccb4f624a7 ("tools/bpf: properly account for libbfd variations") sets the linking flags depending on which flavor of the libbfd feature was detected. However, the flavors except libbfd cannot be detected, as they are not in the feature list. Complete the list of features to detect by adding libbfd-liberty and libbfd-liberty-z. Committer notes: Adjust conflict with with: 1e1613f64cc8a09d ("tools bpftool: Don't display disassembler-four-args feature test") 600b7b26c07a070d ("tools bpftool: Fix compilation error with new binutils") Fixes: 6e8ccb4f624a73c5 ("tools/bpf: properly account for libbfd variations") Signed-off-by: Roberto Sassu Cc: Alexei Starovoitov Cc: Andres Freund Cc: Andrii Nakryiko Cc: bpf@vger.kernel.org Cc: Daniel Borkmann Cc: Ingo Molnar Cc: John Fastabend Cc: KP Singh Cc: llvm@lists.linux.dev Cc: Martin KaFai Lau Cc: Nathan Chancellor Cc: Nick Desaulniers Cc: Nick Terrell Cc: Peter Zijlstra Cc: Quentin Monnet Cc: Song Liu Cc: Stanislav Fomichev Link: https://lore.kernel.org/r/20220719170555.2576993-2-roberto.sassu@huawei.com Signed-off-by: Arnaldo Carvalho de Melo

Merge tag 'perf-tools-for-v6.0-2022-08-04' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

2022-08-06T16:36:08Z

Pull perf tools updates from Arnaldo Carvalho de Melo: - Introduce 'perf lock contention' subtool, using new lock contention tracepoints and using BPF for in kernel aggregation and then userspace processing using the perf tooling infrastructure for resolving symbols, target specification, etc. Since the new lock contention tracepoints don't provide lock names, get up to 8 stack traces and display the first non-lock function symbol name as a caller: $ perf lock report -F acquired,contended,avg_wait,wait_total Name acquired contended avg wait total wait update_blocked_a... 40 40 3.61 us 144.45 us kernfs_fop_open+... 5 5 3.64 us 18.18 us _nohz_idle_balance 3 3 2.65 us 7.95 us tick_do_update_j... 1 1 6.04 us 6.04 us ep_scan_ready_list 1 1 3.93 us 3.93 us Supports the usual 'perf record' + 'perf report' workflow as well as a BCC/bpftrace like mode where you start the tool and then press control+C to get results: $ sudo perf lock contention -b ^C contended total wait max wait avg wait type caller 42 192.67 us 13.64 us 4.59 us spinlock queue_work_on+0x20 23 85.54 us 10.28 us 3.72 us spinlock worker_thread+0x14a 6 13.92 us 6.51 us 2.32 us mutex kernfs_iop_permission+0x30 3 11.59 us 10.04 us 3.86 us mutex kernfs_dop_revalidate+0x3c 1 7.52 us 7.52 us 7.52 us spinlock kthread+0x115 1 7.24 us 7.24 us 7.24 us rwlock:W sys_epoll_wait+0x148 2 7.08 us 3.99 us 3.54 us spinlock delayed_work_timer_fn+0x1b 1 6.41 us 6.41 us 6.41 us spinlock idle_balance+0xa06 2 2.50 us 1.83 us 1.25 us mutex kernfs_iop_lookup+0x2f 1 1.71 us 1.71 us 1.71 us mutex kernfs_iop_getattr+0x2c ... - Add new 'perf kwork' tool to trace time properties of kernel work (such as softirq, and workqueue), uses eBPF skeletons to collect info in kernel space, aggregating data that then gets processed by the userspace tool, e.g.: # perf kwork report Kwork Name | Cpu | Total Runtime | Count | Max runtime | Max runtime start | Max runtime end | ---------------------------------------------------------------------------------------------------- nvme0q5:130 | 004 | 1.101 ms | 49 | 0.051 ms | 26035.056403 s | 26035.056455 s | amdgpu:162 | 002 | 0.176 ms | 9 | 0.046 ms | 26035.268020 s | 26035.268066 s | nvme0q24:149 | 023 | 0.161 ms | 55 | 0.009 ms | 26035.655280 s | 26035.655288 s | nvme0q20:145 | 019 | 0.090 ms | 33 | 0.014 ms | 26035.939018 s | 26035.939032 s | nvme0q31:156 | 030 | 0.075 ms | 21 | 0.010 ms | 26035.052237 s | 26035.052247 s | nvme0q8:133 | 007 | 0.062 ms | 12 | 0.021 ms | 26035.416840 s | 26035.416861 s | nvme0q6:131 | 005 | 0.054 ms | 22 | 0.010 ms | 26035.199919 s | 26035.199929 s | nvme0q19:144 | 018 | 0.052 ms | 14 | 0.010 ms | 26035.110615 s | 26035.110625 s | nvme0q7:132 | 006 | 0.049 ms | 13 | 0.007 ms | 26035.125180 s | 26035.125187 s | nvme0q18:143 | 017 | 0.033 ms | 14 | 0.007 ms | 26035.169698 s | 26035.169705 s | nvme0q17:142 | 016 | 0.013 ms | 1 | 0.013 ms | 26035.565147 s | 26035.565160 s | enp5s0-rx-0:164 | 006 | 0.004 ms | 4 | 0.002 ms | 26035.928882 s | 26035.928884 s | enp5s0-tx-0:166 | 008 | 0.003 ms | 3 | 0.002 ms | 26035.870923 s | 26035.870925 s | -------------------------------------------------------------------------------------------------------- See commit log messages for more examples with extra options to limit the events time window, etc. - Add support for new AMD IBS (Instruction Based Sampling) features: With the DataSrc extensions, the source of data can be decoded among: - Local L3 or other L1/L2 in CCX. - A peer cache in a near CCX. - Data returned from DRAM. - A peer cache in a far CCX. - DRAM address map with "long latency" bit set. - Data returned from MMIO/Config/PCI/APIC. - Extension Memory (S-Link, GenZ, etc - identified by the CS target and/or address map at DF's choice). - Peer Agent Memory. - Support hardware tracing with Intel PT on guest machines, combining the traces with the ones in the host machine. - Add a "-m" option to 'perf buildid-list' to show kernel and modules build-ids, to display all of the information needed to do external symbolization of kernel stack traces, such as those collected by bpf_get_stackid(). - Add arch TSC frequency information to perf.data file headers. - Handle changes in the binutils disassembler function signatures in perf, bpftool and bpf_jit_disasm (Acked by the bpftool maintainer). - Fix building the perf perl binding with the newest gcc in distros such as fedora rawhide, where some new warnings were breaking the build as perf uses -Werror. - Add 'perf test' entry for branch stack sampling. - Add ARM SPE system wide 'perf test' entry. - Add user space counter reading tests to 'perf test'. - Build with python3 by default, if available. - Add python converter script for the vendor JSON event files. - Update vendor JSON files for most Intel cores. - Add vendor JSON File for Intel meteorlake. - Add Arm Cortex-A78C and X1C JSON vendor event files. - Add workaround to symbol address reading from ELF files without phdr, falling back to the previoous equation. - Convert legacy map definition to BTF-defined in the perf BPF script test. - Rework prologue generation code to stop using libbpf deprecated APIs. - Add default hybrid events for 'perf stat' on x86. - Add topdown metrics in the default 'perf stat' on the hybrid machines (big/little cores). - Prefer sampled CPU when exporting JSON in 'perf data convert' - Fix ('perf stat CSV output linter') and ("Check branch stack sampling") 'perf test' entries on s390. * tag 'perf-tools-for-v6.0-2022-08-04' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux: (169 commits) perf stat: Refactor __run_perf_stat() common code perf lock: Print the number of lost entries for BPF perf lock: Add --map-nr-entries option perf lock: Introduce struct lock_contention perf scripting python: Do not build fail on deprecation warnings genelf: Use HAVE_LIBCRYPTO_SUPPORT, not the never defined HAVE_LIBCRYPTO perf build: Suppress openssl v3 deprecation warnings in libcrypto feature test perf parse-events: Break out tracepoint and printing perf parse-events: Don't #define YY_EXTRA_TYPE tools bpftool: Don't display disassembler-four-args feature test tools bpftool: Fix compilation error with new binutils tools bpf_jit_disasm: Don't display disassembler-four-args feature test tools bpf_jit_disasm: Fix compilation error with new binutils tools perf: Fix compilation error with new binutils tools include: add dis-asm-compat.h to handle version differences tools build: Don't display disassembler-four-args feature test tools build: Add feature test for init_disassemble_info API changes perf test: Add ARM SPE system wide test perf tools: Rework prologue generation code perf bpf: Convert legacy map definition to BTF-defined ...

tools bpftool: Don't display disassembler-four-args feature test

2022-08-01T18:34:05Z

The feature check does not seem important enough to display. Requested-by: Jiri Olsa Signed-off-by: Andres Freund Acked-by: Quentin Monnet Tested-by: Arnaldo Carvalho de Melo Cc: Alexei Starovoitov Cc: Ben Hutchings Cc: Quentin Monnet Cc: Sedat Dilek Cc: bpf@vger.kernel.org Link: http://lore.kernel.org/lkml/20220622181918.ykrs5rsnmx3og4sv@alap3.anarazel.de Link: https://lore.kernel.org/r/20220801013834.156015-9-andres@anarazel.de Signed-off-by: Arnaldo Carvalho de Melo

tools bpftool: Fix compilation error with new binutils

2022-08-01T18:32:46Z

binutils changed the signature of init_disassemble_info(), which now causes compilation to fail for tools/bpf/bpftool/jit_disasm.c, e.g. on debian unstable. Relevant binutils commit: https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=60a3da00bd5407f07 Wire up the feature test and switch to init_disassemble_info_compat(), which were introduced in prior commits, fixing the compilation failure. I verified that bpftool can still disassemble bpf programs, both with an old and new dis-asm.h API. There are no output changes for plain and json formats. When comparing the output from old binutils (2.35) to new bintuils with the patch (upstream snapshot) there are a few output differences, but they are unrelated to this patch. An example hunk is: 2f: pop %r14 31: pop %r13 33: pop %rbx - 34: leaveq - 35: retq + 34: leave + 35: ret Signed-off-by: Andres Freund Acked-by: Quentin Monnet Cc: Alexei Starovoitov Cc: Ben Hutchings Cc: Jiri Olsa Cc: Quentin Monnet Cc: Sedat Dilek Cc: bpf@vger.kernel.org Link: http://lore.kernel.org/lkml/20220622181918.ykrs5rsnmx3og4sv@alap3.anarazel.de Link: https://lore.kernel.org/r/20220801013834.156015-8-andres@anarazel.de Signed-off-by: Arnaldo Carvalho de Melo

bpftool: Remove zlib feature test from Makefile

2022-07-06T05:31:17Z

The feature test to detect the availability of zlib in bpftool's Makefile does not bring much. The library is not optional: it may or may not be required along libbfd for disassembling instructions, but in any case it is necessary to build feature.o or even libbpf, on which bpftool depends. If we remove the feature test, we lose the nicely formatted error message, but we get a compiler error about "zlib.h: No such file or directory", which is equally informative. Let's get rid of the test. Suggested-by: Andrii Nakryiko Signed-off-by: Quentin Monnet Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20220705200456.285943-1-quentin@isovalent.com

bpftool: Fix bootstrapping during a cross compilation

2022-06-09T12:01:21Z

This change adjusts the Makefile to use "HOSTAR" as the archive tool to keep the sanity of the build process for the bootstrap part in check. For the rationale, please continue reading. When cross compiling bpftool with buildroot, it leads to an invocation like: $ AR="/path/to/buildroot/host/bin/arc-linux-gcc-ar" \ CC="/path/to/buildroot/host/bin/arc-linux-gcc" \ ... make Which in return fails while building the bootstrap section: ----------------------------------8<---------------------------------- make: Entering directory '/src/bpftool-v6.7.0/src' ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ OFF ] ... clang-bpf-co-re: [ on ] <-- triggers bootstrap . . . LINK /src/bpftool-v6.7.0/src/bootstrap/bpftool /usr/bin/ld: /src/bpftool-v6.7.0/src/bootstrap/libbpf/libbpf.a: error adding symbols: archive has no index; run ranlib to add one collect2: error: ld returned 1 exit status make: *** [Makefile:211: /src/bpftool-v6.7.0/src/bootstrap/bpftool] Error 1 make: *** Waiting for unfinished jobs.... AR /src/bpftool-v6.7.0/src/libbpf/libbpf.a make[1]: Leaving directory '/src/bpftool-v6.7.0/libbpf/src' make: Leaving directory '/src/bpftool-v6.7.0/src' ---------------------------------->8---------------------------------- This occurs because setting "AR" confuses the build process for the bootstrap section and it calls "arc-linux-gcc-ar" to create and index "libbpf.a" instead of the host "ar". Signed-off-by: Shahab Vahedi Signed-off-by: Daniel Borkmann Reviewed-by: Quentin Monnet Cc: Jean-Philippe Brucker Link: https://lore.kernel.org/bpf/8d297f0c-cfd0-ef6f-3970-6dddb3d9a87a@synopsys.com

Merge tag 'kbuild-v5.18-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

2022-03-31T18:59:03Z

Pull Kbuild updates from Masahiro Yamada: - Add new environment variables, USERCFLAGS and USERLDFLAGS to allow additional flags to be passed to user-space programs. - Fix missing fflush() bugs in Kconfig and fixdep - Fix a minor bug in the comment format of the .config file - Make kallsyms ignore llvm's local labels, .L* - Fix UAPI compile-test for cross-compiling with Clang - Extend the LLVM= syntax to support LLVM= form for using a particular version of LLVm, and LLVM= form for using custom LLVM in a particular directory path. - Clean up Makefiles * tag 'kbuild-v5.18-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: kbuild: Make $(LLVM) more flexible kbuild: add --target to correctly cross-compile UAPI headers with Clang fixdep: use fflush() and ferror() to ensure successful write to files arch: syscalls: simplify uapi/kapi directory creation usr/include: replace extra-y with always-y certs: simplify empty certs creation in certs/Makefile certs: include certs/signing_key.x509 unconditionally kallsyms: ignore all local labels prefixed by '.L' kconfig: fix missing '# end of' for empty menu kconfig: add fflush() before ferror() check kbuild: replace $(if A,A,B) with $(or A,B) kbuild: Add environment variables for userprogs flags kbuild: unify cmd_copy and cmd_shipped

bpftool: Remove redundant slashes

2022-02-28T15:06:21Z

Because the OUTPUT variable ends with a slash but CURDIR doesn't, to keep the _OUTPUT value consistent, we add a trailing slash to CURDIR when defining _OUTPUT variable. Since the _OUTPUT variable holds a value ending with a trailing slash, there is no need to add another one when defining BOOTSTRAP_OUTPUT and LIBBPF_OUTPUT variables. Likewise, when defining LIBBPF_INCLUDE and LIBBPF_BOOTSTRAP_INCLUDE, we shouldn't add an extra slash either for the same reason. When building libbpf, the value of the DESTDIR argument should also not end with a trailing slash. Signed-off-by: Yuntao Wang Signed-off-by: Daniel Borkmann Reviewed-by: Quentin Monnet Link: https://lore.kernel.org/bpf/20220226163815.520133-1-ytcoode@gmail.com

bpftool: Remove usage of reallocarray()

2022-02-23T21:55:01Z

This commit fixes a compilation error on systems with glibc < 2.26 [0]: ``` In file included from main.h:14:0, from gen.c:24: linux/tools/include/tools/libc_compat.h:11:21: error: attempt to use poisoned "reallocarray" static inline void *reallocarray(void *ptr, size_t nmemb, size_t size) ``` This happens because gen.c pulls , and then (through main.h). When COMPAT_NEED_REALLOCARRAY is set, libc_compat.h defines reallocarray() which libbpf_internal.h poisons with a GCC pragma. This commit reuses libbpf_reallocarray() implemented in commit 029258d7b228 ("libbpf: Remove any use of reallocarray() in libbpf"). v1 -> v2: - reuse libbpf_reallocarray() instead of reimplementing it Fixes: a9caaba399f9 ("bpftool: Implement "gen min_core_btf" logic") Reported-by: Quentin Monnet Signed-off-by: Mauricio Vásquez Signed-off-by: Andrii Nakryiko Reviewed-by: Quentin Monnet Acked-by: Song Liu Link: https://lore.kernel.org/bpf/20220221125617.39610-1-mauricio@kinvolk.io [0]: https://lore.kernel.org/bpf/3bf2bd49-9f2d-a2df-5536-bc0dde70a83b@isovalent.com/

bpftool: Implement "gen min_core_btf" logic

2022-02-16T18:05:45Z

This commit implements the logic for the gen min_core_btf command. Specifically, it implements the following functions: - minimize_btf(): receives the path of a source and destination BTF files and a list of BPF objects. This function records the relocations for all objects and then generates the BTF file by calling btfgen_get_btf() (implemented in the following commit). - btfgen_record_obj(): loads the BTF and BTF.ext sections of the BPF objects and loops through all CO-RE relocations. It uses bpf_core_calc_relo_insn() from libbpf and passes the target spec to btfgen_record_reloc(), that calls one of the following functions depending on the relocation kind. - btfgen_record_field_relo(): uses the target specification to mark all the types that are involved in a field-based CO-RE relocation. In this case types resolved and marked recursively using btfgen_mark_type(). Only the struct and union members (and their types) involved in the relocation are marked to optimize the size of the generated BTF file. - btfgen_record_type_relo(): marks the types involved in a type-based CO-RE relocation. In this case no members for the struct and union types are marked as libbpf doesn't use them while performing this kind of relocation. Pointed types are marked as they are used by libbpf in this case. - btfgen_record_enumval_relo(): marks the whole enum type for enum-based relocations. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20220215225856.671072-5-mauricio@kinvolk.io