| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
Pull Kbuild updates from Masahiro Yamada:
- Support 'make compile_commands.json' to generate the compilation
database more easily, avoiding stale entries
- Support 'make clang-analyzer' and 'make clang-tidy' for static checks
using clang-tidy
- Preprocess scripts/modules.lds.S to allow CONFIG options in the
module linker script
- Drop cc-option tests from compiler flags supported by our minimal
GCC/Clang versions
- Use always 12-digits commit hash for CONFIG_LOCALVERSION_AUTO=y
- Use sha1 build id for both BFD linker and LLD
- Improve deb-pkg for reproducible builds and rootless builds
- Remove stale, useless scripts/namespace.pl
- Turn -Wreturn-type warning into error
- Fix build error of deb-pkg when CONFIG_MODULES=n
- Replace 'hostname' command with more portable 'uname -n'
- Various Makefile cleanups
* tag 'kbuild-v5.10' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (34 commits)
kbuild: Use uname for LINUX_COMPILE_HOST detection
kbuild: Only add -fno-var-tracking-assignments for old GCC versions
kbuild: remove leftover comment for filechk utility
treewide: remove DISABLE_LTO
kbuild: deb-pkg: clean up package name variables
kbuild: deb-pkg: do not build linux-headers package if CONFIG_MODULES=n
kbuild: enforce -Werror=return-type
scripts: remove namespace.pl
builddeb: Add support for all required debian/rules targets
builddeb: Enable rootless builds
builddeb: Pass -n to gzip for reproducible packages
kbuild: split the build log of kallsyms
kbuild: explicitly specify the build id style
scripts/setlocalversion: make git describe output more reliable
kbuild: remove cc-option test of -Werror=date-time
kbuild: remove cc-option test of -fno-stack-check
kbuild: remove cc-option test of -fno-strict-overflow
kbuild: move CFLAGS_{KASAN,UBSAN,KCSAN} exports to relevant Makefiles
kbuild: remove redundant CONFIG_KASAN check from scripts/Makefile.kasan
kbuild: do not create built-in objects for external module builds
...
|
|
Some old GCC versions between 4.5.0 and 4.9.1 might miscompile code
with -fvar-tracking-assingments (which is enabled by default with -g -O2).
Commit 2062afb4f804 ("Fix gcc-4.9.0 miscompilation of load_balance()
in scheduler") added -fno-var-tracking-assignments unconditionally to
work around this. But newer versions of GCC no longer have this bug, so
only add it for versions of GCC before 5.0. This allows various tools
such as a perf probe or gdb debuggers or systemtap to resolve variable
locations using dwarf locations in more code.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Acked-by: Ian Rogers <irogers@google.com>
Reviewed-by: Andi Kleen <andi@firstfloor.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Pull networking updates from Jakub Kicinski:
- Add redirect_neigh() BPF packet redirect helper, allowing to limit
stack traversal in common container configs and improving TCP
back-pressure.
Daniel reports ~10Gbps => ~15Gbps single stream TCP performance gain.
- Expand netlink policy support and improve policy export to user
space. (Ge)netlink core performs request validation according to
declared policies. Expand the expressiveness of those policies
(min/max length and bitmasks). Allow dumping policies for particular
commands. This is used for feature discovery by user space (instead
of kernel version parsing or trial and error).
- Support IGMPv3/MLDv2 multicast listener discovery protocols in
bridge.
- Allow more than 255 IPv4 multicast interfaces.
- Add support for Type of Service (ToS) reflection in SYN/SYN-ACK
packets of TCPv6.
- In Multi-patch TCP (MPTCP) support concurrent transmission of data on
multiple subflows in a load balancing scenario. Enhance advertising
addresses via the RM_ADDR/ADD_ADDR options.
- Support SMC-Dv2 version of SMC, which enables multi-subnet
deployments.
- Allow more calls to same peer in RxRPC.
- Support two new Controller Area Network (CAN) protocols - CAN-FD and
ISO 15765-2:2016.
- Add xfrm/IPsec compat layer, solving the 32bit user space on 64bit
kernel problem.
- Add TC actions for implementing MPLS L2 VPNs.
- Improve nexthop code - e.g. handle various corner cases when nexthop
objects are removed from groups better, skip unnecessary
notifications and make it easier to offload nexthops into HW by
converting to a blocking notifier.
- Support adding and consuming TCP header options by BPF programs,
opening the doors for easy experimental and deployment-specific TCP
option use.
- Reorganize TCP congestion control (CC) initialization to simplify
life of TCP CC implemented in BPF.
- Add support for shipping BPF programs with the kernel and loading
them early on boot via the User Mode Driver mechanism, hence reusing
all the user space infra we have.
- Support sleepable BPF programs, initially targeting LSM and tracing.
- Add bpf_d_path() helper for returning full path for given 'struct
path'.
- Make bpf_tail_call compatible with bpf-to-bpf calls.
- Allow BPF programs to call map_update_elem on sockmaps.
- Add BPF Type Format (BTF) support for type and enum discovery, as
well as support for using BTF within the kernel itself (current use
is for pretty printing structures).
- Support listing and getting information about bpf_links via the bpf
syscall.
- Enhance kernel interfaces around NIC firmware update. Allow
specifying overwrite mask to control if settings etc. are reset
during update; report expected max time operation may take to users;
support firmware activation without machine reboot incl. limits of
how much impact reset may have (e.g. dropping link or not).
- Extend ethtool configuration interface to report IEEE-standard
counters, to limit the need for per-vendor logic in user space.
- Adopt or extend devlink use for debug, monitoring, fw update in many
drivers (dsa loop, ice, ionic, sja1105, qed, mlxsw, mv88e6xxx,
dpaa2-eth).
- In mlxsw expose critical and emergency SFP module temperature alarms.
Refactor port buffer handling to make the defaults more suitable and
support setting these values explicitly via the DCBNL interface.
- Add XDP support for Intel's igb driver.
- Support offloading TC flower classification and filtering rules to
mscc_ocelot switches.
- Add PTP support for Marvell Octeontx2 and PP2.2 hardware, as well as
fixed interval period pulse generator and one-step timestamping in
dpaa-eth.
- Add support for various auth offloads in WiFi APs, e.g. SAE (WPA3)
offload.
- Add Lynx PHY/PCS MDIO module, and convert various drivers which have
this HW to use it. Convert mvpp2 to split PCS.
- Support Marvell Prestera 98DX3255 24-port switch ASICs, as well as
7-port Mediatek MT7531 IP.
- Add initial support for QCA6390 and IPQ6018 in ath11k WiFi driver,
and wcn3680 support in wcn36xx.
- Improve performance for packets which don't require much offloads on
recent Mellanox NICs by 20% by making multiple packets share a
descriptor entry.
- Move chelsio inline crypto drivers (for TLS and IPsec) from the
crypto subtree to drivers/net. Move MDIO drivers out of the phy
directory.
- Clean up a lot of W=1 warnings, reportedly the actively developed
subsections of networking drivers should now build W=1 warning free.
- Make sure drivers don't use in_interrupt() to dynamically adapt their
code. Convert tasklets to use new tasklet_setup API (sadly this
conversion is not yet complete).
* tag 'net-next-5.10' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (2583 commits)
Revert "bpfilter: Fix build error with CONFIG_BPFILTER_UMH"
net, sockmap: Don't call bpf_prog_put() on NULL pointer
bpf, selftest: Fix flaky tcp_hdr_options test when adding addr to lo
bpf, sockmap: Add locking annotations to iterator
netfilter: nftables: allow re-computing sctp CRC-32C in 'payload' statements
net: fix pos incrementment in ipv6_route_seq_next
net/smc: fix invalid return code in smcd_new_buf_create()
net/smc: fix valid DMBE buffer sizes
net/smc: fix use-after-free of delayed events
bpfilter: Fix build error with CONFIG_BPFILTER_UMH
cxgb4/ch_ipsec: Replace the module name to ch_ipsec from chcr
net: sched: Fix suspicious RCU usage while accessing tcf_tunnel_info
bpf: Fix register equivalence tracking.
rxrpc: Fix loss of final ack on shutdown
rxrpc: Fix bundle counting for exclusive connections
netfilter: restore NF_INET_NUMHOOKS
ibmveth: Identify ingress large send packets.
ibmveth: Switch order of ibmveth_helper calls.
cxgb4: handle 4-tuple PEDIT to NAT mode translation
selftests: Add VRF route leaking tests
...
|
|
This reverts commit 87e0d4f0f37fb0c8c4aeeac46fff5e957738df79.
-fno-merge-all-constants has been the default since clang-6; the minimum
supported version of clang in the kernel is clang-10 (10.0.1).
Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Reviewed-by: Fangrui Song <maskray@google.com>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Will Deacon <will@kernel.org>
Link: https://lkml.kernel.org/r/20200902225911.209899-3-ndesaulniers@google.com
Link: https://reviews.llvm.org/rL329300.
Link: https://github.com/ClangBuiltLinux/linux/issues/9
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Catch errors which at least gcc tolerates by default:
warning: 'return' with no value, in function returning non-void [-Wreturn-type]
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
|
|
namespace.pl is intended to help locate symbols which are defined but
are not used externally. The goal is to avoid bloat of the namespace in
the resulting kernel image.
The script relies on object data, and only finds unused symbols for the
configuration used to generate that object data. This results in a lot
of false positive warnings such as symbols only used by a single
architecture, or symbols which are used externally only under certain
configurations.
Running namespace.pl using allyesconfig, allmodconfig, and
x86_64_defconfig yields the following results:
* allmodconfig
* 11122 unique symbol names with no external reference
* 1194 symbols listed as multiply defined
* 214 symbols it can't resolve
* allyesconfig
* 10997 unique symbol names with no external reference
* 1194 symbols listed as multiply defined
* 214 symbols it can't resolve
* x86_64_defconfig
* 5757 unique symbol names with no external reference
* 528 symbols listed as multiply defined
* 154 symbols it can't resolve
The script also has no way to easily limit the scope of the checks to
a given subset of the kernel, such as only checking for symbols defined
within a module or subsystem.
Discussion on public mailing lists seems to indicate that many view the
tool output as suspect or not very useful (see discussions at [1] and
[2] for further context).
As described by Masahiro Yamada at [2], namespace.pl provides 3 types of
checks: listing multiply defined symbols, resolving external symbols,
and warnings about symbols with no reference.
The first category of issues is easily caught by the linker as any set
of multiply defined symbols should fail to link. The second category of
issues is also caught by linking, as undefined symbols would cause
issues. Even with modules, these types of issues where a module relies
on an external symbol are caught by modpost.
The remaining category of issues reported is the list of symbols with no
external reference, and is the primary motivation of this script.
However, it ought to be clear from the above examples that the output is
difficult to sort through. Even allyesconfig has ~10000 entries.
The current submit-checklist indicates that patches ought to go through
namespacecheck and fix any new issues arising. But that itself presents
problems. As described at [1], many cases of reports are due to
configuration where a function is used externally by some configuration
settings. Prominent maintainers appear to dislike changes modify code
such that symbols become static based on CONFIG_* flags ([3], and [4])
One possible solution is to adjust the advice and indicate that we only
care about the output of namespacecheck on allyesconfig or allmodconfig
builds...
However, given the discussion at [2], I suspect that few people are
actively using this tool. It doesn't have a maintainer in the
MAINTAINERS flie, and it produces so many warnings for unused symbols
that it is difficult to use effectively. Thus, I propose we simply
remove it.
[1] https://lore.kernel.org/netdev/20200708164812.384ae8ea@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com/
[2] https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller@intel.com/
[3] https://lore.kernel.org/netdev/20190828.154744.2058157956381129672.davem@davemloft.net/
[4] https://lore.kernel.org/netdev/20190827210928.576c5fef@cakuba.netronome.com/
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
ld's --build-id defaults to "sha1" style, while lld defaults to "fast".
The build IDs are very different between the two, which may confuse
programs that reference them.
Signed-off-by: Bill Wendling <morbo@google.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Rejecting non-native endian BTF overlapped with the addition
of support for it.
The rest were more simple overlapping changes, except the
renesas ravb binding update, which had to follow a file
move as well as a YAML conversion.
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
|
|
|
|
The minimal compiler versions, GCC 4.9 and Clang 10 support this flag.
Here is the godbolt:
https://godbolt.org/z/xvjcMa
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Acked-by: Will Deacon <will@kernel.org>
|
|
The minimal compiler version, GCC 4.9 supports this flag.
Nathan Chancellor pointed out:
"This flag is technically ignored by clang (see commit
05b0798916f01690b5903302e51f3136274e291f) but that obviously
does not matter for the sake of this."
Here is the godbolt:
https://godbolt.org/z/59cK6o
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
|
|
The minimal compiler versions, GCC 4.9 and Clang 10 support this flag.
Here is the godbolt:
https://godbolt.org/z/odq8h9
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Acked-by: Will Deacon <will@kernel.org>
|
|
Move CFLAGS_KASAN*, CFLAGS_UBSAN, CFLAGS_KCSAN to Makefile.kasan,
Makefile.ubsan, Makefile.kcsan, respectively.
This commit also avoids the same -fsanitize=* flags being added to
CFLAGS_UBSAN multiple times.
Prior to this commit, the ubsan flags were appended by the '+='
operator, without any initialization. Some build targets such as
'make bindeb-pkg' recurses to the top Makefile, and ended up with
adding the same flags to CFLAGS_UBSAN twice.
Clear CFLAGS_UBSAN with ':=' to make it a simply expanded variable.
This is better than a recursively expanded variable, which evaluates
$(call cc-option, ...) multiple times before Kbuild starts descending
to subdirectories.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Marco Elver <elver@google.com>
|
|
'make M=/path/to/your/external/module' creates a pointless built-in.a
in the top of the external module directory because KBUILD_BUILTIN is
set to 1.
Clear KBUILD_BUILTIN when we are building external modules so that
'make M=...' and 'make M=... modules' work equivalently.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
There was a request to preprocess the module linker script like we
do for the vmlinux one. (https://lkml.org/lkml/2020/8/21/512)
The difference between vmlinux.lds and module.lds is that the latter
is needed for external module builds, thus must be cleaned up by
'make mrproper' instead of 'make clean'. Also, it must be created
by 'make modules_prepare'.
You cannot put it in arch/$(SRCARCH)/kernel/, which is cleaned up by
'make clean'. I moved arch/$(SRCARCH)/kernel/module.lds to
arch/$(SRCARCH)/include/asm/module.lds.h, which is included from
scripts/module.lds.S.
scripts/module.lds is fine because 'make clean' keeps all the
build artifacts under scripts/.
You can add arch-specific sections in <asm/module.lds.h>.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Jessica Yu <jeyu@kernel.org>
Acked-by: Will Deacon <will@kernel.org>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Jessica Yu <jeyu@kernel.org>
|
|
Alexei Starovoitov says:
====================
pull-request: bpf-next 2020-09-23
The following pull-request contains BPF updates for your *net-next* tree.
We've added 95 non-merge commits during the last 22 day(s) which contain
a total of 124 files changed, 4211 insertions(+), 2040 deletions(-).
The main changes are:
1) Full multi function support in libbpf, from Andrii.
2) Refactoring of function argument checks, from Lorenz.
3) Make bpf_tail_call compatible with functions (subprograms), from Maciej.
4) Program metadata support, from YiFei.
5) bpf iterator optimizations, from Yonghong.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
Currently all the resolve_btfids 'users' are under CONFIG_BPF
code, so if we have CONFIG_BPF disabled, resolve_btfids will
fail, because there's no data to resolve.
Disabling resolve_btfids if there's CONFIG_BPF disabled,
so we won't fail such builds.
Suggested-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20200923185735.3048198-1-jolsa@kernel.org
|
|
|
|
|
|
Pull livepatching fix from Petr Mladek:
"Workaround for 'unreachable instruction' objtool warnings that happen
with some compiler versions"
* tag 'livepatching-for-5.9-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/livepatching/livepatching:
Revert "kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled"
|
|
|
|
Use of the new -flive-patching flag was introduced with the following
commit:
43bd3a95c98e ("kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled")
This flag has several drawbacks:
- It disables some optimizations, so it can have a negative effect on
performance.
- According to the GCC documentation it's not compatible with LTO, which
will become a compatibility issue as LTO support gets upstreamed in
the kernel.
- It was intended to be used for source-based patch generation tooling,
as opposed to binary-based patch generation tooling (e.g.,
kpatch-build). It probably should have at least been behind a
separate config option so as not to negatively affect other livepatch
users.
- Clang doesn't have the flag, so as far as I can tell, this method of
generating patches is incompatible with Clang, which like LTO is
becoming more mainstream.
- It breaks GCC's implicit noreturn detection for local functions. This
is the cause of several "unreachable instruction" objtool warnings.
- The broken noreturn detection is an obvious GCC regression, but we
haven't yet gotten GCC developers to acknowledge that, which doesn't
inspire confidence in their willingness to keep the feature working as
optimizations are added or changed going forward.
- While there *is* a distro which relies on this flag for their distro
livepatch module builds, there's not a publicly documented way to
create safe livepatch modules with it. Its use seems to be based on
tribal knowledge. It serves no benefit to those who don't know how to
use it.
(In fact, I believe the current livepatch documentation and samples
are misleading and dangerous, and should be corrected. Or at least
amended with a disclaimer. But I don't feel qualified to make such
changes.)
Also, we have an idea for using objtool to detect function changes,
which could potentially obsolete the need for this flag anyway.
At this point the flag has no benefits for upstream which would
counteract the above drawbacks. Revert it until it becomes more ready.
This reverts commit 43bd3a95c98e1a86b8b55d97f745c224ecff02b9.
Fixes: 43bd3a95c98e ("kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled")
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Acked-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/696262e997359666afa053fe7d1a9fb2bb373964.1595010490.git.jpoimboe@redhat.com
|
|
|
|
This patch adds clang-tidy and the clang static-analyzer as make
targets. The goal of this patch is to make static analysis tools
usable and extendable by any developer or researcher who is familiar
with basic c++.
The current static analysis tools require intimate knowledge of the
internal workings of the static analysis. Clang-tidy and the clang
static analyzers expose an easy to use api and allow users unfamiliar
with clang to write new checks with relative ease.
===Clang-tidy===
Clang-tidy is an easily extendable 'linter' that runs on the AST.
Clang-tidy checks are easy to write and understand. A check consists of
two parts, a matcher and a checker. The matcher is created using a
domain specific language that acts on the AST
(https://clang.llvm.org/docs/LibASTMatchersReference.html). When AST
nodes are found by the matcher a callback is made to the checker. The
checker can then execute additional checks and issue warnings.
Here is an example clang-tidy check to report functions that have calls
to local_irq_disable without calls to local_irq_enable and vice-versa.
Functions flagged with __attribute((annotation("ignore_irq_balancing")))
are ignored for analysis. (https://reviews.llvm.org/D65828)
===Clang static analyzer===
The clang static analyzer is a more powerful static analysis tool that
uses symbolic execution to find bugs. Currently there is a check that
looks for potential security bugs from invalid uses of kmalloc and
kfree. There are several more general purpose checks that are useful for
the kernel.
The clang static analyzer is well documented and designed to be
extensible.
(https://clang-analyzer.llvm.org/checker_dev_manual.html)
(https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf)
The main draw of the clang tools is how accessible they are. The clang
documentation is very nice and these tools are built specifically to be
easily extendable by any developer. They provide an accessible method of
bug-finding and research to people who are not overly familiar with the
kernel codebase.
Signed-off-by: Nathan Huckleberry <nhuck@google.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Currently, you need to manually run scripts/gen_compile_commands.py
to create compile_commands.json. It parses all the .*.cmd files found
under the specified directory.
If you rebuild the kernel over again without 'make clean',
.*.cmd files from older builds will create stale entries in
compile_commands.json.
This commit wires up the compile_commands.json rule to Makefile, and
makes it parse only the .*.cmd files involved in the current build.
Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order
to the script. The objects or archives linked to vmlinux are listed in
$(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are
listed in modules.order.
You can create compile_commands.json from Make:
$ make -j$(nproc) CC=clang compile_commands.json
You can also build vmlinux, modules, and compile_commands.json all
together in a single command:
$ make -j$(nproc) CC=clang all compile_commands.json
It works for M= builds as well. In this case, compile_commands.json
is created in the top directory of the external module.
This is convenient, but it has a drawback; the coverage of the
compile_commands.json is reduced because only the objects linked to
vmlinux or modules are handled. For example, the following C files are
not included in the compile_commands.json:
- Decompressor source files (arch/*/boot/)
- VDSO source files
- C files used to generate intermediates (e.g. kernel/bounds.c)
- Standalone host programs
I think it is fine for most developers because our main interest is
the kernel-space code.
If you want to cover all the compiled C files, please build the kernel,
then run the script manually as you did before:
$ make clean # if you want to remove stale .cmd files [optional]
$ make -j$(nproc) CC=clang
$ scripts/gen_compile_commands.py
Here is a note for out-of-tree builds. 'make compile_commands.json'
works with O= option, but please notice compile_commands.json is
created in the object tree instead of the source tree.
Some people may want to have compile_commands.json in the source tree
because Clang Tools searches for it through all parent paths of the
first input source file.
However, you cannot do this for O= builds. Kbuild should never generate
any build artifact in the source tree when O= is given because the
source tree might be read-only. Any write attempt to the source tree
is monitored and the violation may be reported. See the commit log of
8ef14c2c41d9.
So, the only possible way is to create compile_commands.json in the
object tree, then specify '-p <build-path>' when you use clang-check,
clang-tidy, etc.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Nick Desaulniers <ndesaulniers@google.com>
|
|
Some targets (localyesconfig, localmodconfig, defconfig) hide the
command running, but the others do not.
Users know which Kconfig flavor they are running, so it is OK to hide
the command. Add $(Q) to all commands consistently. If you want to see
the full command running, pass V=1 from the command line.
syncconfig is the exceptional case, which occurs without explicit
command invocation by the user. Display the Kbuild-style log for it.
The ugly bare log will go away.
[Before]
scripts/kconfig/conf --syncconfig Kconfig
[After]
SYNC include/config/auto.conf
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
While playing with [1] I saw that the handling
of CONFIG_DEBUG_INFO can be simplified.
[1] https://patchwork.kernel.org/patch/11716107/
Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
|
|
The '%' in filter/filter-out matches to any number of any characters,
including empty string.
So, '%config' matches to 'config', and '%install' to 'install'.
Drop the redundant patterns.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
|
|
Recently 0day reported many strange performance changes (regression or
improvement), in which there was no obvious relation between the culprit
commit and the benchmark at the first look, and it causes people to doubt
the test itself is wrong.
Upon further check, many of these cases are caused by the change to the
alignment of kernel text or data, as whole text/data of kernel are linked
together, change in one domain may affect alignments of other domains.
gcc has an option '-falign-functions=n' to force text aligned, and with
that option enabled, some of those performance changes will be gone, like
[1][2][3].
Add this option so that developers and 0day can easily find performance
bump caused by text alignment change, as tracking these strange bump is
quite time consuming. Though it can't help in other cases like data
alignment changes like [4].
Following is some size data for v5.7 kernel built with a RHEL config used
in 0day:
text data bss dec filename
19738771 13292906 5554236 38585913 vmlinux.noalign
19758591 13297002 5529660 38585253 vmlinux.align32
Raw vmlinux size in bytes:
v5.7 v5.7+align32
253950832 254018000 +0.02%
Some benchmark data, most of them have no big change:
* hackbench: [ -1.8%, +0.5%]
* fsmark: [ -3.2%, +3.4%] # ext4/xfs/btrfs
* kbuild: [ -2.0%, +0.9%]
* will-it-scale: [ -0.5%, +1.8%] # mmap1/pagefault3
* netperf:
- TCP_CRR [+16.6%, +97.4%]
- TCP_RR [-18.5%, -1.8%]
- TCP_STREAM [ -1.1%, +1.9%]
[1] https://lore.kernel.org/lkml/20200114085637.GA29297@shao2-debian/
[2] https://lore.kernel.org/lkml/20200330011254.GA14393@feng-iot/
[3] https://lore.kernel.org/lkml/1d98d1f0-fe84-6df7-f5bd-f4cb2cdb7f45@intel.com/
[4] https://lore.kernel.org/lkml/20200205123216.GO12867@shao2-debian/
Signed-off-by: Feng Tang <feng.tang@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Andi Kleen <andi.kleen@intel.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Andy Shevchenko <andriy.shevchenko@intel.com>
Link: http://lkml.kernel.org/r/1595475001-90945-1-git-send-email-feng.tang@intel.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Pull Kbuild updates from Masahiro Yamada:
- run the checker (e.g. sparse) after the compiler
- remove unneeded cc-option tests for old compiler flags
- fix tar-pkg to install dtbs
- introduce ccflags-remove-y and asflags-remove-y syntax
- allow to trace functions in sub-directories of lib/
- introduce hostprogs-always-y and userprogs-always-y syntax
- various Makefile cleanups
* tag 'kbuild-v5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
kbuild: stop filtering out $(GCC_PLUGINS_CFLAGS) from cc-option base
kbuild: include scripts/Makefile.* only when relevant CONFIG is enabled
kbuild: introduce hostprogs-always-y and userprogs-always-y
kbuild: sort hostprogs before passing it to ifneq
kbuild: move host .so build rules to scripts/gcc-plugins/Makefile
kbuild: Replace HTTP links with HTTPS ones
kbuild: trace functions in subdirectories of lib/
kbuild: introduce ccflags-remove-y and asflags-remove-y
kbuild: do not export LDFLAGS_vmlinux
kbuild: always create directories of targets
powerpc/boot: add DTB to 'targets'
kbuild: buildtar: add dtbs support
kbuild: remove cc-option test of -ffreestanding
kbuild: remove cc-option test of -fno-stack-protector
Revert "kbuild: Create directory for target DTB"
kbuild: run the checker after the compiler
|
|
Commit d26e94149276 ("kbuild: no gcc-plugins during cc-option tests")
was neeeded because scripts/Makefile.gcc-plugins was too early.
This is unneeded by including scripts/Makefile.gcc-plugins last,
and being careful to not add cc-option tests after it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Currently, the top Makefile includes all of scripts/Makefile.<feature>
even if the associated CONFIG option is disabled.
Do not include unneeded Makefiles in order to slightly optimize the
parse stage.
Include $(include-y), and ignore $(include-).
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
When you clean the build tree for ARCH=arm, you may see the following
error message from 'nm' command:
$ make -j24 ARCH=arm clean
CLEAN arch/arm/crypto
CLEAN arch/arm/kernel
CLEAN arch/arm/mach-at91
CLEAN arch/arm/mach-omap2
CLEAN arch/arm/vdso
CLEAN certs
CLEAN lib
CLEAN usr
CLEAN net/wireless
CLEAN drivers/firmware/efi/libstub
nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
/bin/sh: 1: arithmetic expression: expecting primary: " "
CLEAN arch/arm/boot/compressed
CLEAN drivers/scsi
CLEAN drivers/tty/vt
CLEAN arch/arm/boot
CLEAN vmlinux.symvers modules.builtin modules.builtin.modinfo
Even if you rerun the same command, the error message will not be
shown despite vmlinux is already gone.
To reproduce it, the parallel option -j is needed. Single thread
cleaning always executes 'archclean', 'vmlinuxclean' in this order,
so vmlinux still exists when arch/arm/boot/compressed/ is cleaned.
Looking at arch/arm/boot/compressed/Makefile does not help understand
the reason of the error message. Both KBSS_SZ and LDFLAGS_vmlinux are
assigned with '=' operator, hence, they are not expanded unless used.
Obviously, 'make clean' does not use them.
In fact, the root cause exists in the top Makefile:
export LDFLAGS_vmlinux
Since LDFLAGS_vmlinux is an exported variable, LDFLAGS_vmlinux in
arch/arm/boot/compressed/Makefile is expanded when scripts/Makefile.clean
has a command to execute. This is why the error message shows up only
when there exist build artifacts in arch/arm/boot/compressed/.
Adding 'unexport LDFLAGS_vmlinux' to arch/arm/boot/compressed/Makefile
will fix it as far as ARCH=arm is concerned, but I think the proper fix
is to get rid of 'export LDFLAGS_vmlinux' from the top Makefile.
LDFLAGS_vmlinux in the top Makefile contains linker flags for the top
vmlinux. LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is for
arch/arm/boot/compressed/vmlinux. They just happen to have the same
variable name, but are used for different purposes. Stop shadowing
LDFLAGS_vmlinux.
This commit passes LDFLAGS_vmlinux to scripts/link-vmlinux.sh via a
command line parameter instead of via an environment variable. LD and
KBUILD_LDFLAGS are exported, but I did the same for consistency. Anyway,
they must be included in cmd_link-vmlinux to allow if_changed to detect
the changes in LD or KBUILD_LDFLAGS.
The following Makefiles are not affected:
arch/arm/boot/compressed/Makefile
arch/h8300/boot/compressed/Makefile
arch/nios2/boot/compressed/Makefile
arch/parisc/boot/compressed/Makefile
arch/s390/boot/compressed/Makefile
arch/sh/boot/compressed/Makefile
arch/sh/boot/romimage/Makefile
arch/x86/boot/compressed/Makefile
They use ':=' or '=' to clear the LDFLAGS_vmlinux inherited from the
top Makefile.
We need to take a closer look at the impact to unicore32 and xtensa.
arch/unicore32/boot/compressed/Makefile only uses '+=' operator for
LDFLAGS_vmlinux. So, the decompressor previously inherited the linker
flags from the top Makefile.
However, commit 70fac51feaf2 ("unicore32 additional architecture files:
boot process") was merged before commit 1f2bfbd00e46 ("kbuild: link of
vmlinux moved to a script"). So, I rather consider this is a bug fix of
1f2bfbd00e46.
arch/xtensa/boot/boot-elf/Makefile is also affected, but this is also
considered a fix for the same reason. It did not inherit LDFLAGS_vmlinux
when commit 4bedea945451 ("[PATCH] xtensa: Architecture support for
Tensilica Xtensa Part 2") was merged. I deleted $(LDFLAGS_vmlinux),
which is now empty.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
|
|
Pull networking updates from David Miller:
1) Support 6Ghz band in ath11k driver, from Rajkumar Manoharan.
2) Support UDP segmentation in code TSO code, from Eric Dumazet.
3) Allow flashing different flash images in cxgb4 driver, from Vishal
Kulkarni.
4) Add drop frames counter and flow status to tc flower offloading,
from Po Liu.
5) Support n-tuple filters in cxgb4, from Vishal Kulkarni.
6) Various new indirect call avoidance, from Eric Dumazet and Brian
Vazquez.
7) Fix BPF verifier failures on 32-bit pointer arithmetic, from
Yonghong Song.
8) Support querying and setting hardware address of a port function via
devlink, use this in mlx5, from Parav Pandit.
9) Support hw ipsec offload on bonding slaves, from Jarod Wilson.
10) Switch qca8k driver over to phylink, from Jonathan McDowell.
11) In bpftool, show list of processes holding BPF FD references to
maps, programs, links, and btf objects. From Andrii Nakryiko.
12) Several conversions over to generic power management, from Vaibhav
Gupta.
13) Add support for SO_KEEPALIVE et al. to bpf_setsockopt(), from Dmitry
Yakunin.
14) Various https url conversions, from Alexander A. Klimov.
15) Timestamping and PHC support for mscc PHY driver, from Antoine
Tenart.
16) Support bpf iterating over tcp and udp sockets, from Yonghong Song.
17) Support 5GBASE-T i40e NICs, from Aleksandr Loktionov.
18) Add kTLS RX HW offload support to mlx5e, from Tariq Toukan.
19) Fix the ->ndo_start_xmit() return type to be netdev_tx_t in several
drivers. From Luc Van Oostenryck.
20) XDP support for xen-netfront, from Denis Kirjanov.
21) Support receive buffer autotuning in MPTCP, from Florian Westphal.
22) Support EF100 chip in sfc driver, from Edward Cree.
23) Add XDP support to mvpp2 driver, from Matteo Croce.
24) Support MPTCP in sock_diag, from Paolo Abeni.
25) Commonize UDP tunnel offloading code by creating udp_tunnel_nic
infrastructure, from Jakub Kicinski.
26) Several pci_ --> dma_ API conversions, from Christophe JAILLET.
27) Add FLOW_ACTION_POLICE support to mlxsw, from Ido Schimmel.
28) Add SK_LOOKUP bpf program type, from Jakub Sitnicki.
29) Refactor a lot of networking socket option handling code in order to
avoid set_fs() calls, from Christoph Hellwig.
30) Add rfc4884 support to icmp code, from Willem de Bruijn.
31) Support TBF offload in dpaa2-eth driver, from Ioana Ciornei.
32) Support XDP_REDIRECT in qede driver, from Alexander Lobakin.
33) Support PCI relaxed ordering in mlx5 driver, from Aya Levin.
34) Support TCP syncookies in MPTCP, from Flowian Westphal.
35) Fix several tricky cases of PMTU handling wrt. briding, from Stefano
Brivio.
* git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (2056 commits)
net: thunderx: initialize VF's mailbox mutex before first usage
usb: hso: remove bogus check for EINPROGRESS
usb: hso: no complaint about kmalloc failure
hso: fix bailout in error case of probe
ip_tunnel_core: Fix build for archs without _HAVE_ARCH_IPV6_CSUM
selftests/net: relax cpu affinity requirement in msg_zerocopy test
mptcp: be careful on subflow creation
selftests: rtnetlink: make kci_test_encap() return sub-test result
selftests: rtnetlink: correct the final return value for the test
net: dsa: sja1105: use detected device id instead of DT one on mismatch
tipc: set ub->ifindex for local ipv6 address
ipv6: add ipv6_dev_find()
net: openvswitch: silence suspicious RCU usage warning
Revert "vxlan: fix tos value before xmit"
ptp: only allow phase values lower than 1 period
farsync: switch from 'pci_' to 'dma_' API
wan: wanxl: switch from 'pci_' to 'dma_' API
hv_netvsc: do not use VF device if link is down
dpaa2-eth: Fix passing zero to 'PTR_ERR' warning
net: macb: Properly handle phylink on at91sam9x
...
|
|
Pull automatic variable initialization updates from Kees Cook:
"This adds the "zero" init option from Clang, which is being used
widely in production builds of Android and Chrome OS (though it also
keeps the "pattern" init, which is better for debug builds).
- Introduce CONFIG_INIT_STACK_ALL_ZERO (Alexander Potapenko)"
* tag 'var-init-v5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux:
security: allow using Clang's zero initialization for stack variables
|
|
Pull x86 boot updates from Ingo Molnar:
"The main change in this cycle was to add support for ZSTD-compressed
kernel and initrd images.
ZSTD has a very fast decompressor, yet it compresses better than gzip"
* tag 'x86-boot-2020-08-03' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
Documentation: dontdiff: Add zstd compressed files
.gitignore: Add ZSTD-compressed files
x86: Add support for ZSTD compressed kernel
x86: Bump ZO_z_extra_bytes margin for zstd
usr: Add support for zstd compressed initramfs
init: Add support for zstd compressed kernel
lib: Add zstd support to decompress
lib: Prepare zstd for preboot environment, improve performance
|
|
|
|
Resolved kernel/bpf/btf.c using instructions from merge commit
69138b34a7248d2396ab85c8652e20c0c39beaba
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
- Add the zstd and zstd22 cmds to scripts/Makefile.lib
- Add the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options
Architecture specific support is still needed for decompression.
Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20200730190841.2071656-4-nickrterrell@gmail.com
|
|
|
|
Pull Kbuild fixes from Masahiro Yamada:
- do not use non-portable strsep() in a host program
- fix single target builds for external modules
- change Clang's --prefix option to make it work for the latest Clang
* tag 'kbuild-fixes-v5.8-3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
Makefile: Fix GCC_TOOLCHAIN_DIR prefix for Clang cross compilation
kbuild: fix single target builds for external modules
modpost: remove use of non-standard strsep() in HOSTCC code
|
|
The UDP reuseport conflict was a little bit tricky.
The net-next code, via bpf-next, extracted the reuseport handling
into a helper so that the BPF sk lookup code could invoke it.
At the same time, the logic for reuseport handling of unconnected
sockets changed via commit efc6b6f6c3113e8b203b9debfb72d81e0f3dcace
which changed the logic to carry on the reuseport result into the
rest of the lookup loop if we do not return immediately.
This requires moving the reuseport_has_conns() logic into the callers.
While we are here, get rid of inline directives as they do not belong
in foo.c files.
The other changes were cases of more straightforward overlapping
modifications.
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
When CROSS_COMPILE is set (e.g. aarch64-linux-gnu-), if
$(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-elfedit,
GCC_TOOLCHAIN_DIR will be set to /usr/bin/. --prefix= will be set to
/usr/bin/ and Clang as of 11 will search for both
$(prefix)aarch64-linux-gnu-$needle and $(prefix)$needle.
GCC searchs for $(prefix)aarch64-linux-gnu/$version/$needle,
$(prefix)aarch64-linux-gnu/$needle and $(prefix)$needle. In practice,
$(prefix)aarch64-linux-gnu/$needle rarely contains executables.
To better model how GCC's -B/--prefix takes in effect in practice, newer
Clang (since
https://github.com/llvm/llvm-project/commit/3452a0d8c17f7166f479706b293caf6ac76ffd90)
only searches for $(prefix)$needle. Currently it will find /usr/bin/as
instead of /usr/bin/aarch64-linux-gnu-as.
Set --prefix= to $(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
(/usr/bin/aarch64-linux-gnu-) so that newer Clang can find the
appropriate cross compiling GNU as (when -no-integrated-as is in
effect).
Cc: stable@vger.kernel.org
Reported-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Fangrui Song <maskray@google.com>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Tested-by: Nathan Chancellor <natechancellor@gmail.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/1099
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
|
|
Alexei Starovoitov says:
====================
pull-request: bpf-next 2020-07-13
The following pull-request contains BPF updates for your *net-next* tree.
We've added 36 non-merge commits during the last 7 day(s) which contain
a total of 62 files changed, 2242 insertions(+), 468 deletions(-).
The main changes are:
1) Avoid trace_printk warning banner by switching bpf_trace_printk to use
its own tracing event, from Alan.
2) Better libbpf support on older kernels, from Andrii.
3) Additional AF_XDP stats, from Ciara.
4) build time resolution of BTF IDs, from Jiri.
5) BPF_CGROUP_INET_SOCK_RELEASE hook, from Stanislav.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
|
Linus Torvalds
Mark Wielaard
Nick Desaulniers
Olaf Hering
Jacob Keller
Bill Wendling
David S. Miller
Masahiro Yamada
Jiri Olsa
Josh Poimboeuf
Nathan Huckleberry
Sedat Dilek
Feng Tang
Nick Terrell
Fangrui Song