aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
diff options
context:
space:
mode:
authorLinus Torvalds <torvalds@linux-foundation.org>2021-04-29 11:57:23 -0700
committerLinus Torvalds <torvalds@linux-foundation.org>2021-04-29 11:57:23 -0700
commit9d31d2338950293ec19d9b095fbaa9030899dcb4 (patch)
treee688040d0557c24a2eeb9f6c9c223d949f6f7ef9 /scripts
parentMerge tag 'x86-mm-2021-04-29' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip (diff)
parentnet: selftest: fix build issue if INET is disabled (diff)
downloadlinux-dev-9d31d2338950293ec19d9b095fbaa9030899dcb4.tar.xz
linux-dev-9d31d2338950293ec19d9b095fbaa9030899dcb4.zip
Merge tag 'net-next-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next
Pull networking updates from Jakub Kicinski: "Core: - bpf: - allow bpf programs calling kernel functions (initially to reuse TCP congestion control implementations) - enable task local storage for tracing programs - remove the need to store per-task state in hash maps, and allow tracing programs access to task local storage previously added for BPF_LSM - add bpf_for_each_map_elem() helper, allowing programs to walk all map elements in a more robust and easier to verify fashion - sockmap: support UDP and cross-protocol BPF_SK_SKB_VERDICT redirection - lpm: add support for batched ops in LPM trie - add BTF_KIND_FLOAT support - mostly to allow use of BTF on s390 which has floats in its headers files - improve BPF syscall documentation and extend the use of kdoc parsing scripts we already employ for bpf-helpers - libbpf, bpftool: support static linking of BPF ELF files - improve support for encapsulation of L2 packets - xdp: restructure redirect actions to avoid a runtime lookup, improving performance by 4-8% in microbenchmarks - xsk: build skb by page (aka generic zerocopy xmit) - improve performance of software AF_XDP path by 33% for devices which don't need headers in the linear skb part (e.g. virtio) - nexthop: resilient next-hop groups - improve path stability on next-hops group changes (incl. offload for mlxsw) - ipv6: segment routing: add support for IPv4 decapsulation - icmp: add support for RFC 8335 extended PROBE messages - inet: use bigger hash table for IP ID generation - tcp: deal better with delayed TX completions - make sure we don't give up on fast TCP retransmissions only because driver is slow in reporting that it completed transmitting the original - tcp: reorder tcp_congestion_ops for better cache locality - mptcp: - add sockopt support for common TCP options - add support for common TCP msg flags - include multiple address ids in RM_ADDR - add reset option support for resetting one subflow - udp: GRO L4 improvements - improve 'forward' / 'frag_list' co-existence with UDP tunnel GRO, allowing the first to take place correctly even for encapsulated UDP traffic - micro-optimize dev_gro_receive() and flow dissection, avoid retpoline overhead on VLAN and TEB GRO - use less memory for sysctls, add a new sysctl type, to allow using u8 instead of "int" and "long" and shrink networking sysctls - veth: allow GRO without XDP - this allows aggregating UDP packets before handing them off to routing, bridge, OvS, etc. - allow specifing ifindex when device is moved to another namespace - netfilter: - nft_socket: add support for cgroupsv2 - nftables: add catch-all set element - special element used to define a default action in case normal lookup missed - use net_generic infra in many modules to avoid allocating per-ns memory unnecessarily - xps: improve the xps handling to avoid potential out-of-bound accesses and use-after-free when XPS change race with other re-configuration under traffic - add a config knob to turn off per-cpu netdev refcnt to catch underflows in testing Device APIs: - add WWAN subsystem to organize the WWAN interfaces better and hopefully start driving towards more unified and vendor- independent APIs - ethtool: - add interface for reading IEEE MIB stats (incl. mlx5 and bnxt support) - allow network drivers to dump arbitrary SFP EEPROM data, current offset+length API was a poor fit for modern SFP which define EEPROM in terms of pages (incl. mlx5 support) - act_police, flow_offload: add support for packet-per-second policing (incl. offload for nfp) - psample: add additional metadata attributes like transit delay for packets sampled from switch HW (and corresponding egress and policy-based sampling in the mlxsw driver) - dsa: improve support for sandwiched LAGs with bridge and DSA - netfilter: - flowtable: use direct xmit in topologies with IP forwarding, bridging, vlans etc. - nftables: counter hardware offload support - Bluetooth: - improvements for firmware download w/ Intel devices - add support for reading AOSP vendor capabilities - add support for virtio transport driver - mac80211: - allow concurrent monitor iface and ethernet rx decap - set priority and queue mapping for injected frames - phy: add support for Clause-45 PHY Loopback - pci/iov: add sysfs MSI-X vector assignment interface to distribute MSI-X resources to VFs (incl. mlx5 support) New hardware/drivers: - dsa: mv88e6xxx: add support for Marvell mv88e6393x - 11-port Ethernet switch with 8x 1-Gigabit Ethernet and 3x 10-Gigabit interfaces. - dsa: support for legacy Broadcom tags used on BCM5325, BCM5365 and BCM63xx switches - Microchip KSZ8863 and KSZ8873; 3x 10/100Mbps Ethernet switches - ath11k: support for QCN9074 a 802.11ax device - Bluetooth: Broadcom BCM4330 and BMC4334 - phy: Marvell 88X2222 transceiver support - mdio: add BCM6368 MDIO mux bus controller - r8152: support RTL8153 and RTL8156 (USB Ethernet) chips - mana: driver for Microsoft Azure Network Adapter (MANA) - Actions Semi Owl Ethernet MAC - can: driver for ETAS ES58X CAN/USB interfaces Pure driver changes: - add XDP support to: enetc, igc, stmmac - add AF_XDP support to: stmmac - virtio: - page_to_skb() use build_skb when there's sufficient tailroom (21% improvement for 1000B UDP frames) - support XDP even without dedicated Tx queues - share the Tx queues with the stack when necessary - mlx5: - flow rules: add support for mirroring with conntrack, matching on ICMP, GTP, flex filters and more - support packet sampling with flow offloads - persist uplink representor netdev across eswitch mode changes - allow coexistence of CQE compression and HW time-stamping - add ethtool extended link error state reporting - ice, iavf: support flow filters, UDP Segmentation Offload - dpaa2-switch: - move the driver out of staging - add spanning tree (STP) support - add rx copybreak support - add tc flower hardware offload on ingress traffic - ionic: - implement Rx page reuse - support HW PTP time-stamping - octeon: support TC hardware offloads - flower matching on ingress and egress ratelimitting. - stmmac: - add RX frame steering based on VLAN priority in tc flower - support frame preemption (FPE) - intel: add cross time-stamping freq difference adjustment - ocelot: - support forwarding of MRP frames in HW - support multiple bridges - support PTP Sync one-step timestamping - dsa: mv88e6xxx, dpaa2-switch: offload bridge port flags like learning, flooding etc. - ipa: add IPA v4.5, v4.9 and v4.11 support (Qualcomm SDX55, SM8350, SC7280 SoCs) - mt7601u: enable TDLS support - mt76: - add support for 802.3 rx frames (mt7915/mt7615) - mt7915 flash pre-calibration support - mt7921/mt7663 runtime power management fixes" * tag 'net-next-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (2451 commits) net: selftest: fix build issue if INET is disabled net: netrom: nr_in: Remove redundant assignment to ns net: tun: Remove redundant assignment to ret net: phy: marvell: add downshift support for M88E1240 net: dsa: ksz: Make reg_mib_cnt a u8 as it never exceeds 255 net/sched: act_ct: Remove redundant ct get and check icmp: standardize naming of RFC 8335 PROBE constants bpf, selftests: Update array map tests for per-cpu batched ops bpf: Add batched ops support for percpu array bpf: Implement formatted output helpers with bstr_printf seq_file: Add a seq_bprintf function sfc: adjust efx->xdp_tx_queue_count with the real number of initialized queues net:nfc:digital: Fix a double free in digital_tg_recv_dep_req net: fix a concurrency bug in l2tp_tunnel_register() net/smc: Remove redundant assignment to rc mpls: Remove redundant assignment to err llc2: Remove redundant assignment to rc net/tls: Remove redundant initialization of record rds: Remove redundant assignment to nr_sig dt-bindings: net: mdio-gpio: add compatible for microchip,mdio-smi0 ...
Diffstat (limited to 'scripts')
-rwxr-xr-xscripts/bpf_doc.py (renamed from scripts/bpf_helpers_doc.py)191
-rwxr-xr-xscripts/link-vmlinux.sh7
2 files changed, 160 insertions, 38 deletions
diff --git a/scripts/bpf_helpers_doc.py b/scripts/bpf_doc.py
index 867ada23281c..2d94025b38e9 100755
--- a/scripts/bpf_helpers_doc.py
+++ b/scripts/bpf_doc.py
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
# Copyright (C) 2018-2019 Netronome Systems, Inc.
+# Copyright (C) 2021 Isovalent, Inc.
# In case user attempts to run with Python 2.
from __future__ import print_function
@@ -13,6 +14,9 @@ import sys, os
class NoHelperFound(BaseException):
pass
+class NoSyscallCommandFound(BaseException):
+ pass
+
class ParsingError(BaseException):
def __init__(self, line='<line not provided>', reader=None):
if reader:
@@ -22,18 +26,27 @@ class ParsingError(BaseException):
else:
BaseException.__init__(self, 'Error parsing line: %s' % line)
-class Helper(object):
+
+class APIElement(object):
"""
- An object representing the description of an eBPF helper function.
- @proto: function prototype of the helper function
- @desc: textual description of the helper function
- @ret: description of the return value of the helper function
+ An object representing the description of an aspect of the eBPF API.
+ @proto: prototype of the API symbol
+ @desc: textual description of the symbol
+ @ret: (optional) description of any associated return value
"""
def __init__(self, proto='', desc='', ret=''):
self.proto = proto
self.desc = desc
self.ret = ret
+
+class Helper(APIElement):
+ """
+ An object representing the description of an eBPF helper function.
+ @proto: function prototype of the helper function
+ @desc: textual description of the helper function
+ @ret: description of the return value of the helper function
+ """
def proto_break_down(self):
"""
Break down helper function protocol into smaller chunks: return type,
@@ -60,6 +73,7 @@ class Helper(object):
return res
+
class HeaderParser(object):
"""
An object used to parse a file in order to extract the documentation of a
@@ -72,6 +86,13 @@ class HeaderParser(object):
self.reader = open(filename, 'r')
self.line = ''
self.helpers = []
+ self.commands = []
+
+ def parse_element(self):
+ proto = self.parse_symbol()
+ desc = self.parse_desc()
+ ret = self.parse_ret()
+ return APIElement(proto=proto, desc=desc, ret=ret)
def parse_helper(self):
proto = self.parse_proto()
@@ -79,6 +100,18 @@ class HeaderParser(object):
ret = self.parse_ret()
return Helper(proto=proto, desc=desc, ret=ret)
+ def parse_symbol(self):
+ p = re.compile(' \* ?(.+)$')
+ capture = p.match(self.line)
+ if not capture:
+ raise NoSyscallCommandFound
+ end_re = re.compile(' \* ?NOTES$')
+ end = end_re.match(self.line)
+ if end:
+ raise NoSyscallCommandFound
+ self.line = self.reader.readline()
+ return capture.group(1)
+
def parse_proto(self):
# Argument can be of shape:
# - "void"
@@ -140,16 +173,29 @@ class HeaderParser(object):
break
return ret
- def run(self):
- # Advance to start of helper function descriptions.
- offset = self.reader.read().find('* Start of BPF helper function descriptions:')
+ def seek_to(self, target, help_message):
+ self.reader.seek(0)
+ offset = self.reader.read().find(target)
if offset == -1:
- raise Exception('Could not find start of eBPF helper descriptions list')
+ raise Exception(help_message)
self.reader.seek(offset)
self.reader.readline()
self.reader.readline()
self.line = self.reader.readline()
+ def parse_syscall(self):
+ self.seek_to('* DOC: eBPF Syscall Commands',
+ 'Could not find start of eBPF syscall descriptions list')
+ while True:
+ try:
+ command = self.parse_element()
+ self.commands.append(command)
+ except NoSyscallCommandFound:
+ break
+
+ def parse_helpers(self):
+ self.seek_to('* Start of BPF helper function descriptions:',
+ 'Could not find start of eBPF helper descriptions list')
while True:
try:
helper = self.parse_helper()
@@ -157,6 +203,9 @@ class HeaderParser(object):
except NoHelperFound:
break
+ def run(self):
+ self.parse_syscall()
+ self.parse_helpers()
self.reader.close()
###############################################################################
@@ -165,10 +214,11 @@ class Printer(object):
"""
A generic class for printers. Printers should be created with an array of
Helper objects, and implement a way to print them in the desired fashion.
- @helpers: array of Helper objects to print to standard output
+ @parser: A HeaderParser with objects to print to standard output
"""
- def __init__(self, helpers):
- self.helpers = helpers
+ def __init__(self, parser):
+ self.parser = parser
+ self.elements = []
def print_header(self):
pass
@@ -181,19 +231,23 @@ class Printer(object):
def print_all(self):
self.print_header()
- for helper in self.helpers:
- self.print_one(helper)
+ for elem in self.elements:
+ self.print_one(elem)
self.print_footer()
+
class PrinterRST(Printer):
"""
- A printer for dumping collected information about helpers as a ReStructured
- Text page compatible with the rst2man program, which can be used to
- generate a manual page for the helpers.
- @helpers: array of Helper objects to print to standard output
+ A generic class for printers that print ReStructured Text. Printers should
+ be created with a HeaderParser object, and implement a way to print API
+ elements in the desired fashion.
+ @parser: A HeaderParser with objects to print to standard output
"""
- def print_header(self):
- header = '''\
+ def __init__(self, parser):
+ self.parser = parser
+
+ def print_license(self):
+ license = '''\
.. Copyright (C) All BPF authors and contributors from 2014 to present.
.. See git log include/uapi/linux/bpf.h in kernel tree for details.
..
@@ -221,9 +275,39 @@ class PrinterRST(Printer):
..
.. Please do not edit this file. It was generated from the documentation
.. located in file include/uapi/linux/bpf.h of the Linux kernel sources
-.. (helpers description), and from scripts/bpf_helpers_doc.py in the same
+.. (helpers description), and from scripts/bpf_doc.py in the same
.. repository (header and footer).
+'''
+ print(license)
+ def print_elem(self, elem):
+ if (elem.desc):
+ print('\tDescription')
+ # Do not strip all newline characters: formatted code at the end of
+ # a section must be followed by a blank line.
+ for line in re.sub('\n$', '', elem.desc, count=1).split('\n'):
+ print('{}{}'.format('\t\t' if line else '', line))
+
+ if (elem.ret):
+ print('\tReturn')
+ for line in elem.ret.rstrip().split('\n'):
+ print('{}{}'.format('\t\t' if line else '', line))
+
+ print('')
+
+
+class PrinterHelpersRST(PrinterRST):
+ """
+ A printer for dumping collected information about helpers as a ReStructured
+ Text page compatible with the rst2man program, which can be used to
+ generate a manual page for the helpers.
+ @parser: A HeaderParser with Helper objects to print to standard output
+ """
+ def __init__(self, parser):
+ self.elements = parser.helpers
+
+ def print_header(self):
+ header = '''\
===========
BPF-HELPERS
===========
@@ -264,6 +348,7 @@ kernel at the top).
HELPERS
=======
'''
+ PrinterRST.print_license(self)
print(header)
def print_footer(self):
@@ -380,27 +465,50 @@ SEE ALSO
def print_one(self, helper):
self.print_proto(helper)
+ self.print_elem(helper)
- if (helper.desc):
- print('\tDescription')
- # Do not strip all newline characters: formatted code at the end of
- # a section must be followed by a blank line.
- for line in re.sub('\n$', '', helper.desc, count=1).split('\n'):
- print('{}{}'.format('\t\t' if line else '', line))
- if (helper.ret):
- print('\tReturn')
- for line in helper.ret.rstrip().split('\n'):
- print('{}{}'.format('\t\t' if line else '', line))
+class PrinterSyscallRST(PrinterRST):
+ """
+ A printer for dumping collected information about the syscall API as a
+ ReStructured Text page compatible with the rst2man program, which can be
+ used to generate a manual page for the syscall.
+ @parser: A HeaderParser with APIElement objects to print to standard
+ output
+ """
+ def __init__(self, parser):
+ self.elements = parser.commands
+
+ def print_header(self):
+ header = '''\
+===
+bpf
+===
+-------------------------------------------------------------------------------
+Perform a command on an extended BPF object
+-------------------------------------------------------------------------------
+
+:Manual section: 2
+
+COMMANDS
+========
+'''
+ PrinterRST.print_license(self)
+ print(header)
+
+ def print_one(self, command):
+ print('**%s**' % (command.proto))
+ self.print_elem(command)
- print('')
class PrinterHelpers(Printer):
"""
A printer for dumping collected information about helpers as C header to
be included from BPF program.
- @helpers: array of Helper objects to print to standard output
+ @parser: A HeaderParser with Helper objects to print to standard output
"""
+ def __init__(self, parser):
+ self.elements = parser.helpers
type_fwds = [
'struct bpf_fib_lookup',
@@ -511,7 +619,7 @@ class PrinterHelpers(Printer):
def print_header(self):
header = '''\
-/* This is auto-generated file. See bpf_helpers_doc.py for details. */
+/* This is auto-generated file. See bpf_doc.py for details. */
/* Forward declarations of BPF structs */'''
@@ -589,8 +697,13 @@ script = os.path.abspath(sys.argv[0])
linuxRoot = os.path.dirname(os.path.dirname(script))
bpfh = os.path.join(linuxRoot, 'include/uapi/linux/bpf.h')
+printers = {
+ 'helpers': PrinterHelpersRST,
+ 'syscall': PrinterSyscallRST,
+}
+
argParser = argparse.ArgumentParser(description="""
-Parse eBPF header file and generate documentation for eBPF helper functions.
+Parse eBPF header file and generate documentation for the eBPF API.
The RST-formatted output produced can be turned into a manual page with the
rst2man utility.
""")
@@ -601,6 +714,8 @@ if (os.path.isfile(bpfh)):
default=bpfh)
else:
argParser.add_argument('--filename', help='path to include/uapi/linux/bpf.h')
+argParser.add_argument('target', nargs='?', default='helpers',
+ choices=printers.keys(), help='eBPF API target')
args = argParser.parse_args()
# Parse file.
@@ -609,7 +724,9 @@ headerParser.run()
# Print formatted output to standard output.
if args.header:
- printer = PrinterHelpers(headerParser.helpers)
+ if args.target != 'helpers':
+ raise NotImplementedError('Only helpers header generation is supported')
+ printer = PrinterHelpers(headerParser)
else:
- printer = PrinterRST(headerParser.helpers)
+ printer = printers[args.target](headerParser)
printer.print_all()
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 3b261b0f74f0..667aacb9261c 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -213,6 +213,7 @@ vmlinux_link()
gen_btf()
{
local pahole_ver
+ local extra_paholeopt=
if ! [ -x "$(command -v ${PAHOLE})" ]; then
echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
@@ -227,8 +228,12 @@ gen_btf()
vmlinux_link ${1}
+ if [ "${pahole_ver}" -ge "121" ]; then
+ extra_paholeopt="${extra_paholeopt} --btf_gen_floats"
+ fi
+
info "BTF" ${2}
- LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
+ LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${extra_paholeopt} ${1}
# Create ${2} which contains just .BTF section but no symbols. Add
# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all