diff options
Diffstat (limited to 'Documentation')
108 files changed, 5515 insertions, 3885 deletions
diff --git a/Documentation/ABI/testing/sysfs-bus-css b/Documentation/ABI/testing/sysfs-bus-css index 2979c40c10e9..966f8504bd7b 100644 --- a/Documentation/ABI/testing/sysfs-bus-css +++ b/Documentation/ABI/testing/sysfs-bus-css @@ -33,3 +33,26 @@ Description: Contains the PIM/PAM/POM values, as reported by the in sync with the values current in the channel subsystem). Note: This is an I/O-subchannel specific attribute. Users: s390-tools, HAL + +What: /sys/bus/css/devices/.../driver_override +Date: June 2019 +Contact: Cornelia Huck <cohuck@redhat.com> + linux-s390@vger.kernel.org +Description: This file allows the driver for a device to be specified. When + specified, only a driver with a name matching the value written + to driver_override will have an opportunity to bind to the + device. The override is specified by writing a string to the + driver_override file (echo vfio-ccw > driver_override) and + may be cleared with an empty string (echo > driver_override). + This returns the device to standard matching rules binding. + Writing to driver_override does not automatically unbind the + device from its current driver or make any attempt to + automatically load the specified driver. If no driver with a + matching name is currently loaded in the kernel, the device + will not bind to any driver. This also allows devices to + opt-out of driver binding using a driver_override name such as + "none". Only a single driver may be specified in the override, + there is no support for parsing delimiters. + Note that unlike the mechanism of the same name for pci, this + file does not allow to override basic matching rules. I.e., + the driver must still match the subchannel type of the device. diff --git a/Documentation/ABI/testing/sysfs-bus-mdio b/Documentation/ABI/testing/sysfs-bus-mdio deleted file mode 100644 index 491baaf4285f..000000000000 --- a/Documentation/ABI/testing/sysfs-bus-mdio +++ /dev/null @@ -1,29 +0,0 @@ -What: /sys/bus/mdio_bus/devices/.../phy_id -Date: November 2012 -KernelVersion: 3.8 -Contact: netdev@vger.kernel.org -Description: - This attribute contains the 32-bit PHY Identifier as reported - by the device during bus enumeration, encoded in hexadecimal. - This ID is used to match the device with the appropriate - driver. - -What: /sys/bus/mdio_bus/devices/.../phy_interface -Date: February 2014 -KernelVersion: 3.15 -Contact: netdev@vger.kernel.org -Description: - This attribute contains the PHY interface as configured by the - Ethernet driver during bus enumeration, encoded in string. - This interface mode is used to configure the Ethernet MAC with the - appropriate mode for its data lines to the PHY hardware. - -What: /sys/bus/mdio_bus/devices/.../phy_has_fixups -Date: February 2014 -KernelVersion: 3.15 -Contact: netdev@vger.kernel.org -Description: - This attribute contains the boolean value whether a given PHY - device has had any "fixup" workaround running on it, encoded as - a boolean. This information is provided to help troubleshooting - PHY configurations. diff --git a/Documentation/ABI/testing/sysfs-bus-siox b/Documentation/ABI/testing/sysfs-bus-siox index fed7c3765a4e..c2a403f20b90 100644 --- a/Documentation/ABI/testing/sysfs-bus-siox +++ b/Documentation/ABI/testing/sysfs-bus-siox @@ -1,6 +1,6 @@ What: /sys/bus/siox/devices/siox-X/active KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: On reading represents the current state of the bus. If it contains a "0" the bus is stopped and connected devices are @@ -12,7 +12,7 @@ Description: What: /sys/bus/siox/devices/siox-X/device_add KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Write-only file. Write @@ -27,13 +27,13 @@ Description: What: /sys/bus/siox/devices/siox-X/device_remove KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Write-only file. A single write removes the last device in the siox chain. What: /sys/bus/siox/devices/siox-X/poll_interval_ns KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Defines the interval between two poll cycles in nano seconds. Note this is rounded to jiffies on writing. On reading the current value @@ -41,33 +41,33 @@ Description: What: /sys/bus/siox/devices/siox-X-Y/connected KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Read-only value. "0" means the Yth device on siox bus X isn't "connected" i.e. communication with it is not ensured. "1" signals a working connection. What: /sys/bus/siox/devices/siox-X-Y/inbytes KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Read-only value reporting the inbytes value provided to siox-X/device_add What: /sys/bus/siox/devices/siox-X-Y/status_errors KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Counts the number of time intervals when the read status byte doesn't yield the expected value. What: /sys/bus/siox/devices/siox-X-Y/type KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Read-only value reporting the type value provided to siox-X/device_add. What: /sys/bus/siox/devices/siox-X-Y/watchdog KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Read-only value reporting if the watchdog of the siox device is active. "0" means the watchdog is not active and the device is expected to @@ -75,13 +75,13 @@ Description: What: /sys/bus/siox/devices/siox-X-Y/watchdog_errors KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Read-only value reporting the number to time intervals when the watchdog was active. What: /sys/bus/siox/devices/siox-X-Y/outbytes KernelVersion: 4.16 -Contact: Gavin Schenk <g.schenk@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> +Contact: Thorsten Scherer <t.scherer@eckelmann.de>, Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Description: Read-only value reporting the outbytes value provided to siox-X/device_add. diff --git a/Documentation/ABI/testing/sysfs-class-net-phydev b/Documentation/ABI/testing/sysfs-class-net-phydev index 6ebabfb27912..2a5723343aba 100644 --- a/Documentation/ABI/testing/sysfs-class-net-phydev +++ b/Documentation/ABI/testing/sysfs-class-net-phydev @@ -11,24 +11,31 @@ Date: February 2014 KernelVersion: 3.15 Contact: netdev@vger.kernel.org Description: - Boolean value indicating whether the PHY device has - any fixups registered against it (phy_register_fixup) + This attribute contains the boolean value whether a given PHY + device has had any "fixup" workaround running on it, encoded as + a boolean. This information is provided to help troubleshooting + PHY configurations. What: /sys/class/mdio_bus/<bus>/<device>/phy_id Date: November 2012 KernelVersion: 3.8 Contact: netdev@vger.kernel.org Description: - 32-bit hexadecimal value corresponding to the PHY device's OUI, - model and revision number. + This attribute contains the 32-bit PHY Identifier as reported + by the device during bus enumeration, encoded in hexadecimal. + This ID is used to match the device with the appropriate + driver. What: /sys/class/mdio_bus/<bus>/<device>/phy_interface Date: February 2014 KernelVersion: 3.15 Contact: netdev@vger.kernel.org Description: - String value indicating the PHY interface, possible - values are:. + This attribute contains the PHY interface as configured by the + Ethernet driver during bus enumeration, encoded in string. + This interface mode is used to configure the Ethernet MAC with the + appropriate mode for its data lines to the PHY hardware. + Possible values are: <empty> (not available), mii, gmii, sgmii, tbi, rev-mii, rmii, rgmii, rgmii-id, rgmii-rxid, rgmii-txid, rtbi, smii xgmii, moca, qsgmii, trgmii, 1000base-x, 2500base-x, rxaui, diff --git a/Documentation/ABI/testing/sysfs-class-net-qmi b/Documentation/ABI/testing/sysfs-class-net-qmi index 7122d6264c49..c310db4ccbc2 100644 --- a/Documentation/ABI/testing/sysfs-class-net-qmi +++ b/Documentation/ABI/testing/sysfs-class-net-qmi @@ -29,7 +29,7 @@ Contact: Bjørn Mork <bjorn@mork.no> Description: Unsigned integer. - Write a number ranging from 1 to 127 to add a qmap mux + Write a number ranging from 1 to 254 to add a qmap mux based network device, supported by recent Qualcomm based modems. @@ -46,5 +46,5 @@ Contact: Bjørn Mork <bjorn@mork.no> Description: Unsigned integer. - Write a number ranging from 1 to 127 to delete a previously + Write a number ranging from 1 to 254 to delete a previously created qmap mux based network device. diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu index 1528239f69b2..923fe2001472 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -538,3 +538,26 @@ Description: Intel Energy and Performance Bias Hint (EPB) This attribute is present for all online CPUs supporting the Intel EPB feature. + +What: /sys/devices/system/cpu/umwait_control + /sys/devices/system/cpu/umwait_control/enable_c02 + /sys/devices/system/cpu/umwait_control/max_time +Date: May 2019 +Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org> +Description: Umwait control + + enable_c02: Read/write interface to control umwait C0.2 state + Read returns C0.2 state status: + 0: C0.2 is disabled + 1: C0.2 is enabled + + Write 'y' or '1' or 'on' to enable C0.2 state. + Write 'n' or '0' or 'off' to disable C0.2 state. + + The interface is case insensitive. + + max_time: Read/write interface to control umwait maximum time + in TSC-quanta that the CPU can reside in either C0.1 + or C0.2 state. The time is an unsigned 32-bit number. + Note that a value of zero means there is no limit. + Low order two bits must be zero. diff --git a/Documentation/RCU/rcuref.txt b/Documentation/RCU/rcuref.txt index 613033ff2b9b..5e6429d66c24 100644 --- a/Documentation/RCU/rcuref.txt +++ b/Documentation/RCU/rcuref.txt @@ -12,6 +12,7 @@ please read on. Reference counting on elements of lists which are protected by traditional reader/writer spinlocks or semaphores are straightforward: +CODE LISTING A: 1. 2. add() search_and_reference() { { @@ -28,7 +29,8 @@ add() search_and_reference() release_referenced() delete() { { ... write_lock(&list_lock); - atomic_dec(&el->rc, relfunc) ... + if(atomic_dec_and_test(&el->rc)) ... + kfree(el); ... remove_element } write_unlock(&list_lock); ... @@ -44,6 +46,7 @@ search_and_reference() could potentially hold reference to an element which has already been deleted from the list/array. Use atomic_inc_not_zero() in this scenario as follows: +CODE LISTING B: 1. 2. add() search_and_reference() { { @@ -79,6 +82,7 @@ search_and_reference() code path. In such cases, the atomic_dec_and_test() may be moved from delete() to el_free() as follows: +CODE LISTING C: 1. 2. add() search_and_reference() { { @@ -114,6 +118,17 @@ element can therefore safely be freed. This in turn guarantees that if any reader finds the element, that reader may safely acquire a reference without checking the value of the reference counter. +A clear advantage of the RCU-based pattern in listing C over the one +in listing B is that any call to search_and_reference() that locates +a given object will succeed in obtaining a reference to that object, +even given a concurrent invocation of delete() for that same object. +Similarly, a clear advantage of both listings B and C over listing A is +that a call to delete() is not delayed even if there are an arbitrarily +large number of calls to search_and_reference() searching for the same +object that delete() was invoked on. Instead, all that is delayed is +the eventual invocation of kfree(), which is usually not a problem on +modern computer systems, even the small ones. + In cases where delete() can sleep, synchronize_rcu() can be called from delete(), so that el_free() can be subsumed into delete as follows: @@ -130,3 +145,7 @@ delete() kfree(el); ... } + +As additional examples in the kernel, the pattern in listing C is used by +reference counting of struct pid, while the pattern in listing B is used by +struct posix_acl. diff --git a/Documentation/RCU/stallwarn.txt b/Documentation/RCU/stallwarn.txt index 1ab70c37921f..13e88fc00f01 100644 --- a/Documentation/RCU/stallwarn.txt +++ b/Documentation/RCU/stallwarn.txt @@ -153,7 +153,7 @@ rcupdate.rcu_task_stall_timeout This boot/sysfs parameter controls the RCU-tasks stall warning interval. A value of zero or less suppresses RCU-tasks stall warnings. A positive value sets the stall-warning interval - in jiffies. An RCU-tasks stall warning starts with the line: + in seconds. An RCU-tasks stall warning starts with the line: INFO: rcu_tasks detected stalls on tasks: diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt index 981651a8b65d..7e1a8721637a 100644 --- a/Documentation/RCU/whatisRCU.txt +++ b/Documentation/RCU/whatisRCU.txt @@ -212,7 +212,7 @@ synchronize_rcu() rcu_assign_pointer() - typeof(p) rcu_assign_pointer(p, typeof(p) v); + void rcu_assign_pointer(p, typeof(p) v); Yes, rcu_assign_pointer() -is- implemented as a macro, though it would be cool to be able to declare a function in this manner. @@ -220,9 +220,9 @@ rcu_assign_pointer() The updater uses this function to assign a new value to an RCU-protected pointer, in order to safely communicate the change - in value from the updater to the reader. This function returns - the new value, and also executes any memory-barrier instructions - required for a given CPU architecture. + in value from the updater to the reader. This macro does not + evaluate to an rvalue, but it does execute any memory-barrier + instructions required for a given CPU architecture. Perhaps just as important, it serves to document (1) which pointers are protected by RCU and (2) the point at which a diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 88e746074252..cf88c1f98270 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -177,6 +177,15 @@ cgroup v2 currently supports the following mount options. ignored on non-init namespace mounts. Please refer to the Delegation section for details. + memory_localevents + + Only populate memory.events with data for the current cgroup, + and not any subtrees. This is legacy behaviour, the default + behaviour without this option is to include subtree counts. + This option is system wide and can only be set on mount or + modified through remount from the init namespace. The mount + option is ignored on non-init namespace mounts. + Organizing Processes and Threads -------------------------------- diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 138f6664b2e2..e6e806285703 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -478,7 +478,7 @@ others). ccw_timeout_log [S390] - See Documentation/s390/CommonIO for details. + See Documentation/s390/common_io.rst for details. cgroup_disable= [KNL] Disable a particular controller Format: {name of the controller(s) to disable} @@ -516,7 +516,7 @@ /selinux/checkreqprot. cio_ignore= [S390] - See Documentation/s390/CommonIO for details. + See Documentation/s390/common_io.rst for details. clk_ignore_unused [CLK] Prevents the clock framework from automatically gating @@ -3752,6 +3752,12 @@ the propagation of recent CPU-hotplug changes up the rcu_node combining tree. + rcutree.use_softirq= [KNL] + If set to zero, move all RCU_SOFTIRQ processing to + per-CPU rcuc kthreads. Defaults to a non-zero + value, meaning that RCU_SOFTIRQ is used by default. + Specify rcutree.use_softirq=0 to use rcuc kthreads. + rcutree.rcu_fanout_exact= [KNL] Disable autobalancing of the rcu_node combining tree. This is used by rcutorture, and might @@ -5100,13 +5106,12 @@ targets for exploits that can control RIP. emulate [default] Vsyscalls turn into traps and are - emulated reasonably safely. + emulated reasonably safely. The vsyscall + page is readable. - native Vsyscalls are native syscall instructions. - This is a little bit faster than trapping - and makes a few dynamic recompilers work - better than they would in emulation mode. - It also makes exploits much easier to write. + xonly Vsyscalls turn into traps and are + emulated reasonably safely. The vsyscall + page is not readable. none Vsyscalls don't work at all. This makes them quite hard to use for exploits but diff --git a/Documentation/admin-guide/mm/index.rst b/Documentation/admin-guide/mm/index.rst index 8edb35f11317..ddf8d8d33377 100644 --- a/Documentation/admin-guide/mm/index.rst +++ b/Documentation/admin-guide/mm/index.rst @@ -31,6 +31,7 @@ the Linux memory management. ksm memory-hotplug numa_memory_policy + numaperf pagemap soft-dirty transhuge diff --git a/Documentation/admin-guide/mm/numaperf.rst b/Documentation/admin-guide/mm/numaperf.rst index b79f70c04397..c067ed145158 100644 --- a/Documentation/admin-guide/mm/numaperf.rst +++ b/Documentation/admin-guide/mm/numaperf.rst @@ -15,7 +15,7 @@ characteristics. Some memory may share the same node as a CPU, and others are provided as memory only nodes. While memory only nodes do not provide CPUs, they may still be local to one or more compute nodes relative to other nodes. The following diagram shows one such example of two compute -nodes with local memory and a memory only node for each of compute node: +nodes with local memory and a memory only node for each of compute node:: +------------------+ +------------------+ | Compute Node 0 +-----+ Compute Node 1 | diff --git a/Documentation/arm64/elf_hwcaps.txt b/Documentation/arm64/elf_hwcaps.txt index b73a2519ecf2..5ae2ef2c12f3 100644 --- a/Documentation/arm64/elf_hwcaps.txt +++ b/Documentation/arm64/elf_hwcaps.txt @@ -207,6 +207,10 @@ HWCAP_FLAGM Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0001. +HWCAP2_FLAGM2 + + Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0010. + HWCAP_SSBS Functionality implied by ID_AA64PFR1_EL1.SSBS == 0b0010. @@ -223,6 +227,10 @@ HWCAP_PACG ID_AA64ISAR1_EL1.GPI == 0b0001, as described by Documentation/arm64/pointer-authentication.txt. +HWCAP2_FRINT + + Functionality implied by ID_AA64ISAR1_EL1.FRINTTS == 0b0001. + 4. Unused AT_HWCAP bits ----------------------- diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt index 68d9b74fd751..2735462d5958 100644 --- a/Documentation/arm64/silicon-errata.txt +++ b/Documentation/arm64/silicon-errata.txt @@ -58,13 +58,14 @@ stable kernels. | ARM | Cortex-A72 | #853709 | N/A | | ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 | | ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 | -| ARM | Cortex-A76 | #1188873 | ARM64_ERRATUM_1188873 | +| ARM | Cortex-A76 | #1188873,1418040| ARM64_ERRATUM_1418040 | | ARM | Cortex-A76 | #1165522 | ARM64_ERRATUM_1165522 | | ARM | Cortex-A76 | #1286807 | ARM64_ERRATUM_1286807 | -| ARM | Neoverse-N1 | #1188873 | ARM64_ERRATUM_1188873 | -| ARM | MMU-500 | #841119,#826419 | N/A | +| ARM | Cortex-A76 | #1463225 | ARM64_ERRATUM_1463225 | +| ARM | Neoverse-N1 | #1188873,1418040| ARM64_ERRATUM_1418040 | +| ARM | MMU-500 | #841119,826419 | N/A | | | | | | -| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 | +| Cavium | ThunderX ITS | #22375,24313 | CAVIUM_ERRATUM_22375 | | Cavium | ThunderX ITS | #23144 | CAVIUM_ERRATUM_23144 | | Cavium | ThunderX GICv3 | #23154 | CAVIUM_ERRATUM_23154 | | Cavium | ThunderX Core | #27456 | CAVIUM_ERRATUM_27456 | diff --git a/Documentation/arm64/sve.txt b/Documentation/arm64/sve.txt index 9940e924a47e..5689fc9a976a 100644 --- a/Documentation/arm64/sve.txt +++ b/Documentation/arm64/sve.txt @@ -56,6 +56,18 @@ model features for SVE is included in Appendix A. is to connect to a target process first and then attempt a ptrace(PTRACE_GETREGSET, pid, NT_ARM_SVE, &iov). +* Whenever SVE scalable register values (Zn, Pn, FFR) are exchanged in memory + between userspace and the kernel, the register value is encoded in memory in + an endianness-invariant layout, with bits [(8 * i + 7) : (8 * i)] encoded at + byte offset i from the start of the memory representation. This affects for + example the signal frame (struct sve_context) and ptrace interface + (struct user_sve_header) and associated data. + + Beware that on big-endian systems this results in a different byte order than + for the FPSIMD V-registers, which are stored as single host-endian 128-bit + values, with bits [(127 - 8 * i) : (120 - 8 * i)] of the register encoded at + byte offset i. (struct fpsimd_context, struct user_fpsimd_state). + 2. Vector length terminology ----------------------------- @@ -124,6 +136,10 @@ the SVE instruction set architecture. size and layout. Macros SVE_SIG_* are defined [1] to facilitate access to the members. +* Each scalable register (Zn, Pn, FFR) is stored in an endianness-invariant + layout, with bits [(8 * i + 7) : (8 * i)] stored at byte offset i from the + start of the register's representation in memory. + * If the SVE context is too big to fit in sigcontext.__reserved[], then extra space is allocated on the stack, an extra_context record is written in __reserved[] referencing this space. sve_context is then written in the diff --git a/Documentation/atomic_t.txt b/Documentation/atomic_t.txt index dca3fb0554db..0ab747e0d5ac 100644 --- a/Documentation/atomic_t.txt +++ b/Documentation/atomic_t.txt @@ -81,9 +81,11 @@ Non-RMW ops: The non-RMW ops are (typically) regular LOADs and STOREs and are canonically implemented using READ_ONCE(), WRITE_ONCE(), smp_load_acquire() and -smp_store_release() respectively. +smp_store_release() respectively. Therefore, if you find yourself only using +the Non-RMW operations of atomic_t, you do not in fact need atomic_t at all +and are doing it wrong. -The one detail to this is that atomic_set{}() should be observable to the RMW +A subtle detail of atomic_set{}() is that it should be observable to the RMW ops. That is: C atomic-set @@ -187,13 +189,22 @@ The barriers: smp_mb__{before,after}_atomic() -only apply to the RMW ops and can be used to augment/upgrade the ordering -inherent to the used atomic op. These barriers provide a full smp_mb(). +only apply to the RMW atomic ops and can be used to augment/upgrade the +ordering inherent to the op. These barriers act almost like a full smp_mb(): +smp_mb__before_atomic() orders all earlier accesses against the RMW op +itself and all accesses following it, and smp_mb__after_atomic() orders all +later accesses against the RMW op and all accesses preceding it. However, +accesses between the smp_mb__{before,after}_atomic() and the RMW op are not +ordered, so it is advisable to place the barrier right next to the RMW atomic +op whenever possible. These helper barriers exist because architectures have varying implicit ordering on their SMP atomic primitives. For example our TSO architectures provide full ordered atomics and these barriers are no-ops. +NOTE: when the atomic RmW ops are fully ordered, they should also imply a +compiler barrier. + Thus: atomic_fetch_add(); @@ -212,7 +223,9 @@ Further, while something like: atomic_dec(&X); is a 'typical' RELEASE pattern, the barrier is strictly stronger than -a RELEASE. Similarly for something like: +a RELEASE because it orders preceding instructions against both the read +and write parts of the atomic_dec(), and against all following instructions +as well. Similarly, something like: atomic_inc(&X); smp_mb__after_atomic(); @@ -244,7 +257,8 @@ strictly stronger than ACQUIRE. As illustrated: This should not happen; but a hypothetical atomic_inc_acquire() -- (void)atomic_fetch_inc_acquire() for instance -- would allow the outcome, -since then: +because it would not order the W part of the RMW against the following +WRITE_ONCE. Thus: P1 P2 diff --git a/Documentation/block/switching-sched.txt b/Documentation/block/switching-sched.txt index 3b2612e342f1..7977f6fb8b20 100644 --- a/Documentation/block/switching-sched.txt +++ b/Documentation/block/switching-sched.txt @@ -13,11 +13,9 @@ you can do so by typing: # mount none /sys -t sysfs -As of the Linux 2.6.10 kernel, it is now possible to change the -IO scheduler for a given block device on the fly (thus making it possible, -for instance, to set the CFQ scheduler for the system default, but -set a specific device to use the deadline or noop schedulers - which -can improve that device's throughput). +It is possible to change the IO scheduler for a given block device on +the fly to select one of mq-deadline, none, bfq, or kyber schedulers - +which can improve that device's throughput. To set a specific scheduler, simply do this: @@ -30,8 +28,8 @@ The list of defined schedulers can be found by simply doing a "cat /sys/block/DEV/queue/scheduler" - the list of valid names will be displayed, with the currently selected scheduler in brackets: -# cat /sys/block/hda/queue/scheduler -noop deadline [cfq] -# echo deadline > /sys/block/hda/queue/scheduler -# cat /sys/block/hda/queue/scheduler -noop [deadline] cfq +# cat /sys/block/sda/queue/scheduler +[mq-deadline] kyber bfq none +# echo none >/sys/block/sda/queue/scheduler +# cat /sys/block/sda/queue/scheduler +[none] mq-deadline kyber bfq diff --git a/Documentation/bpf/btf.rst b/Documentation/bpf/btf.rst index 8820360d00da..35d83e24dbdb 100644 --- a/Documentation/bpf/btf.rst +++ b/Documentation/bpf/btf.rst @@ -131,7 +131,7 @@ The following sections detail encoding of each kind. ``btf_type`` is followed by a ``u32`` with the following bits arrangement:: #define BTF_INT_ENCODING(VAL) (((VAL) & 0x0f000000) >> 24) - #define BTF_INT_OFFSET(VAL) (((VAL & 0x00ff0000)) >> 16) + #define BTF_INT_OFFSET(VAL) (((VAL) & 0x00ff0000) >> 16) #define BTF_INT_BITS(VAL) ((VAL) & 0x000000ff) The ``BTF_INT_ENCODING`` has the following attributes:: diff --git a/Documentation/cgroup-v1/blkio-controller.txt b/Documentation/cgroup-v1/blkio-controller.txt index 673dc34d3f78..d1a1b7bdd03a 100644 --- a/Documentation/cgroup-v1/blkio-controller.txt +++ b/Documentation/cgroup-v1/blkio-controller.txt @@ -8,61 +8,13 @@ both at leaf nodes as well as at intermediate nodes in a storage hierarchy. Plan is to use the same cgroup based management interface for blkio controller and based on user options switch IO policies in the background. -Currently two IO control policies are implemented. First one is proportional -weight time based division of disk policy. It is implemented in CFQ. Hence -this policy takes effect only on leaf nodes when CFQ is being used. The second -one is throttling policy which can be used to specify upper IO rate limits -on devices. This policy is implemented in generic block layer and can be -used on leaf nodes as well as higher level logical devices like device mapper. +One IO control policy is throttling policy which can be used to +specify upper IO rate limits on devices. This policy is implemented in +generic block layer and can be used on leaf nodes as well as higher +level logical devices like device mapper. HOWTO ===== -Proportional Weight division of bandwidth ------------------------------------------ -You can do a very simple testing of running two dd threads in two different -cgroups. Here is what you can do. - -- Enable Block IO controller - CONFIG_BLK_CGROUP=y - -- Enable group scheduling in CFQ - CONFIG_CFQ_GROUP_IOSCHED=y - -- Compile and boot into kernel and mount IO controller (blkio); see - cgroups.txt, Why are cgroups needed?. - - mount -t tmpfs cgroup_root /sys/fs/cgroup - mkdir /sys/fs/cgroup/blkio - mount -t cgroup -o blkio none /sys/fs/cgroup/blkio - -- Create two cgroups - mkdir -p /sys/fs/cgroup/blkio/test1/ /sys/fs/cgroup/blkio/test2 - -- Set weights of group test1 and test2 - echo 1000 > /sys/fs/cgroup/blkio/test1/blkio.weight - echo 500 > /sys/fs/cgroup/blkio/test2/blkio.weight - -- Create two same size files (say 512MB each) on same disk (file1, file2) and - launch two dd threads in different cgroup to read those files. - - sync - echo 3 > /proc/sys/vm/drop_caches - - dd if=/mnt/sdb/zerofile1 of=/dev/null & - echo $! > /sys/fs/cgroup/blkio/test1/tasks - cat /sys/fs/cgroup/blkio/test1/tasks - - dd if=/mnt/sdb/zerofile2 of=/dev/null & - echo $! > /sys/fs/cgroup/blkio/test2/tasks - cat /sys/fs/cgroup/blkio/test2/tasks - -- At macro level, first dd should finish first. To get more precise data, keep - on looking at (with the help of script), at blkio.disk_time and - blkio.disk_sectors files of both test1 and test2 groups. This will tell how - much disk time (in milliseconds), each group got and how many sectors each - group dispatched to the disk. We provide fairness in terms of disk time, so - ideally io.disk_time of cgroups should be in proportion to the weight. - Throttling/Upper Limit policy ----------------------------- - Enable Block IO controller @@ -94,7 +46,7 @@ Throttling/Upper Limit policy Hierarchical Cgroups ==================== -Both CFQ and throttling implement hierarchy support; however, +Throttling implements hierarchy support; however, throttling's hierarchy support is enabled iff "sane_behavior" is enabled from cgroup side, which currently is a development option and not publicly available. @@ -107,9 +59,8 @@ If somebody created a hierarchy like as follows. | test3 -CFQ by default and throttling with "sane_behavior" will handle the -hierarchy correctly. For details on CFQ hierarchy support, refer to -Documentation/block/cfq-iosched.txt. For throttling, all limits apply +Throttling with "sane_behavior" will handle the +hierarchy correctly. For throttling, all limits apply to the whole subtree while all statistics are local to the IOs directly generated by tasks in that cgroup. @@ -130,10 +81,6 @@ CONFIG_DEBUG_BLK_CGROUP - Debug help. Right now some additional stats file show up in cgroup if this option is enabled. -CONFIG_CFQ_GROUP_IOSCHED - - Enables group scheduling in CFQ. Currently only 1 level of group - creation is allowed. - CONFIG_BLK_DEV_THROTTLING - Enable block device throttling support in block layer. @@ -344,32 +291,3 @@ Common files among various policies - blkio.reset_stats - Writing an int to this file will result in resetting all the stats for that cgroup. - -CFQ sysfs tunable -================= -/sys/block/<disk>/queue/iosched/slice_idle ------------------------------------------- -On a faster hardware CFQ can be slow, especially with sequential workload. -This happens because CFQ idles on a single queue and single queue might not -drive deeper request queue depths to keep the storage busy. In such scenarios -one can try setting slice_idle=0 and that would switch CFQ to IOPS -(IO operations per second) mode on NCQ supporting hardware. - -That means CFQ will not idle between cfq queues of a cfq group and hence be -able to driver higher queue depth and achieve better throughput. That also -means that cfq provides fairness among groups in terms of IOPS and not in -terms of disk time. - -/sys/block/<disk>/queue/iosched/group_idle ------------------------------------------- -If one disables idling on individual cfq queues and cfq service trees by -setting slice_idle=0, group_idle kicks in. That means CFQ will still idle -on the group in an attempt to provide fairness among groups. - -By default group_idle is same as slice_idle and does not do anything if -slice_idle is enabled. - -One can experience an overall throughput drop if you have created multiple -groups and put applications in that group which are not driving enough -IO to keep disk busy. In that case set group_idle=0, and CFQ will not idle -on individual groups and throughput should improve. diff --git a/Documentation/cgroup-v1/hugetlb.txt b/Documentation/cgroup-v1/hugetlb.txt index 106245c3aecc..1260e5369b9b 100644 --- a/Documentation/cgroup-v1/hugetlb.txt +++ b/Documentation/cgroup-v1/hugetlb.txt @@ -32,14 +32,18 @@ Brief summary of control files hugetlb.<hugepagesize>.usage_in_bytes # show current usage for "hugepagesize" hugetlb hugetlb.<hugepagesize>.failcnt # show the number of allocation failure due to HugeTLB limit -For a system supporting two hugepage size (16M and 16G) the control +For a system supporting three hugepage sizes (64k, 32M and 1G), the control files include: -hugetlb.16GB.limit_in_bytes -hugetlb.16GB.max_usage_in_bytes -hugetlb.16GB.usage_in_bytes -hugetlb.16GB.failcnt -hugetlb.16MB.limit_in_bytes -hugetlb.16MB.max_usage_in_bytes -hugetlb.16MB.usage_in_bytes -hugetlb.16MB.failcnt +hugetlb.1GB.limit_in_bytes +hugetlb.1GB.max_usage_in_bytes +hugetlb.1GB.usage_in_bytes +hugetlb.1GB.failcnt +hugetlb.64KB.limit_in_bytes +hugetlb.64KB.max_usage_in_bytes +hugetlb.64KB.usage_in_bytes +hugetlb.64KB.failcnt +hugetlb.32MB.limit_in_bytes +hugetlb.32MB.max_usage_in_bytes +hugetlb.32MB.usage_in_bytes +hugetlb.32MB.failcnt diff --git a/Documentation/conf.py b/Documentation/conf.py index 72647a38b5c2..7ace3f8852bd 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -37,7 +37,7 @@ needs_sphinx = '1.3' extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig'] # The name of the math extension changed on Sphinx 1.4 -if major == 1 and minor > 3: +if (major == 1 and minor > 3) or (major > 1): extensions.append("sphinx.ext.imgmath") else: extensions.append("sphinx.ext.pngmath") diff --git a/Documentation/core-api/circular-buffers.rst b/Documentation/core-api/circular-buffers.rst index 53e51caa3347..50966f66e398 100644 --- a/Documentation/core-api/circular-buffers.rst +++ b/Documentation/core-api/circular-buffers.rst @@ -3,7 +3,7 @@ Circular Buffers ================ :Author: David Howells <dhowells@redhat.com> -:Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com> +:Author: Paul E. McKenney <paulmck@linux.ibm.com> Linux provides a number of features that can be used to implement circular diff --git a/Documentation/core-api/timekeeping.rst b/Documentation/core-api/timekeeping.rst index 93cbeb9daec0..20ee447a50f3 100644 --- a/Documentation/core-api/timekeeping.rst +++ b/Documentation/core-api/timekeeping.rst @@ -65,7 +65,7 @@ different format depending on what is required by the user: .. c:function:: u64 ktime_get_ns( void ) u64 ktime_get_boottime_ns( void ) u64 ktime_get_real_ns( void ) - u64 ktime_get_tai_ns( void ) + u64 ktime_get_clocktai_ns( void ) u64 ktime_get_raw_ns( void ) Same as the plain ktime_get functions, but returning a u64 number @@ -99,16 +99,20 @@ Coarse and fast_ns access Some additional variants exist for more specialized cases: -.. c:function:: ktime_t ktime_get_coarse_boottime( void ) +.. c:function:: ktime_t ktime_get_coarse( void ) + ktime_t ktime_get_coarse_boottime( void ) ktime_t ktime_get_coarse_real( void ) ktime_t ktime_get_coarse_clocktai( void ) - ktime_t ktime_get_coarse_raw( void ) + +.. c:function:: u64 ktime_get_coarse_ns( void ) + u64 ktime_get_coarse_boottime_ns( void ) + u64 ktime_get_coarse_real_ns( void ) + u64 ktime_get_coarse_clocktai_ns( void ) .. c:function:: void ktime_get_coarse_ts64( struct timespec64 * ) void ktime_get_coarse_boottime_ts64( struct timespec64 * ) void ktime_get_coarse_real_ts64( struct timespec64 * ) void ktime_get_coarse_clocktai_ts64( struct timespec64 * ) - void ktime_get_coarse_raw_ts64( struct timespec64 * ) These are quicker than the non-coarse versions, but less accurate, corresponding to CLOCK_MONONOTNIC_COARSE and CLOCK_REALTIME_COARSE diff --git a/Documentation/cputopology.txt b/Documentation/cputopology.txt index cb61277e2308..b90dafcc8237 100644 --- a/Documentation/cputopology.txt +++ b/Documentation/cputopology.txt @@ -12,6 +12,12 @@ physical_package_id: socket number, but the actual value is architecture and platform dependent. +die_id: + + the CPU die ID of cpuX. Typically it is the hardware platform's + identifier (rather than the kernel's). The actual value is + architecture and platform dependent. + core_id: the CPU core ID of cpuX. Typically it is the hardware platform's @@ -30,25 +36,33 @@ drawer_id: identifier (rather than the kernel's). The actual value is architecture and platform dependent. -thread_siblings: +core_cpus: - internal kernel map of cpuX's hardware threads within the same - core as cpuX. + internal kernel map of CPUs within the same core. + (deprecated name: "thread_siblings") -thread_siblings_list: +core_cpus_list: - human-readable list of cpuX's hardware threads within the same - core as cpuX. + human-readable list of CPUs within the same core. + (deprecated name: "thread_siblings_list"); -core_siblings: +package_cpus: - internal kernel map of cpuX's hardware threads within the same - physical_package_id. + internal kernel map of the CPUs sharing the same physical_package_id. + (deprecated name: "core_siblings") -core_siblings_list: +package_cpus_list: - human-readable list of cpuX's hardware threads within the same - physical_package_id. + human-readable list of CPUs sharing the same physical_package_id. + (deprecated name: "core_siblings_list") + +die_cpus: + + internal kernel map of CPUs within the same die. + +die_cpus_list: + + human-readable list of CPUs within the same die. book_siblings: @@ -81,11 +95,13 @@ For an architecture to support this feature, it must define some of these macros in include/asm-XXX/topology.h:: #define topology_physical_package_id(cpu) + #define topology_die_id(cpu) #define topology_core_id(cpu) #define topology_book_id(cpu) #define topology_drawer_id(cpu) #define topology_sibling_cpumask(cpu) #define topology_core_cpumask(cpu) + #define topology_die_cpumask(cpu) #define topology_book_cpumask(cpu) #define topology_drawer_cpumask(cpu) @@ -99,9 +115,11 @@ provides default definitions for any of the above macros that are not defined by include/asm-XXX/topology.h: 1) topology_physical_package_id: -1 -2) topology_core_id: 0 -3) topology_sibling_cpumask: just the given CPU -4) topology_core_cpumask: just the given CPU +2) topology_die_id: -1 +3) topology_core_id: 0 +4) topology_sibling_cpumask: just the given CPU +5) topology_core_cpumask: just the given CPU +6) topology_die_cpumask: just the given CPU For architectures that don't support books (CONFIG_SCHED_BOOK) there are no default definitions for topology_book_id() and topology_book_cpumask(). diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile index 63b139f9ae28..8a2774b5834b 100644 --- a/Documentation/devicetree/bindings/Makefile +++ b/Documentation/devicetree/bindings/Makefile @@ -5,7 +5,7 @@ DT_MK_SCHEMA ?= dt-mk-schema DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u) quiet_cmd_chk_binding = CHKDT $(patsubst $(srctree)/%,%,$<) - cmd_chk_binding = $(DT_DOC_CHECKER) $< ; \ + cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \ $(DT_EXTRACT_EX) $< > $@ $(obj)/%.example.dts: $(src)/%.yaml FORCE diff --git a/Documentation/devicetree/bindings/arm/arm-boards b/Documentation/devicetree/bindings/arm/arm-boards index b6e810c2781a..abff8d834a6a 100644 --- a/Documentation/devicetree/bindings/arm/arm-boards +++ b/Documentation/devicetree/bindings/arm/arm-boards @@ -216,7 +216,7 @@ Example: #size-cells = <0>; A57_0: cpu@0 { - compatible = "arm,cortex-a57","arm,armv8"; + compatible = "arm,cortex-a57"; reg = <0x0 0x0>; device_type = "cpu"; enable-method = "psci"; @@ -225,7 +225,7 @@ Example: ..... A53_0: cpu@100 { - compatible = "arm,cortex-a53","arm,armv8"; + compatible = "arm,cortex-a53"; reg = <0x0 0x100>; device_type = "cpu"; enable-method = "psci"; diff --git a/Documentation/devicetree/bindings/arm/cpu-capacity.txt b/Documentation/devicetree/bindings/arm/cpu-capacity.txt index 96fa46cb133c..380e21c5fc7e 100644 --- a/Documentation/devicetree/bindings/arm/cpu-capacity.txt +++ b/Documentation/devicetree/bindings/arm/cpu-capacity.txt @@ -118,7 +118,7 @@ cpus { }; A57_0: cpu@0 { - compatible = "arm,cortex-a57","arm,armv8"; + compatible = "arm,cortex-a57"; reg = <0x0 0x0>; device_type = "cpu"; enable-method = "psci"; @@ -129,7 +129,7 @@ cpus { }; A57_1: cpu@1 { - compatible = "arm,cortex-a57","arm,armv8"; + compatible = "arm,cortex-a57"; reg = <0x0 0x1>; device_type = "cpu"; enable-method = "psci"; @@ -140,7 +140,7 @@ cpus { }; A53_0: cpu@100 { - compatible = "arm,cortex-a53","arm,armv8"; + compatible = "arm,cortex-a53"; reg = <0x0 0x100>; device_type = "cpu"; enable-method = "psci"; @@ -151,7 +151,7 @@ cpus { }; A53_1: cpu@101 { - compatible = "arm,cortex-a53","arm,armv8"; + compatible = "arm,cortex-a53"; reg = <0x0 0x101>; device_type = "cpu"; enable-method = "psci"; @@ -162,7 +162,7 @@ cpus { }; A53_2: cpu@102 { - compatible = "arm,cortex-a53","arm,armv8"; + compatible = "arm,cortex-a53"; reg = <0x0 0x102>; device_type = "cpu"; enable-method = "psci"; @@ -173,7 +173,7 @@ cpus { }; A53_3: cpu@103 { - compatible = "arm,cortex-a53","arm,armv8"; + compatible = "arm,cortex-a53"; reg = <0x0 0x103>; device_type = "cpu"; enable-method = "psci"; diff --git a/Documentation/devicetree/bindings/arm/omap/crossbar.txt b/Documentation/devicetree/bindings/arm/omap/crossbar.txt index 4cd5d873fc3a..a43e4c7aba3d 100644 --- a/Documentation/devicetree/bindings/arm/omap/crossbar.txt +++ b/Documentation/devicetree/bindings/arm/omap/crossbar.txt @@ -41,7 +41,7 @@ Examples: Consumer: ======== See Documentation/devicetree/bindings/interrupt-controller/interrupts.txt and -Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt for +Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml for further details. An interrupt consumer on an SoC using crossbar will use: diff --git a/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt b/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt index 15b48e20a061..a86c83bf9d4e 100644 --- a/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt +++ b/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt @@ -35,7 +35,7 @@ board device tree, including the system base clock, as selected by XOM[0] pin of the SoC. Refer to generic fixed rate clock bindings documentation[1] for more information how to specify these clocks. -[1] Documentation/devicetree/bindings/clock/fixed-clock.txt +[1] Documentation/devicetree/bindings/clock/fixed-clock.yaml Example: Clock controller node: diff --git a/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt b/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt new file mode 100644 index 000000000000..4e82fd575cec --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt @@ -0,0 +1,29 @@ +Amazon's Annapurna Labs Fabric Interrupt Controller + +Required properties: + +- compatible: should be "amazon,al-fic" +- reg: physical base address and size of the registers +- interrupt-controller: identifies the node as an interrupt controller +- #interrupt-cells: must be 2. + First cell defines the index of the interrupt within the controller. + Second cell is used to specify the trigger type and must be one of the + following: + - bits[3:0] trigger type and level flags + 1 = low-to-high edge triggered + 4 = active high level-sensitive +- interrupt-parent: specifies the parent interrupt controller. +- interrupts: describes which input line in the interrupt parent, this + fic's output is connected to. This field property depends on the parent's + binding + +Example: + +amazon_fic: interrupt-controller@0xfd8a8500 { + compatible = "amazon,al-fic"; + interrupt-controller; + #interrupt-cells = <2>; + reg = <0x0 0xfd8a8500 0x0 0x1000>; + interrupt-parent = <&gic>; + interrupts = <GIC_SPI 0x0 IRQ_TYPE_LEVEL_HIGH>; +}; diff --git a/Documentation/devicetree/bindings/interrupt-controller/amlogic,meson-gpio-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/amlogic,meson-gpio-intc.txt index 1502a51548bb..7d531d5fff29 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/amlogic,meson-gpio-intc.txt +++ b/Documentation/devicetree/bindings/interrupt-controller/amlogic,meson-gpio-intc.txt @@ -15,6 +15,7 @@ Required properties: "amlogic,meson-gxbb-gpio-intc" for GXBB SoCs (S905) or "amlogic,meson-gxl-gpio-intc" for GXL SoCs (S905X, S912) "amlogic,meson-axg-gpio-intc" for AXG SoCs (A113D, A113X) + "amlogic,meson-g12a-gpio-intc" for G12A SoCs (S905D2, S905X2, S905Y2) - reg : Specifies base physical address and size of the registers. - interrupt-controller : Identifies the node as an interrupt controller. - #interrupt-cells : Specifies the number of cells needed to encode an diff --git a/Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml b/Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml index 54838d4ea44c..9a47820ef346 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml +++ b/Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml @@ -92,6 +92,8 @@ properties: minItems: 2 maxItems: 4 + ranges: true + interrupts: description: Interrupt source of the parent interrupt controller on secondary GICs, or VGIC maintenance interrupt on primary GIC (see @@ -197,28 +199,28 @@ examples: interrupt-controller@e1101000 { compatible = "arm,gic-400"; #interrupt-cells = <3>; - #address-cells = <2>; - #size-cells = <2>; + #address-cells = <1>; + #size-cells = <1>; interrupt-controller; interrupts = <1 8 0xf04>; - ranges = <0 0 0 0xe1100000 0 0x100000>; - reg = <0x0 0xe1110000 0 0x01000>, - <0x0 0xe112f000 0 0x02000>, - <0x0 0xe1140000 0 0x10000>, - <0x0 0xe1160000 0 0x10000>; + ranges = <0 0xe1100000 0x100000>; + reg = <0xe1110000 0x01000>, + <0xe112f000 0x02000>, + <0xe1140000 0x10000>, + <0xe1160000 0x10000>; - v2m0: v2m@8000 { + v2m0: v2m@80000 { compatible = "arm,gic-v2m-frame"; msi-controller; - reg = <0x0 0x80000 0 0x1000>; + reg = <0x80000 0x1000>; }; //... - v2mN: v2m@9000 { + v2mN: v2m@90000 { compatible = "arm,gic-v2m-frame"; msi-controller; - reg = <0x0 0x90000 0 0x1000>; + reg = <0x90000 0x1000>; }; }; ... diff --git a/Documentation/devicetree/bindings/interrupt-controller/csky,mpintc.txt b/Documentation/devicetree/bindings/interrupt-controller/csky,mpintc.txt index ab921f1698fb..e13405355166 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/csky,mpintc.txt +++ b/Documentation/devicetree/bindings/interrupt-controller/csky,mpintc.txt @@ -6,11 +6,16 @@ C-SKY Multi-processors Interrupt Controller is designed for ck807/ck810/ck860 SMP soc, and it also could be used in non-SMP system. Interrupt number definition: - 0-15 : software irq, and we use 15 as our IPI_IRQ. 16-31 : private irq, and we use 16 as the co-processor timer. 31-1024: common irq for soc ip. +Interrupt triger mode: (Defined in dt-bindings/interrupt-controller/irq.h) + IRQ_TYPE_LEVEL_HIGH (default) + IRQ_TYPE_LEVEL_LOW + IRQ_TYPE_EDGE_RISING + IRQ_TYPE_EDGE_FALLING + ============================= intc node bindings definition ============================= @@ -26,15 +31,22 @@ intc node bindings definition - #interrupt-cells Usage: required Value type: <u32> - Definition: must be <1> + Definition: <2> - interrupt-controller: Usage: required -Examples: +Examples: ("interrupts = <irq_num IRQ_TYPE_XXX>") --------- +#include <dt-bindings/interrupt-controller/irq.h> intc: interrupt-controller { compatible = "csky,mpintc"; - #interrupt-cells = <1>; + #interrupt-cells = <2>; interrupt-controller; }; + + device: device-example { + ... + interrupts = <34 IRQ_TYPE_EDGE_RISING>; + interrupt-parent = <&intc>; + }; diff --git a/Documentation/devicetree/bindings/interrupt-controller/marvell,odmi-controller.txt b/Documentation/devicetree/bindings/interrupt-controller/marvell,odmi-controller.txt index 930fb462fd9f..0ebfc952cb34 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/marvell,odmi-controller.txt +++ b/Documentation/devicetree/bindings/interrupt-controller/marvell,odmi-controller.txt @@ -23,7 +23,7 @@ Required properties: - marvell,spi-base : List of GIC base SPI interrupts, one for each ODMI frame. Those SPI interrupts are 0-based, i.e marvell,spi-base = <128> will use SPI #96. - See Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt + See Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml for details about the GIC Device Tree binding. Example: diff --git a/Documentation/devicetree/bindings/interrupt-controller/renesas,rza1-irqc.txt b/Documentation/devicetree/bindings/interrupt-controller/renesas,rza1-irqc.txt new file mode 100644 index 000000000000..727b7e4cd6e0 --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/renesas,rza1-irqc.txt @@ -0,0 +1,43 @@ +DT bindings for the Renesas RZ/A1 Interrupt Controller + +The RZ/A1 Interrupt Controller is a front-end for the GIC found on Renesas +RZ/A1 and RZ/A2 SoCs: + - IRQ sense select for 8 external interrupts, 1:1-mapped to 8 GIC SPI + interrupts, + - NMI edge select. + +Required properties: + - compatible: Must be "renesas,<soctype>-irqc", and "renesas,rza1-irqc" as + fallback. + Examples with soctypes are: + - "renesas,r7s72100-irqc" (RZ/A1H) + - "renesas,r7s9210-irqc" (RZ/A2M) + - #interrupt-cells: Must be 2 (an interrupt index and flags, as defined + in interrupts.txt in this directory) + - #address-cells: Must be zero + - interrupt-controller: Marks the device as an interrupt controller + - reg: Base address and length of the memory resource used by the interrupt + controller + - interrupt-map: Specifies the mapping from external interrupts to GIC + interrupts + - interrupt-map-mask: Must be <7 0> + +Example: + + irqc: interrupt-controller@fcfef800 { + compatible = "renesas,r7s72100-irqc", "renesas,rza1-irqc"; + #interrupt-cells = <2>; + #address-cells = <0>; + interrupt-controller; + reg = <0xfcfef800 0x6>; + interrupt-map = + <0 0 &gic GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>, + <1 0 &gic GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>, + <2 0 &gic GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>, + <3 0 &gic GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>, + <4 0 &gic GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>, + <5 0 &gic GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>, + <6 0 &gic GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>, + <7 0 &gic GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>; + interrupt-map-mask = <7 0>; + }; diff --git a/Documentation/devicetree/bindings/leds/irled/spi-ir-led.txt b/Documentation/devicetree/bindings/leds/irled/spi-ir-led.txt index 896b6997cf30..21882c8d4b0c 100644 --- a/Documentation/devicetree/bindings/leds/irled/spi-ir-led.txt +++ b/Documentation/devicetree/bindings/leds/irled/spi-ir-led.txt @@ -15,7 +15,7 @@ Optional properties: - power-supply: specifies the power source. It can either be a regulator or a gpio which enables a regulator, i.e. a regulator-fixed as described in - Documentation/devicetree/bindings/regulator/fixed-regulator.txt + Documentation/devicetree/bindings/regulator/fixed-regulator.yaml Example: diff --git a/Documentation/devicetree/bindings/mtd/amlogic,meson-nand.txt b/Documentation/devicetree/bindings/mtd/amlogic,meson-nand.txt index 3983c11e062c..5794ab1147c1 100644 --- a/Documentation/devicetree/bindings/mtd/amlogic,meson-nand.txt +++ b/Documentation/devicetree/bindings/mtd/amlogic,meson-nand.txt @@ -24,7 +24,7 @@ Optional children nodes: Children nodes represent the available nand chips. Other properties: -see Documentation/devicetree/bindings/mtd/nand.txt for generic bindings. +see Documentation/devicetree/bindings/mtd/nand-controller.yaml for generic bindings. Example demonstrate on AXG SoC: diff --git a/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt b/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt index bcda1dfc4bac..0b7c3738b66c 100644 --- a/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt +++ b/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt @@ -101,12 +101,12 @@ Required properties: number (e.g., 0, 1, 2, etc.) - #address-cells : see partition.txt - #size-cells : see partition.txt -- nand-ecc-strength : see nand.txt -- nand-ecc-step-size : must be 512 or 1024. See nand.txt +- nand-ecc-strength : see nand-controller.yaml +- nand-ecc-step-size : must be 512 or 1024. See nand-controller.yaml Optional properties: - nand-on-flash-bbt : boolean, to enable the on-flash BBT for this - chip-select. See nand.txt + chip-select. See nand-controller.yaml - brcm,nand-oob-sector-size : integer, to denote the spare area sector size expected for the ECC layout in use. This size, in addition to the strength and step-size, diff --git a/Documentation/devicetree/bindings/mtd/denali-nand.txt b/Documentation/devicetree/bindings/mtd/denali-nand.txt index b14b6751c2f3..b32aed1db46d 100644 --- a/Documentation/devicetree/bindings/mtd/denali-nand.txt +++ b/Documentation/devicetree/bindings/mtd/denali-nand.txt @@ -22,16 +22,16 @@ Sub-nodes: select is connected. Optional properties: - - nand-ecc-step-size: see nand.txt for details. + - nand-ecc-step-size: see nand-controller.yaml for details. If present, the value must be 512 for "altr,socfpga-denali-nand" 1024 for "socionext,uniphier-denali-nand-v5a" 1024 for "socionext,uniphier-denali-nand-v5b" - - nand-ecc-strength: see nand.txt for details. Valid values are: + - nand-ecc-strength: see nand-controller.yaml for details. Valid values are: 8, 15 for "altr,socfpga-denali-nand" 8, 16, 24 for "socionext,uniphier-denali-nand-v5a" 8, 16 for "socionext,uniphier-denali-nand-v5b" - - nand-ecc-maximize: see nand.txt for details + - nand-ecc-maximize: see nand-controller.yaml for details The chip nodes may optionally contain sub-nodes describing partitions of the address space. See partition.txt for more detail. diff --git a/Documentation/devicetree/bindings/mtd/fsmc-nand.txt b/Documentation/devicetree/bindings/mtd/fsmc-nand.txt index 32636eb77304..6762d3c4d5a4 100644 --- a/Documentation/devicetree/bindings/mtd/fsmc-nand.txt +++ b/Documentation/devicetree/bindings/mtd/fsmc-nand.txt @@ -30,9 +30,9 @@ Optional properties: command is asserted. Zero means one cycle, 255 means 256 cycles. - bank: default NAND bank to use (0-3 are valid, 0 is the default). -- nand-ecc-mode : see nand.txt -- nand-ecc-strength : see nand.txt -- nand-ecc-step-size : see nand.txt +- nand-ecc-mode : see nand-controller.yaml +- nand-ecc-strength : see nand-controller.yaml +- nand-ecc-step-size : see nand-controller.yaml Can support 1-bit HW ECC (default) or if stronger correction is required, software-based BCH. diff --git a/Documentation/devicetree/bindings/mtd/gpmc-nand.txt b/Documentation/devicetree/bindings/mtd/gpmc-nand.txt index c059ab74ed88..44919d48d241 100644 --- a/Documentation/devicetree/bindings/mtd/gpmc-nand.txt +++ b/Documentation/devicetree/bindings/mtd/gpmc-nand.txt @@ -8,7 +8,7 @@ explained in a separate documents - please refer to Documentation/devicetree/bindings/memory-controllers/omap-gpmc.txt For NAND specific properties such as ECC modes or bus width, please refer to -Documentation/devicetree/bindings/mtd/nand.txt +Documentation/devicetree/bindings/mtd/nand-controller.yaml Required properties: diff --git a/Documentation/devicetree/bindings/mtd/hisi504-nand.txt b/Documentation/devicetree/bindings/mtd/hisi504-nand.txt index 2e35f0662912..8963983ae7cb 100644 --- a/Documentation/devicetree/bindings/mtd/hisi504-nand.txt +++ b/Documentation/devicetree/bindings/mtd/hisi504-nand.txt @@ -7,7 +7,7 @@ Required properties: NAND controller's registers. The second contains base physical address and size of NAND controller's buffer. - interrupts: Interrupt number for nfc. -- nand-bus-width: See nand.txt. +- nand-bus-width: See nand-controller.yaml. - nand-ecc-mode: Support none and hw ecc mode. - #address-cells: Partition address, should be set 1. - #size-cells: Partition size, should be set 1. diff --git a/Documentation/devicetree/bindings/mtd/marvell-nand.txt b/Documentation/devicetree/bindings/mtd/marvell-nand.txt index e0c790706b9b..a2d9a0f2b683 100644 --- a/Documentation/devicetree/bindings/mtd/marvell-nand.txt +++ b/Documentation/devicetree/bindings/mtd/marvell-nand.txt @@ -36,29 +36,29 @@ Children nodes represent the available NAND chips. Required properties: - reg: shall contain the native Chip Select ids (0-3). -- nand-rb: see nand.txt (0-1). +- nand-rb: see nand-controller.yaml (0-1). Optional properties: - marvell,nand-keep-config: orders the driver not to take the timings from the core and leaving them completely untouched. Bootloader timings will then be used. - label: MTD name. -- nand-on-flash-bbt: see nand.txt. -- nand-ecc-mode: see nand.txt. Will use hardware ECC if not specified. -- nand-ecc-algo: see nand.txt. This property is essentially useful when +- nand-on-flash-bbt: see nand-controller.yaml. +- nand-ecc-mode: see nand-controller.yaml. Will use hardware ECC if not specified. +- nand-ecc-algo: see nand-controller.yaml. This property is essentially useful when not using hardware ECC. Howerver, it may be added when using hardware ECC for clarification but will be ignored by the driver because ECC mode is chosen depending on the page size and the strength required by the NAND chip. This value may be overwritten with nand-ecc-strength property. -- nand-ecc-strength: see nand.txt. -- nand-ecc-step-size: see nand.txt. Marvell's NAND flash controller does +- nand-ecc-strength: see nand-controller.yaml. +- nand-ecc-step-size: see nand-controller.yaml. Marvell's NAND flash controller does use fixed strength (1-bit for Hamming, 16-bit for BCH), so the actual step size will shrink or grow in order to fit the required strength. Step sizes are not completely random for all and follow certain patterns described in AN-379, "Marvell SoC NFC ECC". -See Documentation/devicetree/bindings/mtd/nand.txt for more details on +See Documentation/devicetree/bindings/mtd/nand-controller.yaml for more details on generic bindings. diff --git a/Documentation/devicetree/bindings/mtd/mxc-nand.txt b/Documentation/devicetree/bindings/mtd/mxc-nand.txt index b5833d11c7be..2857c628fba4 100644 --- a/Documentation/devicetree/bindings/mtd/mxc-nand.txt +++ b/Documentation/devicetree/bindings/mtd/mxc-nand.txt @@ -4,9 +4,9 @@ Required properties: - compatible: "fsl,imxXX-nand" - reg: address range of the nfc block - interrupts: irq to be used -- nand-bus-width: see nand.txt -- nand-ecc-mode: see nand.txt -- nand-on-flash-bbt: see nand.txt +- nand-bus-width: see nand-controller.yaml +- nand-ecc-mode: see nand-controller.yaml +- nand-on-flash-bbt: see nand-controller.yaml Example: diff --git a/Documentation/devicetree/bindings/mtd/nvidia-tegra20-nand.txt b/Documentation/devicetree/bindings/mtd/nvidia-tegra20-nand.txt index b2f2ca12f9e6..e737e5beb7bf 100644 --- a/Documentation/devicetree/bindings/mtd/nvidia-tegra20-nand.txt +++ b/Documentation/devicetree/bindings/mtd/nvidia-tegra20-nand.txt @@ -26,14 +26,14 @@ Optional children node properties: "hw" is supported. - nand-ecc-algo: string, algorithm of NAND ECC. Supported values with "hw" ECC mode are: "rs", "bch". -- nand-bus-width : See nand.txt -- nand-on-flash-bbt: See nand.txt +- nand-bus-width : See nand-controller.yaml +- nand-on-flash-bbt: See nand-controller.yaml - nand-ecc-strength: integer representing the number of bits to correct per ECC step (always 512). Supported strength using HW ECC modes are: - RS: 4, 6, 8 - BCH: 4, 8, 14, 16 -- nand-ecc-maximize: See nand.txt +- nand-ecc-maximize: See nand-controller.yaml - nand-is-boot-medium: Makes sure only ECC strengths supported by the boot ROM are chosen. - wp-gpios: GPIO specifier for the write protect pin. diff --git a/Documentation/devicetree/bindings/mtd/oxnas-nand.txt b/Documentation/devicetree/bindings/mtd/oxnas-nand.txt index 56d5c19da41d..2ba07fc8b79c 100644 --- a/Documentation/devicetree/bindings/mtd/oxnas-nand.txt +++ b/Documentation/devicetree/bindings/mtd/oxnas-nand.txt @@ -1,6 +1,6 @@ * Oxford Semiconductor OXNAS NAND Controller -Please refer to nand.txt for generic information regarding MTD NAND bindings. +Please refer to nand-controller.yaml for generic information regarding MTD NAND bindings. Required properties: - compatible: "oxsemi,ox820-nand" diff --git a/Documentation/devicetree/bindings/mtd/qcom_nandc.txt b/Documentation/devicetree/bindings/mtd/qcom_nandc.txt index 1123cc6d56ef..5c2fba4b30fe 100644 --- a/Documentation/devicetree/bindings/mtd/qcom_nandc.txt +++ b/Documentation/devicetree/bindings/mtd/qcom_nandc.txt @@ -47,8 +47,8 @@ Required properties: - #size-cells: see partition.txt Optional properties: -- nand-bus-width: see nand.txt -- nand-ecc-strength: see nand.txt. If not specified, then ECC strength will +- nand-bus-width: see nand-controller.yaml +- nand-ecc-strength: see nand-controller.yaml. If not specified, then ECC strength will be used according to chip requirement and available OOB size. diff --git a/Documentation/devicetree/bindings/mtd/samsung-s3c2410.txt b/Documentation/devicetree/bindings/mtd/samsung-s3c2410.txt index 0040eb8895e0..09815c40fc8a 100644 --- a/Documentation/devicetree/bindings/mtd/samsung-s3c2410.txt +++ b/Documentation/devicetree/bindings/mtd/samsung-s3c2410.txt @@ -6,7 +6,7 @@ Required properties: "samsung,s3c2412-nand" "samsung,s3c2440-nand" - reg : register's location and length. -- #address-cells, #size-cells : see nand.txt +- #address-cells, #size-cells : see nand-controller.yaml - clocks : phandle to the nand controller clock - clock-names : must contain "nand" @@ -14,8 +14,8 @@ Optional child nodes: Child nodes representing the available nand chips. Optional child properties: -- nand-ecc-mode : see nand.txt -- nand-on-flash-bbt : see nand.txt +- nand-ecc-mode : see nand-controller.yaml +- nand-on-flash-bbt : see nand-controller.yaml Each child device node may optionally contain a 'partitions' sub-node, which further contains sub-nodes describing the flash partition mapping. diff --git a/Documentation/devicetree/bindings/mtd/stm32-fmc2-nand.txt b/Documentation/devicetree/bindings/mtd/stm32-fmc2-nand.txt index ad2bef826582..e55895e8dae4 100644 --- a/Documentation/devicetree/bindings/mtd/stm32-fmc2-nand.txt +++ b/Documentation/devicetree/bindings/mtd/stm32-fmc2-nand.txt @@ -24,9 +24,9 @@ Required properties: - reg: describes the CS lines assigned to the NAND device. Optional properties: -- nand-on-flash-bbt: see nand.txt -- nand-ecc-strength: see nand.txt -- nand-ecc-step-size: see nand.txt +- nand-on-flash-bbt: see nand-controller.yaml +- nand-ecc-strength: see nand-controller.yaml +- nand-ecc-step-size: see nand-controller.yaml The following ECC strength and step size are currently supported: - nand-ecc-strength = <1>, nand-ecc-step-size = <512> (Hamming) diff --git a/Documentation/devicetree/bindings/mtd/tango-nand.txt b/Documentation/devicetree/bindings/mtd/tango-nand.txt index cd1bf2ac9055..91c8420241af 100644 --- a/Documentation/devicetree/bindings/mtd/tango-nand.txt +++ b/Documentation/devicetree/bindings/mtd/tango-nand.txt @@ -11,7 +11,7 @@ Required properties: - #size-cells: <0> Children nodes represent the available NAND chips. -See Documentation/devicetree/bindings/mtd/nand.txt for generic bindings. +See Documentation/devicetree/bindings/mtd/nand-controller.yaml for generic bindings. Example: diff --git a/Documentation/devicetree/bindings/mtd/vf610-nfc.txt b/Documentation/devicetree/bindings/mtd/vf610-nfc.txt index c96eeb65f450..7db5e6e609df 100644 --- a/Documentation/devicetree/bindings/mtd/vf610-nfc.txt +++ b/Documentation/devicetree/bindings/mtd/vf610-nfc.txt @@ -25,14 +25,14 @@ only handle one NAND chip. Required properties: - compatible: Should be set to "fsl,vf610-nfc-cs". -- nand-bus-width: see nand.txt -- nand-ecc-mode: see nand.txt +- nand-bus-width: see nand-controller.yaml +- nand-ecc-mode: see nand-controller.yaml Required properties for hardware ECC: -- nand-ecc-strength: supported strengths are 24 and 32 bit (see nand.txt) +- nand-ecc-strength: supported strengths are 24 and 32 bit (see nand-controller.yaml) - nand-ecc-step-size: step size equals page size, currently only 2k pages are supported -- nand-on-flash-bbt: see nand.txt +- nand-on-flash-bbt: see nand-controller.yaml Example: diff --git a/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt b/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt index 188c8bd4eb67..5a0111d4de58 100644 --- a/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt +++ b/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt @@ -4,6 +4,7 @@ Required properties: - compatible: Should be one of the following: - "microchip,mcp2510" for MCP2510. - "microchip,mcp2515" for MCP2515. + - "microchip,mcp25625" for MCP25625. - reg: SPI chip select. - clocks: The clock feeding the CAN controller. - interrupts: Should contain IRQ line for the CAN controller. diff --git a/Documentation/devicetree/bindings/perf/fsl-imx-ddr.txt b/Documentation/devicetree/bindings/perf/fsl-imx-ddr.txt new file mode 100644 index 000000000000..d77e3f26f9e6 --- /dev/null +++ b/Documentation/devicetree/bindings/perf/fsl-imx-ddr.txt @@ -0,0 +1,21 @@ +* Freescale(NXP) IMX8 DDR performance monitor + +Required properties: + +- compatible: should be one of: + "fsl,imx8-ddr-pmu" + "fsl,imx8m-ddr-pmu" + +- reg: physical address and size + +- interrupts: single interrupt + generated by the control block + +Example: + + ddr-pmu@5c020000 { + compatible = "fsl,imx8-ddr-pmu"; + reg = <0x5c020000 0x10000>; + interrupt-parent = <&gic>; + interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>; + }; diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml new file mode 100644 index 000000000000..f97a4ecd7b91 --- /dev/null +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -0,0 +1,170 @@ +# SPDX-License-Identifier: (GPL-2.0 OR MIT) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/riscv/cpus.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V bindings for 'cpus' DT nodes + +maintainers: + - Paul Walmsley <paul.walmsley@sifive.com> + - Palmer Dabbelt <palmer@sifive.com> + +allOf: + - $ref: /schemas/cpus.yaml# + +properties: + $nodename: + const: cpus + description: Container of cpu nodes + + '#address-cells': + const: 1 + description: | + A single unsigned 32-bit integer uniquely identifies each RISC-V + hart in a system. (See the "reg" node under the "cpu" node, + below). + + '#size-cells': + const: 0 + +patternProperties: + '^cpu@[0-9a-f]+$': + properties: + compatible: + type: array + items: + - enum: + - sifive,rocket0 + - sifive,e5 + - sifive,e51 + - sifive,u54-mc + - sifive,u54 + - sifive,u5 + - const: riscv + description: + Identifies that the hart uses the RISC-V instruction set + and identifies the type of the hart. + + mmu-type: + allOf: + - $ref: "/schemas/types.yaml#/definitions/string" + - enum: + - riscv,sv32 + - riscv,sv39 + - riscv,sv48 + description: + Identifies the MMU address translation mode used on this + hart. These values originate from the RISC-V Privileged + Specification document, available from + https://riscv.org/specifications/ + + riscv,isa: + allOf: + - $ref: "/schemas/types.yaml#/definitions/string" + - enum: + - rv64imac + - rv64imafdc + description: + Identifies the specific RISC-V instruction set architecture + supported by the hart. These are documented in the RISC-V + User-Level ISA document, available from + https://riscv.org/specifications/ + + timebase-frequency: + type: integer + minimum: 1 + description: + Specifies the clock frequency of the system timer in Hz. + This value is common to all harts on a single system image. + + interrupt-controller: + type: object + description: Describes the CPU's local interrupt controller + + properties: + '#interrupt-cells': + const: 1 + + compatible: + const: riscv,cpu-intc + + interrupt-controller: true + + required: + - '#interrupt-cells' + - compatible + - interrupt-controller + + required: + - riscv,isa + - timebase-frequency + - interrupt-controller + +examples: + - | + // Example 1: SiFive Freedom U540G Development Kit + cpus { + #address-cells = <1>; + #size-cells = <0>; + timebase-frequency = <1000000>; + cpu@0 { + clock-frequency = <0>; + compatible = "sifive,rocket0", "riscv"; + device_type = "cpu"; + i-cache-block-size = <64>; + i-cache-sets = <128>; + i-cache-size = <16384>; + reg = <0>; + riscv,isa = "rv64imac"; + cpu_intc0: interrupt-controller { + #interrupt-cells = <1>; + compatible = "riscv,cpu-intc"; + interrupt-controller; + }; + }; + cpu@1 { + clock-frequency = <0>; + compatible = "sifive,rocket0", "riscv"; + d-cache-block-size = <64>; + d-cache-sets = <64>; + d-cache-size = <32768>; + d-tlb-sets = <1>; + d-tlb-size = <32>; + device_type = "cpu"; + i-cache-block-size = <64>; + i-cache-sets = <64>; + i-cache-size = <32768>; + i-tlb-sets = <1>; + i-tlb-size = <32>; + mmu-type = "riscv,sv39"; + reg = <1>; + riscv,isa = "rv64imafdc"; + tlb-split; + cpu_intc1: interrupt-controller { + #interrupt-cells = <1>; + compatible = "riscv,cpu-intc"; + interrupt-controller; + }; + }; + }; + + - | + // Example 2: Spike ISA Simulator with 1 Hart + cpus { + #address-cells = <1>; + #size-cells = <0>; + cpu@0 { + device_type = "cpu"; + reg = <0>; + compatible = "riscv"; + riscv,isa = "rv64imafdc"; + mmu-type = "riscv,sv48"; + interrupt-controller { + #interrupt-cells = <1>; + interrupt-controller; + compatible = "riscv,cpu-intc"; + }; + }; + }; +... diff --git a/Documentation/devicetree/bindings/riscv/sifive.yaml b/Documentation/devicetree/bindings/riscv/sifive.yaml new file mode 100644 index 000000000000..9d17dc2f3f84 --- /dev/null +++ b/Documentation/devicetree/bindings/riscv/sifive.yaml @@ -0,0 +1,25 @@ +# SPDX-License-Identifier: (GPL-2.0 OR MIT) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/riscv/sifive.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: SiFive SoC-based boards + +maintainers: + - Paul Walmsley <paul.walmsley@sifive.com> + - Palmer Dabbelt <palmer@sifive.com> + +description: + SiFive SoC-based boards + +properties: + $nodename: + const: '/' + compatible: + items: + - enum: + - sifive,freedom-unleashed-a00 + - const: sifive,fu540-c000 + - const: sifive,fu540 +... diff --git a/Documentation/devicetree/bindings/sifive/sifive-blocks-ip-versioning.txt b/Documentation/devicetree/bindings/sifive/sifive-blocks-ip-versioning.txt new file mode 100644 index 000000000000..beaa3b64084e --- /dev/null +++ b/Documentation/devicetree/bindings/sifive/sifive-blocks-ip-versioning.txt @@ -0,0 +1,38 @@ +DT compatible string versioning for SiFive open-source IP blocks + +This document describes the version specification for DT "compatible" +strings for open-source SiFive IP blocks. HDL for these IP blocks +can be found in this public repository: + +https://github.com/sifive/sifive-blocks + +IP block-specific DT compatible strings are contained within the HDL, +in the form "sifive,<ip-block-name><integer version number>". + +An example is "sifive,uart0" from: + +https://github.com/sifive/sifive-blocks/blob/v1.0/src/main/scala/devices/uart/UART.scala#L43 + +Until these IP blocks (or IP integration) support version +auto-discovery, the maintainers of these IP blocks intend to increment +the suffixed number in the compatible string whenever the software +interface to these IP blocks changes, or when the functionality of the +underlying IP blocks changes in a way that software should be aware of. + +Driver developers can use compatible string "match" values such as +"sifive,uart0" to indicate that their driver is compatible with the +register interface and functionality associated with the relevant +upstream sifive-blocks commits. It is expected that most drivers will +match on these IP block-specific compatible strings. + +DT data authors, when writing data for a particular SoC, should +continue to specify an SoC-specific compatible string value, such as +"sifive,fu540-c000-uart". This way, if SoC-specific +integration-specific bug fixes or workarounds are needed, the kernel +or other system software can match on this string to apply them. The +IP block-specific compatible string (such as "sifive,uart0") should +then be specified as a subsequent value. + +An example of this style: + + compatible = "sifive,fu540-c000-uart", "sifive,uart0"; diff --git a/Documentation/devicetree/bindings/timer/nxp,sysctr-timer.txt b/Documentation/devicetree/bindings/timer/nxp,sysctr-timer.txt new file mode 100644 index 000000000000..d57659996d62 --- /dev/null +++ b/Documentation/devicetree/bindings/timer/nxp,sysctr-timer.txt @@ -0,0 +1,25 @@ +NXP System Counter Module(sys_ctr) + +The system counter(sys_ctr) is a programmable system counter which provides +a shared time base to Cortex A15, A7, A53, A73, etc. it is intended for use in +applications where the counter is always powered and support multiple, +unrelated clocks. The compare frame inside can be used for timer purpose. + +Required properties: + +- compatible : should be "nxp,sysctr-timer" +- reg : Specifies the base physical address and size of the comapre + frame and the counter control, read & compare. +- interrupts : should be the first compare frames' interrupt +- clocks : Specifies the counter clock. +- clock-names: Specifies the clock's name of this module + +Example: + + system_counter: timer@306a0000 { + compatible = "nxp,sysctr-timer"; + reg = <0x306a0000 0x20000>;/* system-counter-rd & compare */ + clocks = <&clk_8m>; + clock-names = "per"; + interrupts = <GIC_SPI 47 IRQ_TYPE_LEVEL_HIGH>; + }; diff --git a/Documentation/driver-api/generic-counter.rst b/Documentation/driver-api/generic-counter.rst index f51db893f595..0c161b1a3be6 100644 --- a/Documentation/driver-api/generic-counter.rst +++ b/Documentation/driver-api/generic-counter.rst @@ -251,7 +251,7 @@ for defining a counter device. .. kernel-doc:: include/linux/counter.h :internal: -.. kernel-doc:: drivers/counter/generic-counter.c +.. kernel-doc:: drivers/counter/counter.c :export: Implementation diff --git a/Documentation/driver-api/s390-drivers.rst b/Documentation/driver-api/s390-drivers.rst index 30e6aa7e160b..5158577bc29b 100644 --- a/Documentation/driver-api/s390-drivers.rst +++ b/Documentation/driver-api/s390-drivers.rst @@ -27,7 +27,7 @@ not strictly considered I/O devices. They are considered here as well, although they are not the focus of this document. Some additional information can also be found in the kernel source under -Documentation/s390/driver-model.txt. +Documentation/s390/driver-model.rst. The css bus =========== @@ -38,7 +38,7 @@ into several categories: * Standard I/O subchannels, for use by the system. They have a child device on the ccw bus and are described below. * I/O subchannels bound to the vfio-ccw driver. See - Documentation/s390/vfio-ccw.txt. + Documentation/s390/vfio-ccw.rst. * Message subchannels. No Linux driver currently exists. * CHSC subchannels (at most one). The chsc subchannel driver can be used to send asynchronous chsc commands. diff --git a/Documentation/driver-api/uio-howto.rst b/Documentation/driver-api/uio-howto.rst index 25f50eace28b..8fecfa11d4ff 100644 --- a/Documentation/driver-api/uio-howto.rst +++ b/Documentation/driver-api/uio-howto.rst @@ -276,8 +276,8 @@ fields of ``struct uio_mem``: - ``int memtype``: Required if the mapping is used. Set this to ``UIO_MEM_PHYS`` if you you have physical memory on your card to be mapped. Use ``UIO_MEM_LOGICAL`` for logical memory (e.g. allocated - with :c:func:`kmalloc()`). There's also ``UIO_MEM_VIRTUAL`` for - virtual memory. + with :c:func:`__get_free_pages()` but not kmalloc()). There's also + ``UIO_MEM_VIRTUAL`` for virtual memory. - ``phys_addr_t addr``: Required if the mapping is used. Fill in the address of your memory block. This address is the one that appears in diff --git a/Documentation/fb/fbcon.txt b/Documentation/fb/fbcon.txt index 60a5ec04e8f0..5a865437b33f 100644 --- a/Documentation/fb/fbcon.txt +++ b/Documentation/fb/fbcon.txt @@ -79,7 +79,7 @@ C. Boot options Select the initial font to use. The value 'name' can be any of the compiled-in fonts: 10x18, 6x10, 7x14, Acorn8x8, MINI4x6, - PEARL8x8, ProFont6x11, SUN12x22, SUN8x16, VGA8x16, VGA8x8. + PEARL8x8, ProFont6x11, SUN12x22, SUN8x16, TER16x32, VGA8x16, VGA8x8. Note, not all drivers can handle font with widths not divisible by 8, such as vga16fb. diff --git a/Documentation/filesystems/overlayfs.txt b/Documentation/filesystems/overlayfs.txt index eef7d9d259e8..1da2f1668f08 100644 --- a/Documentation/filesystems/overlayfs.txt +++ b/Documentation/filesystems/overlayfs.txt @@ -336,8 +336,20 @@ the copied layers will fail the verification of the lower root file handle. Non-standard behavior --------------------- -Overlayfs can now act as a POSIX compliant filesystem with the following -features turned on: +Current version of overlayfs can act as a mostly POSIX compliant +filesystem. + +This is the list of cases that overlayfs doesn't currently handle: + +a) POSIX mandates updating st_atime for reads. This is currently not +done in the case when the file resides on a lower layer. + +b) If a file residing on a lower layer is opened for read-only and then +memory mapped with MAP_SHARED, then subsequent changes to the file are not +reflected in the memory mapping. + +The following options allow overlayfs to act more like a standards +compliant filesystem: 1) "redirect_dir" diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index 66cad5c86171..a226061fa109 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -45,6 +45,7 @@ Table of Contents 3.9 /proc/<pid>/map_files - Information about memory mapped files 3.10 /proc/<pid>/timerslack_ns - Task timerslack value 3.11 /proc/<pid>/patch_state - Livepatch patch operation state + 3.12 /proc/<pid>/arch_status - Task architecture specific information 4 Configuring procfs 4.1 Mount options @@ -1948,6 +1949,45 @@ patched. If the patch is being enabled, then the task has already been patched. If the patch is being disabled, then the task hasn't been unpatched yet. +3.12 /proc/<pid>/arch_status - task architecture specific status +------------------------------------------------------------------- +When CONFIG_PROC_PID_ARCH_STATUS is enabled, this file displays the +architecture specific status of the task. + +Example +------- + $ cat /proc/6753/arch_status + AVX512_elapsed_ms: 8 + +Description +----------- + +x86 specific entries: +--------------------- + AVX512_elapsed_ms: + ------------------ + If AVX512 is supported on the machine, this entry shows the milliseconds + elapsed since the last time AVX512 usage was recorded. The recording + happens on a best effort basis when a task is scheduled out. This means + that the value depends on two factors: + + 1) The time which the task spent on the CPU without being scheduled + out. With CPU isolation and a single runnable task this can take + several seconds. + + 2) The time since the task was scheduled out last. Depending on the + reason for being scheduled out (time slice exhausted, syscall ...) + this can be arbitrary long time. + + As a consequence the value cannot be considered precise and authoritative + information. The application which uses this information has to be aware + of the overall scenario on the system in order to determine whether a + task is a real AVX512 user or not. Precise information can be obtained + with performance counters. + + A special value of '-1' indicates that no AVX512 usage was recorded, thus + the task is unlikely an AVX512 user, but depends on the workload and the + scheduling scenario, it also could be a false negative mentioned above. ------------------------------------------------------------------------------ Configuring procfs diff --git a/Documentation/firmware-guide/acpi/enumeration.rst b/Documentation/firmware-guide/acpi/enumeration.rst index 6b32b7be8c85..850be9696931 100644 --- a/Documentation/firmware-guide/acpi/enumeration.rst +++ b/Documentation/firmware-guide/acpi/enumeration.rst @@ -423,7 +423,7 @@ will be enumerated to depends on the device ID returned by _HID. For example, the following ACPI sample might be used to enumerate an lm75-type I2C temperature sensor and match it to the driver using the Device Tree -namespace link: +namespace link:: Device (TMP0) { diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt index 03c065855eaf..d65ad5746f94 100644 --- a/Documentation/kbuild/makefiles.txt +++ b/Documentation/kbuild/makefiles.txt @@ -437,20 +437,6 @@ more details, with real examples. The second argument is optional, and if supplied will be used if first argument is not supported. - cc-ldoption - cc-ldoption is used to check if $(CC) when used to link object files - supports the given option. An optional second option may be - specified if first option are not supported. - - Example: - #arch/x86/kernel/Makefile - vsyscall-flags += $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) - - In the above example, vsyscall-flags will be assigned the option - -Wl$(comma)--hash-style=sysv if it is supported by $(CC). - The second argument is optional, and if supplied will be used - if first argument is not supported. - as-instr as-instr checks if the assembler reports a specific instruction and then outputs either option1 or option2 diff --git a/Documentation/kdump/kdump.txt b/Documentation/kdump/kdump.txt index 51814450a7f8..3162eeb8c262 100644 --- a/Documentation/kdump/kdump.txt +++ b/Documentation/kdump/kdump.txt @@ -410,7 +410,7 @@ Notes on loading the dump-capture kernel: * Boot parameter "1" boots the dump-capture kernel into single-user mode without networking. If you want networking, use "3". -* We generally don' have to bring up a SMP kernel just to capture the +* We generally don't have to bring up a SMP kernel just to capture the dump. Hence generally it is useful either to build a UP dump-capture kernel or specify maxcpus=1 option while loading dump-capture kernel. Note, though maxcpus always works, you had better replace it with diff --git a/Documentation/locking/lockdep-design.txt b/Documentation/locking/lockdep-design.txt index 39fae143c9cb..f189d130e543 100644 --- a/Documentation/locking/lockdep-design.txt +++ b/Documentation/locking/lockdep-design.txt @@ -15,34 +15,48 @@ tens of thousands of) instantiations. For example a lock in the inode struct is one class, while each inode has its own instantiation of that lock class. -The validator tracks the 'state' of lock-classes, and it tracks -dependencies between different lock-classes. The validator maintains a -rolling proof that the state and the dependencies are correct. - -Unlike an lock instantiation, the lock-class itself never goes away: when -a lock-class is used for the first time after bootup it gets registered, -and all subsequent uses of that lock-class will be attached to this -lock-class. +The validator tracks the 'usage state' of lock-classes, and it tracks +the dependencies between different lock-classes. Lock usage indicates +how a lock is used with regard to its IRQ contexts, while lock +dependency can be understood as lock order, where L1 -> L2 suggests that +a task is attempting to acquire L2 while holding L1. From lockdep's +perspective, the two locks (L1 and L2) are not necessarily related; that +dependency just means the order ever happened. The validator maintains a +continuing effort to prove lock usages and dependencies are correct or +the validator will shoot a splat if incorrect. + +A lock-class's behavior is constructed by its instances collectively: +when the first instance of a lock-class is used after bootup the class +gets registered, then all (subsequent) instances will be mapped to the +class and hence their usages and dependecies will contribute to those of +the class. A lock-class does not go away when a lock instance does, but +it can be removed if the memory space of the lock class (static or +dynamic) is reclaimed, this happens for example when a module is +unloaded or a workqueue is destroyed. State ----- -The validator tracks lock-class usage history into 4 * nSTATEs + 1 separate -state bits: +The validator tracks lock-class usage history and divides the usage into +(4 usages * n STATEs + 1) categories: +where the 4 usages can be: - 'ever held in STATE context' - 'ever held as readlock in STATE context' - 'ever held with STATE enabled' - 'ever held as readlock with STATE enabled' -Where STATE can be either one of (kernel/locking/lockdep_states.h) - - hardirq - - softirq +where the n STATEs are coded in kernel/locking/lockdep_states.h and as of +now they include: +- hardirq +- softirq +where the last 1 category is: - 'ever used' [ == !unused ] -When locking rules are violated, these state bits are presented in the -locking error messages, inside curlies. A contrived example: +When locking rules are violated, these usage bits are presented in the +locking error messages, inside curlies, with a total of 2 * n STATEs bits. +A contrived example: modprobe/2287 is trying to acquire lock: (&sio_locks[i].lock){-.-.}, at: [<c02867fd>] mutex_lock+0x21/0x24 @@ -51,28 +65,67 @@ locking error messages, inside curlies. A contrived example: (&sio_locks[i].lock){-.-.}, at: [<c02867fd>] mutex_lock+0x21/0x24 -The bit position indicates STATE, STATE-read, for each of the states listed -above, and the character displayed in each indicates: +For a given lock, the bit positions from left to right indicate the usage +of the lock and readlock (if exists), for each of the n STATEs listed +above respectively, and the character displayed at each bit position +indicates: '.' acquired while irqs disabled and not in irq context '-' acquired in irq context '+' acquired with irqs enabled '?' acquired in irq context with irqs enabled. -Unused mutexes cannot be part of the cause of an error. +The bits are illustrated with an example: + + (&sio_locks[i].lock){-.-.}, at: [<c02867fd>] mutex_lock+0x21/0x24 + |||| + ||| \-> softirq disabled and not in softirq context + || \--> acquired in softirq context + | \---> hardirq disabled and not in hardirq context + \----> acquired in hardirq context + + +For a given STATE, whether the lock is ever acquired in that STATE +context and whether that STATE is enabled yields four possible cases as +shown in the table below. The bit character is able to indicate which +exact case is for the lock as of the reporting time. + + ------------------------------------------- + | | irq enabled | irq disabled | + |-------------------------------------------| + | ever in irq | ? | - | + |-------------------------------------------| + | never in irq | + | . | + ------------------------------------------- + +The character '-' suggests irq is disabled because if otherwise the +charactor '?' would have been shown instead. Similar deduction can be +applied for '+' too. + +Unused locks (e.g., mutexes) cannot be part of the cause of an error. Single-lock state rules: ------------------------ +A lock is irq-safe means it was ever used in an irq context, while a lock +is irq-unsafe means it was ever acquired with irq enabled. + A softirq-unsafe lock-class is automatically hardirq-unsafe as well. The -following states are exclusive, and only one of them is allowed to be -set for any lock-class: +following states must be exclusive: only one of them is allowed to be set +for any lock-class based on its usage: + + <hardirq-safe> or <hardirq-unsafe> + <softirq-safe> or <softirq-unsafe> - <hardirq-safe> and <hardirq-unsafe> - <softirq-safe> and <softirq-unsafe> +This is because if a lock can be used in irq context (irq-safe) then it +cannot be ever acquired with irq enabled (irq-unsafe). Otherwise, a +deadlock may happen. For example, in the scenario that after this lock +was acquired but before released, if the context is interrupted this +lock will be attempted to acquire twice, which creates a deadlock, +referred to as lock recursion deadlock. -The validator detects and reports lock usage that violate these +The validator detects and reports lock usage that violates these single-lock state rules. Multi-lock dependency rules: @@ -81,15 +134,18 @@ Multi-lock dependency rules: The same lock-class must not be acquired twice, because this could lead to lock recursion deadlocks. -Furthermore, two locks may not be taken in different order: +Furthermore, two locks can not be taken in inverse order: <L1> -> <L2> <L2> -> <L1> -because this could lead to lock inversion deadlocks. (The validator -finds such dependencies in arbitrary complexity, i.e. there can be any -other locking sequence between the acquire-lock operations, the -validator will still track all dependencies between locks.) +because this could lead to a deadlock - referred to as lock inversion +deadlock - as attempts to acquire the two locks form a circle which +could lead to the two contexts waiting for each other permanently. The +validator will find such dependency circle in arbitrary complexity, +i.e., there can be any other locking sequence between the acquire-lock +operations; the validator will still find whether these locks can be +acquired in a circular fashion. Furthermore, the following usage based lock dependencies are not allowed between any two lock-classes: diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index f70ebcdfe592..e4e07c8ab89e 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -3,7 +3,7 @@ ============================ By: David Howells <dhowells@redhat.com> - Paul E. McKenney <paulmck@linux.vnet.ibm.com> + Paul E. McKenney <paulmck@linux.ibm.com> Will Deacon <will.deacon@arm.com> Peter Zijlstra <peterz@infradead.org> diff --git a/Documentation/networking/af_xdp.rst b/Documentation/networking/af_xdp.rst index e14d7d40fc75..50bccbf68308 100644 --- a/Documentation/networking/af_xdp.rst +++ b/Documentation/networking/af_xdp.rst @@ -316,16 +316,16 @@ A: When a netdev of a physical NIC is initialized, Linux usually all the traffic, you can force the netdev to only have 1 queue, queue id 0, and then bind to queue 0. You can use ethtool to do this:: - sudo ethtool -L <interface> combined 1 + sudo ethtool -L <interface> combined 1 If you want to only see part of the traffic, you can program the NIC through ethtool to filter out your traffic to a single queue id that you can bind your XDP socket to. Here is one example in which UDP traffic to and from port 4242 are sent to queue 2:: - sudo ethtool -N <interface> rx-flow-hash udp4 fn - sudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \ - 4242 action 2 + sudo ethtool -N <interface> rx-flow-hash udp4 fn + sudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \ + 4242 action 2 A number of other ways are possible all up to the capabilitites of the NIC you have. diff --git a/Documentation/networking/device_drivers/index.rst b/Documentation/networking/device_drivers/index.rst new file mode 100644 index 000000000000..75fa537763a4 --- /dev/null +++ b/Documentation/networking/device_drivers/index.rst @@ -0,0 +1,30 @@ +.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +Vendor Device Drivers +===================== + +Contents: + +.. toctree:: + :maxdepth: 2 + + freescale/dpaa2/index + intel/e100 + intel/e1000 + intel/e1000e + intel/fm10k + intel/igb + intel/igbvf + intel/ixgb + intel/ixgbe + intel/ixgbevf + intel/i40e + intel/iavf + intel/ice + +.. only:: subproject + + Indices + ======= + + * :ref:`genindex` diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst index f390fe3cfdfb..a46fca264bee 100644 --- a/Documentation/networking/index.rst +++ b/Documentation/networking/index.rst @@ -11,19 +11,7 @@ Contents: batman-adv can can_ucan_protocol - device_drivers/freescale/dpaa2/index - device_drivers/intel/e100 - device_drivers/intel/e1000 - device_drivers/intel/e1000e - device_drivers/intel/fm10k - device_drivers/intel/igb - device_drivers/intel/igbvf - device_drivers/intel/ixgb - device_drivers/intel/ixgbe - device_drivers/intel/ixgbevf - device_drivers/intel/i40e - device_drivers/intel/iavf - device_drivers/intel/ice + device_drivers/index dsa/index devlink-info-versions ieee802154 @@ -40,6 +28,8 @@ Contents: checksum-offloads segmentation-offloads scaling + tls + tls-offload .. only:: subproject diff --git a/Documentation/networking/ip-sysctl.txt b/Documentation/networking/ip-sysctl.txt index 725b8bea58a7..22f6b8b1110a 100644 --- a/Documentation/networking/ip-sysctl.txt +++ b/Documentation/networking/ip-sysctl.txt @@ -255,6 +255,14 @@ tcp_base_mss - INTEGER Path MTU discovery (MTU probing). If MTU probing is enabled, this is the initial MSS used by the connection. +tcp_min_snd_mss - INTEGER + TCP SYN and SYNACK messages usually advertise an ADVMSS option, + as described in RFC 1122 and RFC 6691. + If this ADVMSS option is smaller than tcp_min_snd_mss, + it is silently capped to tcp_min_snd_mss. + + Default : 48 (at least 8 bytes of payload per segment) + tcp_congestion_control - STRING Set the congestion control algorithm to be used for new connections. The algorithm "reno" is always available, but @@ -560,10 +568,10 @@ tcp_comp_sack_delay_ns - LONG INTEGER Default : 1,000,000 ns (1 ms) tcp_comp_sack_nr - INTEGER - Max numer of SACK that can be compressed. + Max number of SACK that can be compressed. Using 0 disables SACK compression. - Detault : 44 + Default : 44 tcp_slow_start_after_idle - BOOLEAN If set, provide RFC2861 behavior and time out the congestion @@ -772,6 +780,14 @@ tcp_challenge_ack_limit - INTEGER in RFC 5961 (Improving TCP's Robustness to Blind In-Window Attacks) Default: 100 +tcp_rx_skb_cache - BOOLEAN + Controls a per TCP socket cache of one skb, that might help + performance of some workloads. This might be dangerous + on systems with a lot of TCP sockets, since it increases + memory usage. + + Default: 0 (disabled) + UDP variables: udp_l3mdev_accept - BOOLEAN diff --git a/Documentation/networking/rds.txt b/Documentation/networking/rds.txt index 0235ae69af2a..f2a0147c933d 100644 --- a/Documentation/networking/rds.txt +++ b/Documentation/networking/rds.txt @@ -389,7 +389,7 @@ Multipath RDS (mprds) a common (to all paths) part, and a per-path struct rds_conn_path. All I/O workqs and reconnect threads are driven from the rds_conn_path. Transports such as TCP that are multipath capable may then set up a - TPC socket per rds_conn_path, and this is managed by the transport via + TCP socket per rds_conn_path, and this is managed by the transport via the transport privatee cp_transport_data pointer. Transports announce themselves as multipath capable by setting the diff --git a/Documentation/networking/segmentation-offloads.rst b/Documentation/networking/segmentation-offloads.rst index 89d1ee933e9f..085e8fab03fd 100644 --- a/Documentation/networking/segmentation-offloads.rst +++ b/Documentation/networking/segmentation-offloads.rst @@ -18,7 +18,7 @@ The following technologies are described: * Generic Segmentation Offload - GSO * Generic Receive Offload - GRO * Partial Generic Segmentation Offload - GSO_PARTIAL - * SCTP accelleration with GSO - GSO_BY_FRAGS + * SCTP acceleration with GSO - GSO_BY_FRAGS TCP Segmentation Offload @@ -148,7 +148,7 @@ that the IPv4 ID field is incremented in the case that a given header does not have the DF bit set. -SCTP accelleration with GSO +SCTP acceleration with GSO =========================== SCTP - despite the lack of hardware support - can still take advantage of diff --git a/Documentation/networking/tls-offload-layers.svg b/Documentation/networking/tls-offload-layers.svg new file mode 100644 index 000000000000..cf72f05dbb21 --- /dev/null +++ b/Documentation/networking/tls-offload-layers.svg @@ -0,0 +1 @@ +<svg version="1.1" viewBox="0.0 0.0 460.0 500.0" fill="none" stroke="none" stroke-linecap="square" stroke-miterlimit="10" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"><clipPath id="p.0"><path d="m0 0l960.0 0l0 720.0l-960.0 0l0 -720.0z" clip-rule="nonzero"/></clipPath><g clip-path="url(#p.0)"><path fill="#000000" fill-opacity="0.0" d="m0 0l960.0 0l0 720.0l-960.0 0z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m117.02887 0l72.28346 0l0 40.25197l-72.28346 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m117.02887 0l72.28346 0l0 40.25197l-72.28346 0z" fill-rule="evenodd"/><path fill="#000000" d="m135.71944 27.045982l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0zm12.853302 -3.109375l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625zm15.453842 4.578125q-0.921875 0.765625 -1.765625 1.09375q-0.828125 0.3125 -1.796875 0.3125q-1.59375 0 -2.453125 -0.78125q-0.859375 -0.78125 -0.859375 -1.984375q0 -0.71875 0.328125 -1.296875q0.328125 -0.59375 0.84375 -0.9375q0.53125 -0.359375 1.1875 -0.546875q0.46875 -0.125 1.453125 -0.25q1.984375 -0.234375 2.921875 -0.5625q0.015625 -0.34375 0.015625 -0.421875q0 -1.0 -0.46875 -1.421875q-0.625 -0.546875 -1.875 -0.546875q-1.15625 0 -1.703125 0.40625q-0.546875 0.40625 -0.8125 1.421875l-1.609375 -0.21875q0.21875 -1.015625 0.71875 -1.640625q0.5 -0.640625 1.453125 -0.984375q0.953125 -0.34375 2.1875 -0.34375q1.25 0 2.015625 0.296875q0.78125 0.28125 1.140625 0.734375q0.375 0.4375 0.515625 1.109375q0.078125 0.421875 0.078125 1.515625l0 2.1875q0 2.28125 0.109375 2.890625q0.109375 0.59375 0.40625 1.15625l-1.703125 0q-0.265625 -0.515625 -0.328125 -1.1875zm-0.140625 -3.671875q-0.890625 0.375 -2.671875 0.625q-1.015625 0.140625 -1.4375 0.328125q-0.421875 0.1875 -0.65625 0.53125q-0.21875 0.34375 -0.21875 0.78125q0 0.65625 0.5 1.09375q0.5 0.4375 1.453125 0.4375q0.9375 0 1.671875 -0.40625q0.75 -0.421875 1.09375 -1.140625q0.265625 -0.5625 0.265625 -1.640625l0 -0.609375zm10.469467 4.859375l0 -1.21875q-0.90625 1.4375 -2.703125 1.4375q-1.15625 0 -2.125 -0.640625q-0.96875 -0.640625 -1.5 -1.78125q-0.53125 -1.140625 -0.53125 -2.625q0 -1.453125 0.484375 -2.625q0.484375 -1.1875 1.4375 -1.8125q0.96875 -0.625 2.171875 -0.625q0.875 0 1.546875 0.375q0.6875 0.359375 1.109375 0.953125l0 -4.796875l1.640625 0l0 13.359375l-1.53125 0zm-5.171875 -4.828125q0 1.859375 0.78125 2.78125q0.78125 0.921875 1.84375 0.921875q1.078125 0 1.828125 -0.875q0.75 -0.890625 0.75 -2.6875q0 -1.984375 -0.765625 -2.90625q-0.765625 -0.9375 -1.890625 -0.9375q-1.078125 0 -1.8125 0.890625q-0.734375 0.890625 -0.734375 2.8125z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m309.02887 0l72.28348 0l0 40.25197l-72.28348 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m309.02887 0l72.28348 0l0 40.25197l-72.28348 0z" fill-rule="evenodd"/><path fill="#000000" d="m328.4915 27.045982l-2.96875 -9.671875l1.703125 0l1.53125 5.578125l0.578125 2.078125q0.046875 -0.15625 0.5 -2.0l1.546875 -5.65625l1.6875 0l1.4375 5.609375l0.484375 1.84375l0.5625 -1.859375l1.65625 -5.59375l1.59375 0l-3.03125 9.671875l-1.703125 0l-1.53125 -5.796875l-0.375 -1.640625l-1.953125 7.4375l-1.71875 0zm11.676086 0l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0zm6.228302 -11.46875l0 -1.890625l1.640625 0l0 1.890625l-1.640625 0zm0 11.46875l0 -9.671875l1.640625 0l0 9.671875l-1.640625 0zm7.722931 -1.46875l0.234375 1.453125q-0.6875 0.140625 -1.234375 0.140625q-0.890625 0 -1.390625 -0.28125q-0.484375 -0.28125 -0.6875 -0.734375q-0.203125 -0.46875 -0.203125 -1.9375l0 -5.578125l-1.203125 0l0 -1.265625l1.203125 0l0 -2.390625l1.625 -0.984375l0 3.375l1.65625 0l0 1.265625l-1.65625 0l0 5.671875q0 0.6875 0.078125 0.890625q0.09375 0.203125 0.28125 0.328125q0.203125 0.109375 0.578125 0.109375q0.265625 0 0.71875 -0.0625zm8.230194 -1.640625l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m73.64304 101.46588l351.0551 0l0 53.70079l-351.0551 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m73.64304 101.46588l351.0551 0l0 53.70079l-351.0551 0z" fill-rule="evenodd"/><path fill="#000000" d="m215.67503 135.23627l0 -13.359367l1.640625 0l0 7.6249924l3.890625 -3.9374924l2.109375 0l-3.6875 3.5937424l4.0625 6.078125l-2.015625 0l-3.203125 -4.953125l-1.15625 1.125l0 3.828125l-1.640625 0zm12.90625 -1.46875l0.234375 1.453125q-0.6875 0.140625 -1.234375 0.140625q-0.890625 0 -1.390625 -0.28125q-0.484375 -0.28125 -0.6875 -0.734375q-0.203125 -0.46875 -0.203125 -1.9375l0 -5.5781174l-1.203125 0l0 -1.265625l1.203125 0l0 -2.390625l1.625 -0.984375l0 3.375l1.65625 0l0 1.265625l-1.65625 0l0 5.6718674q0 0.6875 0.078125 0.890625q0.09375 0.203125 0.28125 0.328125q0.203125 0.109375 0.578125 0.109375q0.265625 0 0.71875 -0.0625zm1.5583038 1.46875l0 -13.359367l1.640625 0l0 13.359367l-1.640625 0zm3.5354462 -2.890625l1.625 -0.25q0.125 0.96875 0.75 1.5q0.625 0.515625 1.75 0.515625q1.125 0 1.671875 -0.453125q0.546875 -0.46875 0.546875 -1.09375q0 -0.546875 -0.484375 -0.875q-0.328125 -0.21875 -1.671875 -0.546875q-1.8125 -0.46875 -2.515625 -0.796875q-0.6875 -0.328125 -1.046875 -0.90625q-0.359375 -0.59375 -0.359375 -1.3125q0 -0.6406174 0.296875 -1.1874924q0.296875 -0.5625 0.8125 -0.921875q0.375 -0.28125 1.03125 -0.46875q0.671875 -0.203125 1.421875 -0.203125q1.140625 0 2.0 0.328125q0.859375 0.328125 1.265625 0.890625q0.421875 0.5625 0.578125 1.4999924l-1.609375 0.21875q-0.109375 -0.7499924 -0.640625 -1.1718674q-0.515625 -0.421875 -1.46875 -0.421875q-1.140625 0 -1.625 0.375q-0.46875 0.375 -0.46875 0.875q0 0.31249237 0.1875 0.5781174q0.203125 0.265625 0.640625 0.4375q0.234375 0.09375 1.4375 0.421875q1.75 0.453125 2.4375 0.75q0.6875 0.296875 1.078125 0.859375q0.390625 0.5625 0.390625 1.40625q0 0.828125 -0.484375 1.546875q-0.46875 0.71875 -1.375 1.125q-0.90625 0.390625 -2.046875 0.390625q-1.875 0 -2.875 -0.78125q-0.984375 -0.78125 -1.25 -2.328125zm24.136429 -10.468742l1.765625 0l0 7.7187424q0 2.015625 -0.453125 3.203125q-0.453125 1.1875 -1.640625 1.9375q-1.1875 0.734375 -3.125 0.734375q-1.875 0 -3.078125 -0.640625q-1.1875 -0.65625 -1.703125 -1.875q-0.5 -1.234375 -0.5 -3.359375l0 -7.7187424l1.765625 0l0 7.7187424q0 1.734375 0.3125 2.5625q0.328125 0.8125 1.109375 1.265625q0.796875 0.453125 1.9375 0.453125q1.953125 0 2.78125 -0.890625q0.828125 -0.890625 0.828125 -3.390625l0 -7.7187424zm4.629181 13.359367l0 -13.359367l1.78125 0l0 11.781242l6.5625 0l0 1.578125l-8.34375 0zm10.453857 0l0 -13.359367l5.046875 0q1.328125 0 2.03125 0.125q0.96875 0.171875 1.640625 0.640625q0.671875 0.453125 1.078125 1.28125q0.40625 0.828125 0.40625 1.828125q0 1.703125 -1.09375 2.8906174q-1.078125 1.171875 -3.921875 1.171875l-3.421875 0l0 5.421875l-1.765625 0zm1.765625 -7.0l3.453125 0q1.71875 0 2.4375 -0.6406174q0.71875 -0.640625 0.71875 -1.796875q0 -0.84375 -0.421875 -1.4375q-0.421875 -0.59375 -1.125 -0.78125q-0.4375 -0.125 -1.640625 -0.125l-3.421875 0l0 4.7812424z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m73.64304 216.38058l351.0551 0l0 53.700775l-351.0551 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m73.64304 216.38058l351.0551 0l0 53.700775l-351.0551 0z" fill-rule="evenodd"/><path fill="#000000" d="m211.16338 250.15097l0 -11.78125l-4.40625 0l0 -1.578125l10.578125 0l0 1.578125l-4.40625 0l0 11.78125l-1.765625 0zm17.52098 -4.6875l1.765625 0.453125q-0.5625 2.171875 -2.0 3.328125q-1.4375 1.140625 -3.53125 1.140625q-2.15625 0 -3.515625 -0.875q-1.34375 -0.890625 -2.0625 -2.546875q-0.703125 -1.671875 -0.703125 -3.59375q0 -2.078125 0.796875 -3.625q0.796875 -1.5625 2.265625 -2.359375q1.484375 -0.8125 3.25 -0.8125q2.0 0 3.359375 1.015625q1.375 1.015625 1.90625 2.875l-1.734375 0.40625q-0.46875 -1.453125 -1.359375 -2.109375q-0.875 -0.671875 -2.203125 -0.671875q-1.546875 0 -2.578125 0.734375q-1.03125 0.734375 -1.453125 1.984375q-0.421875 1.234375 -0.421875 2.5625q0 1.703125 0.5 2.96875q0.5 1.265625 1.546875 1.90625q1.046875 0.625 2.265625 0.625q1.484375 0 2.515625 -0.859375q1.03125 -0.859375 1.390625 -2.546875zm3.9416962 4.6875l0 -13.359375l5.046875 0q1.328125 0 2.03125 0.125q0.96875 0.171875 1.640625 0.640625q0.671875 0.453125 1.078125 1.28125q0.40625 0.828125 0.40625 1.828125q0 1.703125 -1.09375 2.890625q-1.078125 1.171875 -3.921875 1.171875l-3.421875 0l0 5.421875l-1.765625 0zm1.765625 -7.0l3.453125 0q1.71875 0 2.4375 -0.640625q0.71875 -0.640625 0.71875 -1.796875q0 -0.84375 -0.421875 -1.4375q-0.421875 -0.59375 -1.125 -0.78125q-0.4375 -0.125 -1.640625 -0.125l-3.421875 0l0 4.78125zm14.664642 4.109375l1.625 -0.25q0.125 0.96875 0.75 1.5q0.625 0.515625 1.75 0.515625q1.125 0 1.671875 -0.453125q0.546875 -0.46875 0.546875 -1.09375q0 -0.546875 -0.484375 -0.875q-0.328125 -0.21875 -1.671875 -0.546875q-1.8125 -0.46875 -2.515625 -0.796875q-0.6875 -0.328125 -1.046875 -0.90625q-0.359375 -0.59375 -0.359375 -1.3125q0 -0.640625 0.296875 -1.1875q0.296875 -0.5625 0.8125 -0.921875q0.375 -0.28125 1.03125 -0.46875q0.671875 -0.203125 1.421875 -0.203125q1.140625 0 2.0 0.328125q0.859375 0.328125 1.2656097 0.890625q0.421875 0.5625 0.578125 1.5l-1.6093597 0.21875q-0.109375 -0.75 -0.640625 -1.171875q-0.515625 -0.421875 -1.46875 -0.421875q-1.140625 0 -1.625 0.375q-0.46875 0.375 -0.46875 0.875q0 0.3125 0.1875 0.578125q0.203125 0.265625 0.640625 0.4375q0.234375 0.09375 1.4375 0.421875q1.75 0.453125 2.4375 0.75q0.68748474 0.296875 1.0781097 0.859375q0.390625 0.5625 0.390625 1.40625q0 0.828125 -0.484375 1.546875q-0.46875 0.71875 -1.3749847 1.125q-0.90625 0.390625 -2.046875 0.390625q-1.875 0 -2.875 -0.78125q-0.984375 -0.78125 -1.25 -2.328125zm13.562485 1.421875l0.234375 1.453125q-0.6875 0.140625 -1.234375 0.140625q-0.890625 0 -1.390625 -0.28125q-0.484375 -0.28125 -0.6875 -0.734375q-0.203125 -0.46875 -0.203125 -1.9375l0 -5.578125l-1.203125 0l0 -1.265625l1.203125 0l0 -2.390625l1.625 -0.984375l0 3.375l1.65625 0l0 1.265625l-1.65625 0l0 5.671875q0 0.6875 0.078125 0.890625q0.09375 0.203125 0.28125 0.328125q0.203125 0.109375 0.578125 0.109375q0.265625 0 0.71875 -0.0625zm7.917694 0.28125q-0.921875 0.765625 -1.765625 1.09375q-0.828125 0.3125 -1.796875 0.3125q-1.59375 0 -2.453125 -0.78125q-0.859375 -0.78125 -0.859375 -1.984375q0 -0.71875 0.328125 -1.296875q0.328125 -0.59375 0.84375 -0.9375q0.53125 -0.359375 1.1875 -0.546875q0.46875 -0.125 1.453125 -0.25q1.984375 -0.234375 2.921875 -0.5625q0.015625 -0.34375 0.015625 -0.421875q0 -1.0 -0.46875 -1.421875q-0.625 -0.546875 -1.875 -0.546875q-1.15625 0 -1.703125 0.40625q-0.546875 0.40625 -0.8125 1.421875l-1.609375 -0.21875q0.21875 -1.015625 0.71875 -1.640625q0.5 -0.640625 1.453125 -0.984375q0.953125 -0.34375 2.1875 -0.34375q1.25 0 2.015625 0.296875q0.78125 0.28125 1.140625 0.734375q0.375 0.4375 0.515625 1.109375q0.078125 0.421875 0.078125 1.515625l0 2.1875q0 2.28125 0.109375 2.890625q0.109375 0.59375 0.40625 1.15625l-1.703125 0q-0.265625 -0.515625 -0.328125 -1.1875zm-0.140625 -3.671875q-0.890625 0.375 -2.671875 0.625q-1.015625 0.140625 -1.4375 0.328125q-0.421875 0.1875 -0.65625 0.53125q-0.21875 0.34375 -0.21875 0.78125q0 0.65625 0.5 1.09375q0.5 0.4375 1.453125 0.4375q0.9375 0 1.671875 -0.40625q0.75 -0.421875 1.09375 -1.140625q0.265625 -0.5625 0.265625 -1.640625l0 -0.609375zm10.516327 1.3125l1.609375 0.21875q-0.265625 1.65625 -1.359375 2.609375q-1.078125 0.9375 -2.671875 0.9375q-1.984375 0 -3.1875 -1.296875q-1.203125 -1.296875 -1.203125 -3.71875q0 -1.578125 0.515625 -2.75q0.515625 -1.171875 1.578125 -1.75q1.0625 -0.59375 2.3125 -0.59375q1.578125 0 2.578125 0.796875q1.0 0.796875 1.28125 2.265625l-1.59375 0.234375q-0.234375 -0.96875 -0.8125 -1.453125q-0.578125 -0.5 -1.390625 -0.5q-1.234375 0 -2.015625 0.890625q-0.78125 0.890625 -0.78125 2.8125q0 1.953125 0.75 2.84375q0.75 0.875 1.953125 0.875q0.96875 0 1.609375 -0.59375q0.65625 -0.59375 0.828125 -1.828125zm3.015625 3.546875l0 -13.359375l1.640625 0l0 7.625l3.890625 -3.9375l2.109375 0l-3.6875 3.59375l4.0625 6.078125l-2.015625 0l-3.203125 -4.953125l-1.15625 1.125l0 3.828125l-1.640625 0z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m73.64304 331.2953l351.0551 0l0 53.700775l-351.0551 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m73.64304 331.2953l351.0551 0l0 53.700775l-351.0551 0z" fill-rule="evenodd"/><path fill="#000000" d="m225.73463 365.06567l0 -13.359375l4.609375 0q1.546875 0 2.375 0.203125q1.140625 0.25 1.953125 0.953125q1.0625 0.890625 1.578125 2.28125q0.53125 1.390625 0.53125 3.171875q0 1.515625 -0.359375 2.703125q-0.359375 1.171875 -0.921875 1.9375q-0.546875 0.765625 -1.203125 1.21875q-0.65625 0.4375 -1.59375 0.671875q-0.9375 0.21875 -2.140625 0.21875l-4.828125 0zm1.765625 -1.578125l2.859375 0q1.3125 0 2.0625 -0.234375q0.75 -0.25 1.203125 -0.703125q0.625 -0.625 0.96875 -1.6875q0.359375 -1.0625 0.359375 -2.578125q0 -2.09375 -0.6875 -3.21875q-0.6875 -1.125 -1.671875 -1.5q-0.703125 -0.28125 -2.28125 -0.28125l-2.8125 0l0 10.203125zm11.488571 1.578125l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0zm6.228302 -11.46875l0 -1.890625l1.640625 0l0 1.890625l-1.640625 0zm0 11.46875l0 -9.671875l1.640625 0l0 9.671875l-1.640625 0zm6.832321 0l-3.6875 -9.671875l1.734375 0l2.078125 5.796875q0.328125 0.9375 0.625 1.9375q0.203125 -0.765625 0.609375 -1.828125l2.140625 -5.90625l1.6874847 0l-3.6562347 9.671875l-1.53125 0zm13.26561 -3.109375l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625zm9.125732 5.765625l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m73.64304 446.20996l351.0551 0l0 53.700806l-351.0551 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m73.64304 446.20996l351.0551 0l0 53.700806l-351.0551 0z" fill-rule="evenodd"/><path fill="#000000" d="m222.09538 479.98038l0 -13.359375l4.609375 0q1.546875 0 2.375 0.203125q1.140625 0.25 1.953125 0.953125q1.0625 0.890625 1.578125 2.28125q0.53125 1.390625 0.53125 3.171875q0 1.515625 -0.359375 2.703125q-0.359375 1.171875 -0.921875 1.9375q-0.546875 0.765625 -1.203125 1.21875q-0.65625 0.4375 -1.59375 0.671875q-0.9375 0.21875 -2.140625 0.21875l-4.828125 0zm1.765625 -1.578125l2.859375 0q1.3125 0 2.0625 -0.234375q0.75 -0.25 1.203125 -0.703125q0.625 -0.625 0.96875 -1.6875q0.359375 -1.0625 0.359375 -2.578125q0 -2.09375 -0.6875 -3.21875q-0.6875 -1.125 -1.671875 -1.5q-0.703125 -0.28125 -2.28125 -0.28125l-2.8125 0l0 10.203125zm18.129196 -1.53125l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625zm11.828842 5.765625l-3.6875 -9.671875l1.734375 0l2.078125 5.796875q0.328125 0.9375 0.625 1.9375q0.203125 -0.765625 0.609375 -1.828125l2.140625 -5.90625l1.6875 0l-3.65625 9.671875l-1.53125 0zm6.640625 -11.46875l0 -1.890625l1.6406097 0l0 1.890625l-1.6406097 0zm0 11.46875l0 -9.671875l1.6406097 0l0 9.671875l-1.6406097 0zm10.457321 -3.546875l1.609375 0.21875q-0.265625 1.65625 -1.359375 2.609375q-1.078125 0.9375 -2.671875 0.9375q-1.984375 0 -3.1875 -1.296875q-1.203125 -1.296875 -1.203125 -3.71875q0 -1.578125 0.515625 -2.75q0.515625 -1.171875 1.578125 -1.75q1.0625 -0.59375 2.3125 -0.59375q1.578125 0 2.578125 0.796875q1.0 0.796875 1.28125 2.265625l-1.59375 0.234375q-0.234375 -0.96875 -0.8125 -1.453125q-0.578125 -0.5 -1.390625 -0.5q-1.234375 0 -2.015625 0.890625q-0.78125 0.890625 -0.78125 2.8125q0 1.953125 0.75 2.84375q0.75 0.875 1.953125 0.875q0.96875 0 1.609375 -0.59375q0.65625 -0.59375 0.828125 -1.828125zm9.640625 0.4375l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m153.17061 40.25197l0 0" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m153.17061 40.25197l0 0" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m48.435696 73.03937l402.67715 0" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" stroke-dasharray="4.0,3.0" d="m48.435696 73.03937l402.67715 0" fill-rule="evenodd"/><path fill="#cfe2f3" d="m177.95801 71.49061l-12.393707 0l0 12.897636l-24.7874 0l0 -12.897636l-12.393692 0l24.7874 -12.89764z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m177.95801 71.49061l-12.393707 0l0 12.897636l-24.7874 0l0 -12.897636l-12.393692 0l24.7874 -12.89764z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m320.3832 71.49061l12.393707 0l0 -12.89764l24.787384 0l0 12.89764l12.393707 0l-24.787415 12.897636z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m320.3832 71.49061l12.393707 0l0 -12.89764l24.787384 0l0 12.89764l12.393707 0l-24.787415 12.897636z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m177.95801 188.3931l-12.393707 0l0 12.897629l-24.7874 0l0 -12.897629l-12.393692 0l24.7874 -12.897644z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m177.95801 188.3931l-12.393707 0l0 12.897629l-24.7874 0l0 -12.897629l-12.393692 0l24.7874 -12.897644z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m320.3832 188.3931l12.393707 0l0 -12.897644l24.787384 0l0 12.897644l12.393707 0l-24.787415 12.897629z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m320.3832 188.3931l12.393707 0l0 -12.897644l24.787384 0l0 12.897644l12.393707 0l-24.787415 12.897629z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m177.95801 301.4256l-12.393707 0l0 12.897644l-24.7874 0l0 -12.897644l-12.393692 0l24.7874 -12.897644z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m177.95801 301.4256l-12.393707 0l0 12.897644l-24.7874 0l0 -12.897644l-12.393692 0l24.7874 -12.897644z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m320.3832 301.4256l12.393707 0l0 -12.897644l24.787384 0l0 12.897644l12.393707 0l-24.787415 12.897644z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m320.3832 301.4256l12.393707 0l0 -12.897644l24.787384 0l0 12.897644l12.393707 0l-24.787415 12.897644z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m177.95801 415.4906l-12.393707 0l0 12.897644l-24.7874 0l0 -12.897644l-12.393692 0l24.7874 -12.897644z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m177.95801 415.4906l-12.393707 0l0 12.897644l-24.7874 0l0 -12.897644l-12.393692 0l24.7874 -12.897644z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m320.3832 415.4906l12.393707 0l0 -12.897644l24.787384 0l0 12.897644l12.393707 0l-24.787415 12.897644z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m320.3832 415.4906l12.393707 0l0 -12.897644l24.787384 0l0 12.897644l12.393707 0l-24.787415 12.897644z" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m198.14961 44.009186l109.44881 0l0 53.70079l-109.44881 0z" fill-rule="evenodd"/><path fill="#000000" d="m224.98174 70.929184l2.78125 -13.359375l1.65625 0l-1.0 4.78125q0.78125 -0.71875 1.421875 -1.015625q0.640625 -0.296875 1.328125 -0.296875q1.359375 0 2.265625 1.015625q0.90625 1.0 0.90625 2.9375q0 1.28125 -0.375 2.359375q-0.359375 1.0625 -0.890625 1.78125q-0.53125 0.71875 -1.109375 1.15625q-0.578125 0.4375 -1.1875 0.640625q-0.59375 0.21875 -1.140625 0.21875q-0.96875 0 -1.703125 -0.5q-0.71875 -0.515625 -1.125 -1.546875l-0.390625 1.828125l-1.4375 0zm2.4375 -3.96875l-0.015625 0.3125q0 1.234375 0.59375 1.890625q0.59375 0.640625 1.484375 0.640625q0.859375 0 1.578125 -0.609375q0.734375 -0.609375 1.1875 -1.890625q0.46875 -1.28125 0.46875 -2.375q0 -1.21875 -0.59375 -1.890625q-0.578125 -0.671875 -1.4375 -0.671875q-0.90625 0 -1.65625 0.6875q-0.734375 0.6875 -1.234375 2.125q-0.375 1.0625 -0.375 1.78125zm14.531967 2.21875q-1.734375 1.96875 -3.5625 1.96875q-1.109375 0 -1.796875 -0.640625q-0.6875 -0.640625 -0.6875 -1.578125q0 -0.609375 0.296875 -2.09375l1.171875 -5.578125l1.65625 0l-1.296875 6.1875q-0.171875 0.765625 -0.171875 1.203125q0 0.546875 0.328125 0.859375q0.34375 0.296875 0.984375 0.296875q0.703125 0 1.359375 -0.328125q0.65625 -0.34375 1.125 -0.921875q0.484375 -0.578125 0.796875 -1.359375q0.1875 -0.5 0.453125 -1.765625l0.875 -4.171875l1.65625 0l-2.03125 9.671875l-1.515625 0l0.359375 -1.75zm4.000717 1.75l1.765625 -8.40625l-1.484375 0l0.265625 -1.265625l1.484375 0l0.28125 -1.375q0.21875 -1.03125 0.4375 -1.484375q0.234375 -0.453125 0.75 -0.75q0.53125 -0.296875 1.4375 -0.296875q0.625 0 1.828125 0.265625l-0.296875 1.4375q-0.84375 -0.21875 -1.40625 -0.21875q-0.484375 0 -0.734375 0.25q-0.25 0.234375 -0.4375 1.125l-0.21875 1.046875l1.84375 0l-0.265625 1.265625l-1.84375 0l-1.75 8.40625l-1.65625 0zm5.183304 0l1.765625 -8.40625l-1.484375 0l0.265625 -1.265625l1.484375 0l0.28125 -1.375q0.21875 -1.03125 0.4375 -1.484375q0.234375 -0.453125 0.75 -0.75q0.53125 -0.296875 1.4375 -0.296875q0.625 0 1.828125 0.265625l-0.296875 1.4375q-0.84375 -0.21875 -1.40625 -0.21875q-0.484375 0 -0.734375 0.25q-0.25 0.234375 -0.4375 1.125l-0.21875 1.046875l1.84375 0l-0.265625 1.265625l-1.84375 0l-1.75 8.40625l-1.65625 0zm12.058319 -3.28125l1.609375 0.15625q-0.34375 1.1875 -1.59375 2.265625q-1.234375 1.078125 -2.96875 1.078125q-1.0625 0 -1.96875 -0.5q-0.890625 -0.5 -1.359375 -1.4375q-0.46875 -0.953125 -0.46875 -2.15625q0 -1.59375 0.734375 -3.078125q0.734375 -1.484375 1.890625 -2.203125q1.171875 -0.734375 2.53125 -0.734375q1.734375 0 2.765625 1.078125q1.03125 1.0625 1.03125 2.921875q0 0.71875 -0.125 1.46875l-7.125 0q-0.046875 0.28125 -0.046875 0.5q0 1.359375 0.625 2.078125q0.625 0.71875 1.53125 0.71875q0.84375 0 1.65625 -0.546875q0.828125 -0.5625 1.28125 -1.609375zm-4.78125 -2.40625l5.421875 0q0.015625 -0.25 0.015625 -0.359375q0 -1.234375 -0.625 -1.890625q-0.625 -0.671875 -1.59375 -0.671875q-1.0625 0 -1.9375 0.734375q-0.859375 0.71875 -1.28125 2.1875zm8.063202 5.6875l2.015625 -9.671875l1.453125 0l-0.40625 1.96875q0.75 -1.109375 1.453125 -1.640625q0.71875 -0.546875 1.46875 -0.546875q0.5 0 1.21875 0.359375l-0.671875 1.53125q-0.4375 -0.3125 -0.9375 -0.3125q-0.875 0 -1.78125 0.96875q-0.90625 0.953125 -1.4375 3.46875l-0.8125 3.875l-1.5625 0zm6.368927 -3.3125l1.640625 -0.09375q0 0.703125 0.21875 1.21875q0.21875 0.5 0.796875 0.8125q0.59375 0.3125 1.375 0.3125q1.09375 0 1.640625 -0.4375q0.546875 -0.4375 0.546875 -1.015625q0 -0.4375 -0.328125 -0.8125q-0.328125 -0.390625 -1.640625 -0.953125q-1.3125 -0.5625 -1.671875 -0.78125q-0.609375 -0.375 -0.921875 -0.875q-0.3125 -0.515625 -0.3125 -1.171875q0 -1.140625 0.90625 -1.953125q0.921875 -0.828125 2.5625 -0.828125q1.828125 0 2.765625 0.84375q0.953125 0.84375 1.0 2.21875l-1.609375 0.109375q-0.046875 -0.875 -0.625 -1.390625q-0.578125 -0.515625 -1.65625 -0.515625q-0.84375 0 -1.328125 0.40625q-0.46875 0.390625 -0.46875 0.84375q0 0.453125 0.40625 0.796875q0.28125 0.234375 1.421875 0.734375q1.890625 0.8125 2.375 1.28125q0.78125 0.765625 0.78125 1.84375q0 0.71875 -0.4375 1.421875q-0.4375 0.6875 -1.34375 1.109375q-0.90625 0.40625 -2.140625 0.40625q-1.671875 0 -2.84375 -0.828125q-1.1875 -0.828125 -1.109375 -2.703125z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m198.14961 164.00919l109.44881 0l0 53.70079l-109.44881 0z" fill-rule="evenodd"/><path fill="#000000" d="m211.6696 187.61668l1.640625 -0.09375q0 0.703125 0.21875 1.21875q0.21875 0.5 0.796875 0.8125q0.59375 0.3125 1.375 0.3125q1.09375 0 1.640625 -0.4375q0.546875 -0.4375 0.546875 -1.015625q0 -0.4375 -0.328125 -0.8125q-0.328125 -0.390625 -1.640625 -0.953125q-1.3125 -0.5625 -1.671875 -0.78125q-0.609375 -0.375 -0.921875 -0.875q-0.3125 -0.515625 -0.3125 -1.171875q0 -1.140625 0.90625 -1.953125q0.921875 -0.828125 2.5625 -0.828125q1.828125 0 2.765625 0.84375q0.953125 0.84375 1.0 2.21875l-1.609375 0.109375q-0.046875 -0.875 -0.625 -1.390625q-0.578125 -0.515625 -1.65625 -0.515625q-0.84375 0 -1.328125 0.40625q-0.46875 0.390625 -0.46875 0.84375q0 0.453125 0.40625 0.796875q0.28125 0.234375 1.421875 0.734375q1.890625 0.8125 2.375 1.28125q0.78125 0.765625 0.78125 1.84375q0 0.71875 -0.4375 1.421875q-0.4375 0.6875 -1.34375 1.109375q-0.90625 0.40625 -2.140625 0.40625q-1.671875 0 -2.84375 -0.828125q-1.1875 -0.828125 -1.109375 -2.703125zm15.84375 -0.21875l1.65625 0.171875q-0.625 1.8125 -1.765625 2.703125q-1.140625 0.875 -2.609375 0.875q-1.578125 0 -2.5625 -1.015625q-0.96875 -1.03125 -0.96875 -2.859375q0 -1.578125 0.625 -3.109375q0.640625 -1.53125 1.796875 -2.328125q1.171875 -0.796875 2.6875 -0.796875q1.546875 0 2.453125 0.875q0.921875 0.875 0.921875 2.328125l-1.625 0.109375q-0.015625 -0.921875 -0.546875 -1.4375q-0.515625 -0.515625 -1.359375 -0.515625q-1.0 0 -1.734375 0.625q-0.71875 0.625 -1.140625 1.90625q-0.40625 1.28125 -0.40625 2.46875q0 1.234375 0.546875 1.859375q0.546875 0.609375 1.34375 0.609375q0.796875 0 1.53125 -0.609375q0.734375 -0.609375 1.15625 -1.859375zm9.171875 2.328125q-0.859375 0.734375 -1.65625 1.078125q-0.78125 0.34375 -1.6875 0.34375q-1.34375 0 -2.171875 -0.78125q-0.8125 -0.796875 -0.8125 -2.03125q0 -0.796875 0.375 -1.421875q0.375 -0.625 0.984375 -1.0q0.609375 -0.390625 1.484375 -0.546875q0.5625 -0.109375 2.109375 -0.171875q1.5625 -0.0625 2.234375 -0.328125q0.1875 -0.671875 0.1875 -1.125q0 -0.578125 -0.421875 -0.90625q-0.5625 -0.453125 -1.671875 -0.453125q-1.03125 0 -1.703125 0.46875q-0.65625 0.453125 -0.953125 1.296875l-1.671875 -0.140625q0.515625 -1.4375 1.609375 -2.203125q1.109375 -0.765625 2.796875 -0.765625q1.796875 0 2.84375 0.859375q0.796875 0.625 0.796875 1.65625q0 0.765625 -0.21875 1.796875l-0.53125 2.40625q-0.265625 1.140625 -0.265625 1.859375q0 0.453125 0.203125 1.3125l-1.671875 0q-0.125 -0.46875 -0.1875 -1.203125zm0.609375 -3.703125q-0.34375 0.140625 -0.75 0.21875q-0.390625 0.0625 -1.3125 0.15625q-1.4375 0.125 -2.03125 0.328125q-0.59375 0.1875 -0.90625 0.625q-0.296875 0.421875 -0.296875 0.9375q0 0.6875 0.484375 1.140625q0.484375 0.4375 1.359375 0.4375q0.828125 0 1.578125 -0.421875q0.75 -0.4375 1.1875 -1.203125q0.4375 -0.78125 0.6875 -2.21875zm7.094467 3.5625l-0.265625 1.359375q-0.59375 0.140625 -1.15625 0.140625q-0.984375 0 -1.5625 -0.46875q-0.4375 -0.375 -0.4375 -1.0q0 -0.3125 0.234375 -1.46875l1.171875 -5.625l-1.296875 0l0.265625 -1.265625l1.296875 0l0.5 -2.375l1.890625 -1.140625l-0.734375 3.515625l1.625 0l-0.28125 1.265625l-1.609375 0l-1.125 5.359375q-0.203125 1.015625 -0.203125 1.21875q0 0.28125 0.15625 0.4375q0.171875 0.15625 0.5625 0.15625q0.546875 0 0.96875 -0.109375zm5.183304 0l-0.265625 1.359375q-0.59375 0.140625 -1.15625 0.140625q-0.984375 0 -1.5625 -0.46875q-0.4375 -0.375 -0.4375 -1.0q0 -0.3125 0.234375 -1.46875l1.171875 -5.625l-1.296875 0l0.265625 -1.265625l1.296875 0l0.5 -2.375l1.890625 -1.140625l-0.734375 3.515625l1.625 0l-0.28125 1.265625l-1.609375 0l-1.125 5.359375q-0.203125 1.015625 -0.203125 1.21875q0 0.28125 0.15625 0.4375q0.171875 0.15625 0.5625 0.15625q0.546875 0 0.96875 -0.109375zm8.433304 -1.9375l1.609375 0.15625q-0.34375 1.1875 -1.59375 2.265625q-1.234375 1.078125 -2.96875 1.078125q-1.0625 0 -1.96875 -0.5q-0.890625 -0.5 -1.359375 -1.4375q-0.46875 -0.953125 -0.46875 -2.15625q0 -1.59375 0.734375 -3.078125q0.734375 -1.484375 1.890625 -2.203125q1.171875 -0.734375 2.53125 -0.734375q1.734375 0 2.765625 1.078125q1.03125 1.0625 1.03125 2.921875q0 0.71875 -0.125 1.46875l-7.125 0q-0.046875 0.28125 -0.046875 0.5q0 1.359375 0.625 2.078125q0.625 0.71875 1.53125 0.71875q0.84375 0 1.65625 -0.546875q0.828125 -0.5625 1.28125 -1.609375zm-4.78125 -2.40625l5.421875 0q0.015625 -0.25 0.015625 -0.359375q0 -1.234375 -0.625 -1.890625q-0.625 -0.671875 -1.59375 -0.671875q-1.0625 0 -1.9375 0.734375q-0.859375 0.71875 -1.28125 2.1875zm8.063202 5.6875l2.015625 -9.671875l1.453125 0l-0.40625 1.96875q0.75 -1.109375 1.453125 -1.640625q0.71875 -0.546875 1.46875 -0.546875q0.5 0 1.21875 0.359375l-0.671875 1.53125q-0.4375 -0.3125 -0.9375 -0.3125q-0.875 0 -1.78125 0.96875q-0.90625 0.953125 -1.4375 3.46875l-0.8125 3.875l-1.5625 0zm11.255371 0l2.796875 -13.359375l1.640625 0l-2.78125 13.359375l-1.65625 0zm6.613556 -11.484375l0.40625 -1.875l1.625 0l-0.390625 1.875l-1.640625 0zm-2.390625 11.484375l2.015625 -9.671875l1.65625 0l-2.03125 9.671875l-1.640625 0zm4.3635864 -3.3125l1.640625 -0.09375q0 0.703125 0.21875 1.21875q0.21875 0.5 0.796875 0.8125q0.59375 0.3125 1.375 0.3125q1.09375 0 1.640625 -0.4375q0.546875 -0.4375 0.546875 -1.015625q0 -0.4375 -0.328125 -0.8125q-0.328125 -0.390625 -1.640625 -0.953125q-1.3125 -0.5625 -1.671875 -0.78125q-0.609375 -0.375 -0.921875 -0.875q-0.3125 -0.515625 -0.3125 -1.171875q0 -1.140625 0.90625 -1.953125q0.921875 -0.828125 2.5625 -0.828125q1.828125 0 2.765625 0.84375q0.953125 0.84375 1.0 2.21875l-1.609375 0.109375q-0.046875 -0.875 -0.625 -1.390625q-0.578125 -0.515625 -1.65625 -0.515625q-0.84375 0 -1.328125 0.40625q-0.46875 0.390625 -0.46875 0.84375q0 0.453125 0.40625 0.796875q0.28125 0.234375 1.421875 0.734375q1.890625 0.8125 2.375 1.28125q0.78125 0.765625 0.78125 1.84375q0 0.71875 -0.4375 1.421875q-0.4375 0.6875 -1.34375 1.109375q-0.90625 0.40625 -2.140625 0.40625q-1.671875 0 -2.84375 -0.828125q-1.1875 -0.828125 -1.109375 -2.703125zm13.015625 1.96875l-0.265625 1.359375q-0.59375 0.140625 -1.15625 0.140625q-0.984375 0 -1.5625 -0.46875q-0.4375 -0.375 -0.4375 -1.0q0 -0.3125 0.234375 -1.46875l1.171875 -5.625l-1.296875 0l0.265625 -1.265625l1.296875 0l0.5 -2.375l1.890625 -1.140625l-0.734375 3.515625l1.625 0l-0.28125 1.265625l-1.609375 0l-1.125 5.359375q-0.203125 1.015625 -0.203125 1.21875q0 0.28125 0.15625 0.4375q0.171875 0.15625 0.5625 0.15625q0.546875 0 0.96875 -0.109375z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m198.14961 280.1392l109.44881 0l0 53.700806l-109.44881 0z" fill-rule="evenodd"/><path fill="#000000" d="m223.58026 303.7467l1.640625 -0.09375q0 0.703125 0.21875 1.21875q0.21875 0.5 0.796875 0.8125q0.59375 0.3125 1.375 0.3125q1.09375 0 1.640625 -0.4375q0.546875 -0.4375 0.546875 -1.015625q0 -0.4375 -0.328125 -0.8125q-0.328125 -0.390625 -1.640625 -0.953125q-1.3125 -0.5625 -1.671875 -0.78125q-0.609375 -0.375 -0.921875 -0.875q-0.3125 -0.515625 -0.3125 -1.171875q0 -1.140625 0.90625 -1.953125q0.921875 -0.828125 2.5625 -0.828125q1.828125 0 2.765625 0.84375q0.953125 0.84375 1.0 2.21875l-1.609375 0.109375q-0.046875 -0.875 -0.625 -1.390625q-0.578125 -0.515625 -1.65625 -0.515625q-0.84375 0 -1.328125 0.40625q-0.46875 0.390625 -0.46875 0.84375q0 0.453125 0.40625 0.796875q0.28125 0.234375 1.421875 0.734375q1.890625 0.8125 2.375 1.28125q0.78125 0.765625 0.78125 1.84375q0 0.71875 -0.4375 1.421875q-0.4375 0.6875 -1.34375 1.109375q-0.90625 0.40625 -2.140625 0.40625q-1.671875 0 -2.84375 -0.828125q-1.1875 -0.828125 -1.109375 -2.703125zm9.1875 3.3125l2.796875 -13.359375l1.640625 0l-1.734375 8.28125l4.8125 -4.59375l2.171875 0l-4.125 3.609375l2.5 6.0625l-1.8125 0l-1.921875 -4.96875l-2.015625 1.734375l-0.671875 3.234375l-1.640625 0zm7.5 3.703125l0 -1.1875l10.875 0l0 1.1875l-10.875 0zm12.188217 -3.703125l2.78125 -13.359375l1.6562653 0l-1.0000153 4.78125q0.78126526 -0.71875 1.4218903 -1.015625q0.640625 -0.296875 1.328125 -0.296875q1.359375 0 2.265625 1.015625q0.90625 1.0 0.90625 2.9375q0 1.28125 -0.375 2.359375q-0.359375 1.0625 -0.890625 1.78125q-0.53125 0.71875 -1.109375 1.15625q-0.578125 0.4375 -1.1875 0.640625q-0.59375 0.21875 -1.140625 0.21875q-0.96875 0 -1.7031403 -0.5q-0.71875 -0.515625 -1.125 -1.546875l-0.390625 1.828125l-1.4375 0zm2.4375 -3.96875l-0.015625 0.3125q0 1.234375 0.59375 1.890625q0.59376526 0.640625 1.4843903 0.640625q0.859375 0 1.578125 -0.609375q0.734375 -0.609375 1.1875 -1.890625q0.46875 -1.28125 0.46875 -2.375q0 -1.21875 -0.59375 -1.890625q-0.578125 -0.671875 -1.4375 -0.671875q-0.90625 0 -1.65625 0.6875q-0.73439026 0.6875 -1.2343903 2.125q-0.375 1.0625 -0.375 1.78125zm14.531967 2.21875q-1.734375 1.96875 -3.5625 1.96875q-1.109375 0 -1.796875 -0.640625q-0.6875 -0.640625 -0.6875 -1.578125q0 -0.609375 0.296875 -2.09375l1.171875 -5.578125l1.65625 0l-1.296875 6.1875q-0.171875 0.765625 -0.171875 1.203125q0 0.546875 0.328125 0.859375q0.34375 0.296875 0.984375 0.296875q0.703125 0 1.359375 -0.328125q0.65625 -0.34375 1.125 -0.921875q0.484375 -0.578125 0.796875 -1.359375q0.1875 -0.5 0.453125 -1.765625l0.875 -4.171875l1.65625 0l-2.03125 9.671875l-1.515625 0l0.359375 -1.75zm4.0007324 1.75l1.765625 -8.40625l-1.484375 0l0.265625 -1.265625l1.484375 0l0.28125 -1.375q0.21875 -1.03125 0.4375 -1.484375q0.234375 -0.453125 0.75 -0.75q0.53125 -0.296875 1.4375 -0.296875q0.625 0 1.828125 0.265625l-0.296875 1.4375q-0.84375 -0.21875 -1.40625 -0.21875q-0.484375 0 -0.734375 0.25q-0.25 0.234375 -0.4375 1.125l-0.21875 1.046875l1.84375 0l-0.265625 1.265625l-1.84375 0l-1.75 8.40625l-1.65625 0zm5.1832886 0l1.765625 -8.40625l-1.484375 0l0.265625 -1.265625l1.484375 0l0.28125 -1.375q0.21875 -1.03125 0.4375 -1.484375q0.234375 -0.453125 0.75 -0.75q0.53125 -0.296875 1.4375 -0.296875q0.625 0 1.828125 0.265625l-0.296875 1.4375q-0.84375 -0.21875 -1.40625 -0.21875q-0.484375 0 -0.734375 0.25q-0.25 0.234375 -0.4375 1.125l-0.21875 1.046875l1.84375 0l-0.265625 1.265625l-1.84375 0l-1.75 8.40625l-1.65625 0z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m163.81627 395.2362l178.11024 0l0 53.700806l-178.11024 0z" fill-rule="evenodd"/><path fill="#000000" d="m195.60925 418.62497l1.65625 0.171875q-0.625 1.8125 -1.765625 2.703125q-1.140625 0.875 -2.609375 0.875q-1.578125 0 -2.5625 -1.015625q-0.96875 -1.03125 -0.96875 -2.859375q0 -1.578125 0.625 -3.109375q0.640625 -1.53125 1.796875 -2.328125q1.171875 -0.796875 2.6875 -0.796875q1.546875 0 2.453125 0.875q0.921875 0.875 0.921875 2.328125l-1.625 0.109375q-0.015625 -0.921875 -0.546875 -1.4375q-0.515625 -0.515625 -1.359375 -0.515625q-1.0 0 -1.734375 0.625q-0.71875 0.625 -1.140625 1.90625q-0.40625 1.28125 -0.40625 2.46875q0 1.234375 0.546875 1.859375q0.546875 0.609375 1.34375 0.609375q0.796875 0 1.53125 -0.609375q0.734375 -0.609375 1.15625 -1.859375zm2.9375 -0.140625q0 -2.828125 1.671875 -4.6875q1.375 -1.53125 3.609375 -1.53125q1.75 0 2.8125 1.09375q1.078125 1.09375 1.078125 2.953125q0 1.65625 -0.671875 3.09375q-0.671875 1.4375 -1.921875 2.203125q-1.25 0.765625 -2.625 0.765625q-1.125 0 -2.046875 -0.484375q-0.921875 -0.484375 -1.421875 -1.359375q-0.484375 -0.890625 -0.484375 -2.046875zm1.65625 -0.15625q0 1.359375 0.65625 2.0625q0.65625 0.703125 1.65625 0.703125q0.53125 0 1.046875 -0.203125q0.53125 -0.21875 0.96875 -0.65625q0.453125 -0.4375 0.765625 -1.0q0.3125 -0.5625 0.5 -1.203125q0.28125 -0.90625 0.28125 -1.734375q0 -1.3125 -0.65625 -2.03125q-0.65625 -0.734375 -1.65625 -0.734375q-0.78125 0 -1.421875 0.375q-0.625 0.375 -1.140625 1.09375q-0.515625 0.703125 -0.765625 1.640625q-0.234375 0.9375 -0.234375 1.6875zm8.438217 3.828125l2.015625 -9.671875l1.5 0l-0.359375 1.6875q0.96875 -1.0 1.8125 -1.453125q0.859375 -0.453125 1.734375 -0.453125q1.1875 0 1.84375 0.640625q0.671875 0.625 0.671875 1.703125q0 0.53125 -0.234375 1.6875l-1.234375 5.859375l-1.640625 0l1.28125 -6.125q0.1875 -0.890625 0.1875 -1.328125q0 -0.484375 -0.328125 -0.78125q-0.328125 -0.296875 -0.953125 -0.296875q-1.28125 0 -2.265625 0.90625q-0.984375 0.90625 -1.453125 3.125l-0.9375 4.5l-1.640625 0zm14.219467 -1.34375l-0.265625 1.359375q-0.59375 0.140625 -1.15625 0.140625q-0.984375 0 -1.5625 -0.46875q-0.4375 -0.375 -0.4375 -1.0q0 -0.3125 0.234375 -1.46875l1.171875 -5.625l-1.296875 0l0.265625 -1.265625l1.296875 0l0.5 -2.375l1.890625 -1.140625l-0.734375 3.515625l1.625 0l-0.28125 1.265625l-1.609375 0l-1.125 5.359375q-0.203125 1.015625 -0.203125 1.21875q0 0.28125 0.15625 0.4375q0.171875 0.15625 0.5625 0.15625q0.546875 0 0.96875 -0.109375zm8.433304 -1.9375l1.609375 0.15625q-0.34375 1.1875 -1.59375 2.265625q-1.234375 1.078125 -2.96875 1.078125q-1.0625 0 -1.96875 -0.5q-0.890625 -0.5 -1.359375 -1.4375q-0.46875 -0.953125 -0.46875 -2.15625q0 -1.59375 0.734375 -3.078125q0.734375 -1.484375 1.890625 -2.203125q1.171875 -0.734375 2.53125 -0.734375q1.734375 0 2.765625 1.078125q1.03125 1.0625 1.03125 2.921875q0 0.71875 -0.125 1.46875l-7.125 0q-0.046875 0.28125 -0.046875 0.5q0 1.359375 0.625 2.078125q0.625 0.71875 1.53125 0.71875q0.84375 0 1.65625 -0.546875q0.828125 -0.5625 1.28125 -1.609375zm-4.78125 -2.40625l5.421875 0q0.015625 -0.25 0.015625 -0.359375q0 -1.234375 -0.625 -1.890625q-0.625 -0.671875 -1.59375 -0.671875q-1.0625 0 -1.9375 0.734375q-0.859375 0.71875 -1.28125 2.1875zm7.406967 5.6875l4.21875 -4.90625l-2.421875 -4.765625l1.828125 0l0.8125 1.71875q0.453125 0.96875 0.828125 1.84375l2.78125 -3.5625l2.015625 0l-4.0625 4.890625l2.4375 4.78125l-1.8125 0l-0.96875 -1.96875q-0.3125 -0.625 -0.703125 -1.5625l-2.875 3.53125l-2.078125 0zm13.828125 -1.34375l-0.265625 1.359375q-0.59375 0.140625 -1.15625 0.140625q-0.984375 0 -1.5625 -0.46875q-0.4375 -0.375 -0.4375 -1.0q0 -0.3125 0.234375 -1.46875l1.171875 -5.625l-1.296875 0l0.265625 -1.265625l1.296875 0l0.5 -2.375l1.890625 -1.140625l-0.734375 3.515625l1.625 0l-0.28125 1.265625l-1.609375 0l-1.125 5.359375q-0.203125 1.015625 -0.203125 1.21875q0 0.28125 0.15625 0.4375q0.171875 0.15625 0.5625 0.15625q0.546875 0 0.96875 -0.109375zm11.210358 -0.8125l0 -3.671875l-3.640625 0l0 -1.515625l3.640625 0l0 -3.640625l1.546875 0l0 3.640625l3.640625 0l0 1.515625l-3.640625 0l0 3.671875l-1.546875 0zm11.390778 2.15625l2.78125 -13.359375l1.65625 0l-1.0 4.78125q0.78125 -0.71875 1.421875 -1.015625q0.640625 -0.296875 1.328125 -0.296875q1.359375 0 2.265625 1.015625q0.90625 1.0 0.90625 2.9375q0 1.28125 -0.375 2.359375q-0.359375 1.0625 -0.890625 1.78125q-0.53125 0.71875 -1.109375 1.15625q-0.578125 0.4375 -1.1875 0.640625q-0.59375 0.21875 -1.140625 0.21875q-0.96875 0 -1.703125 -0.5q-0.71875 -0.515625 -1.125 -1.546875l-0.390625 1.828125l-1.4375 0zm2.4375 -3.96875l-0.015625 0.3125q0 1.234375 0.59375 1.890625q0.59375 0.640625 1.484375 0.640625q0.859375 0 1.578125 -0.609375q0.734375 -0.609375 1.1875 -1.890625q0.46875 -1.28125 0.46875 -2.375q0 -1.21875 -0.59375 -1.890625q-0.578125 -0.671875 -1.4375 -0.671875q-0.90625 0 -1.65625 0.6875q-0.734375 0.6875 -1.234375 2.125q-0.375 1.0625 -0.375 1.78125zm14.531952 2.21875q-1.734375 1.96875 -3.5625 1.96875q-1.109375 0 -1.796875 -0.640625q-0.6875 -0.640625 -0.6875 -1.578125q0 -0.609375 0.296875 -2.09375l1.171875 -5.578125l1.65625 0l-1.296875 6.1875q-0.171875 0.765625 -0.171875 1.203125q0 0.546875 0.328125 0.859375q0.34375 0.296875 0.984375 0.296875q0.703125 0 1.359375 -0.328125q0.65625 -0.34375 1.125 -0.921875q0.484375 -0.578125 0.796875 -1.359375q0.1875 -0.5 0.453125 -1.765625l0.875 -4.171875l1.65625 0l-2.03125 9.671875l-1.515625 0l0.359375 -1.75zm4.0007324 1.75l1.765625 -8.40625l-1.484375 0l0.265625 -1.265625l1.484375 0l0.28125 -1.375q0.21875 -1.03125 0.4375 -1.484375q0.234375 -0.453125 0.75 -0.75q0.53125 -0.296875 1.4375 -0.296875q0.625 0 1.828125 0.265625l-0.296875 1.4375q-0.84375 -0.21875 -1.40625 -0.21875q-0.484375 0 -0.734375 0.25q-0.25 0.234375 -0.4375 1.125l-0.21875 1.046875l1.84375 0l-0.265625 1.265625l-1.84375 0l-1.75 8.40625l-1.65625 0zm5.1832886 0l1.765625 -8.40625l-1.484375 0l0.265625 -1.265625l1.484375 0l0.28125 -1.375q0.21875 -1.03125 0.4375 -1.484375q0.234375 -0.453125 0.75 -0.75q0.53125 -0.296875 1.4375 -0.296875q0.625 0 1.828125 0.265625l-0.296875 1.4375q-0.84375 -0.21875 -1.40625 -0.21875q-0.484375 0 -0.734375 0.25q-0.25 0.234375 -0.4375 1.125l-0.21875 1.046875l1.84375 0l-0.265625 1.265625l-1.84375 0l-1.75 8.40625l-1.65625 0zm12.058319 -3.28125l1.609375 0.15625q-0.34375 1.1875 -1.59375 2.265625q-1.234375 1.078125 -2.96875 1.078125q-1.0625 0 -1.96875 -0.5q-0.890625 -0.5 -1.359375 -1.4375q-0.46875 -0.953125 -0.46875 -2.15625q0 -1.59375 0.734375 -3.078125q0.734375 -1.484375 1.890625 -2.203125q1.171875 -0.734375 2.53125 -0.734375q1.734375 0 2.765625 1.078125q1.03125 1.0625 1.03125 2.921875q0 0.71875 -0.125 1.46875l-7.125 0q-0.046875 0.28125 -0.046875 0.5q0 1.359375 0.625 2.078125q0.625 0.71875 1.53125 0.71875q0.84375 0 1.65625 -0.546875q0.828125 -0.5625 1.28125 -1.609375zm-4.78125 -2.40625l5.421875 0q0.015625 -0.25 0.015625 -0.359375q0 -1.234375 -0.625 -1.890625q-0.625 -0.671875 -1.59375 -0.671875q-1.0625 0 -1.9375 0.734375q-0.859375 0.71875 -1.28125 2.1875zm8.063202 5.6875l2.015625 -9.671875l1.453125 0l-0.40625 1.96875q0.75 -1.109375 1.453125 -1.640625q0.71875 -0.546875 1.46875 -0.546875q0.5 0 1.21875 0.359375l-0.671875 1.53125q-0.4375 -0.3125 -0.9375 -0.3125q-0.875 0 -1.78125 0.96875q-0.90625 0.953125 -1.4375 3.46875l-0.8125 3.875l-1.5625 0z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m0 165.96588l118.74016 0l0 40.25197l-118.74016 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m0 165.96588l118.74016 0l0 40.25197l-118.74016 0z" fill-rule="evenodd"/><path fill="#000000" d="m23.145836 190.12123l1.625 -0.25q0.125 0.96875 0.75 1.5q0.625 0.515625 1.75 0.515625q1.125 0 1.671875 -0.453125q0.546875 -0.46875 0.546875 -1.09375q0 -0.546875 -0.484375 -0.875q-0.328125 -0.21875 -1.671875 -0.546875q-1.8125 -0.46875 -2.515625 -0.796875q-0.6875 -0.328125 -1.046875 -0.90625q-0.359375 -0.59375 -0.359375 -1.3125q0 -0.640625 0.296875 -1.1875q0.296875 -0.5625 0.8125 -0.921875q0.375 -0.28125 1.03125 -0.46875q0.671875 -0.203125 1.421875 -0.203125q1.140625 0 2.0 0.328125q0.859375 0.328125 1.265625 0.890625q0.421875 0.5625 0.578125 1.5l-1.609375 0.21875q-0.109375 -0.75 -0.640625 -1.171875q-0.515625 -0.421875 -1.46875 -0.421875q-1.140625 0 -1.625 0.375q-0.46875 0.375 -0.46875 0.875q0 0.3125 0.1875 0.578125q0.203125 0.265625 0.640625 0.4375q0.234375 0.09375 1.4375 0.421875q1.75 0.453125 2.4375 0.75q0.6875 0.296875 1.078125 0.859375q0.390625 0.5625 0.390625 1.40625q0 0.828125 -0.484375 1.546875q-0.46875 0.71875 -1.375 1.125q-0.90625 0.390625 -2.046875 0.390625q-1.875 0 -2.875 -0.78125q-0.984375 -0.78125 -1.25 -2.328125zm13.5625 1.421875l0.234375 1.453125q-0.6875 0.140625 -1.234375 0.140625q-0.890625 0 -1.390625 -0.28125q-0.484375 -0.28125 -0.6875 -0.734375q-0.203125 -0.46875 -0.203125 -1.9375l0 -5.578125l-1.203125 0l0 -1.265625l1.203125 0l0 -2.390625l1.625 -0.984375l0 3.375l1.65625 0l0 1.265625l-1.65625 0l0 5.671875q0 0.6875 0.078125 0.890625q0.09375 0.203125 0.28125 0.328125q0.203125 0.109375 0.578125 0.109375q0.265625 0 0.71875 -0.0625zm1.5895538 1.46875l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0zm6.228302 3.703125l0 -13.375l1.484375 0l0 1.25q0.53125 -0.734375 1.1875 -1.09375q0.671875 -0.375 1.625 -0.375q1.234375 0 2.171875 0.640625q0.953125 0.625 1.4375 1.796875q0.484375 1.15625 0.484375 2.546875q0 1.484375 -0.53125 2.671875q-0.53125 1.1875 -1.546875 1.828125q-1.015625 0.625 -2.140625 0.625q-0.8125 0 -1.46875 -0.34375q-0.65625 -0.34375 -1.0625 -0.875l0 4.703125l-1.640625 0zm1.484375 -8.484375q0 1.859375 0.75 2.765625q0.765625 0.890625 1.828125 0.890625q1.09375 0 1.875 -0.921875q0.78125 -0.9375 0.78125 -2.875q0 -1.84375 -0.765625 -2.765625q-0.75 -0.921875 -1.8125 -0.921875q-1.046875 0 -1.859375 0.984375q-0.796875 0.96875 -0.796875 2.84375zm15.203842 3.59375q-0.921875 0.765625 -1.765625 1.09375q-0.828125 0.3125 -1.796875 0.3125q-1.59375 0 -2.453125 -0.78125q-0.859375 -0.78125 -0.859375 -1.984375q0 -0.71875 0.328125 -1.296875q0.328125 -0.59375 0.84375 -0.9375q0.53125 -0.359375 1.1875 -0.546875q0.46875 -0.125 1.453125 -0.25q1.984375 -0.234375 2.921875 -0.5625q0.015625 -0.34375 0.015625 -0.421875q0 -1.0 -0.46875 -1.421875q-0.625 -0.546875 -1.875 -0.546875q-1.15625 0 -1.703125 0.40625q-0.546875 0.40625 -0.8125 1.421875l-1.609375 -0.21875q0.21875 -1.015625 0.71875 -1.640625q0.5 -0.640625 1.453125 -0.984375q0.953125 -0.34375 2.1875 -0.34375q1.25 0 2.015625 0.296875q0.78125 0.28125 1.140625 0.734375q0.375 0.4375 0.515625 1.109375q0.078125 0.421875 0.078125 1.515625l0 2.1875q0 2.28125 0.109375 2.890625q0.109375 0.59375 0.40625 1.15625l-1.703125 0q-0.265625 -0.515625 -0.328125 -1.1875zm-0.140625 -3.671875q-0.890625 0.375 -2.671875 0.625q-1.015625 0.140625 -1.4375 0.328125q-0.421875 0.1875 -0.65625 0.53125q-0.21875 0.34375 -0.21875 0.78125q0 0.65625 0.5 1.09375q0.5 0.4375 1.453125 0.4375q0.9375 0 1.671875 -0.40625q0.75 -0.421875 1.09375 -1.140625q0.265625 -0.5625 0.265625 -1.640625l0 -0.609375zm4.188217 4.859375l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0zm5.572052 -2.890625l1.625 -0.25q0.125 0.96875 0.75 1.5q0.625 0.515625 1.75 0.515625q1.125 0 1.671875 -0.453125q0.546875 -0.46875 0.546875 -1.09375q0 -0.546875 -0.484375 -0.875q-0.328125 -0.21875 -1.671875 -0.546875q-1.8125 -0.46875 -2.515625 -0.796875q-0.6875 -0.328125 -1.046875 -0.90625q-0.359375 -0.59375 -0.359375 -1.3125q0 -0.640625 0.296875 -1.1875q0.296875 -0.5625 0.8125 -0.921875q0.375 -0.28125 1.03125 -0.46875q0.671875 -0.203125 1.421875 -0.203125q1.140625 0 2.0 0.328125q0.859375 0.328125 1.265625 0.890625q0.421875 0.5625 0.578125 1.5l-1.609375 0.21875q-0.109375 -0.75 -0.640625 -1.171875q-0.515625 -0.421875 -1.46875 -0.421875q-1.140625 0 -1.625 0.375q-0.46875 0.375 -0.46875 0.875q0 0.3125 0.1875 0.578125q0.203125 0.265625 0.640625 0.4375q0.234375 0.09375 1.4375 0.421875q1.75 0.453125 2.4375 0.75q0.6875 0.296875 1.078125 0.859375q0.390625 0.5625 0.390625 1.40625q0 0.828125 -0.484375 1.546875q-0.46875 0.71875 -1.375 1.125q-0.90625 0.390625 -2.046875 0.390625q-1.875 0 -2.875 -0.78125q-0.984375 -0.78125 -1.25 -2.328125zm16.609375 -0.21875l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625zm9.125717 5.765625l0 -9.671875l1.46875 0l0 1.46875q0.5625 -1.03125 1.03125 -1.359375q0.484375 -0.328125 1.0625 -0.328125q0.828125 0 1.6875 0.53125l-0.5625 1.515625q-0.609375 -0.359375 -1.203125 -0.359375q-0.546875 0 -0.96875 0.328125q-0.421875 0.328125 -0.609375 0.890625q-0.28125 0.875 -0.28125 1.921875l0 5.0625l-1.625 0z" fill-rule="nonzero"/></g></svg> diff --git a/Documentation/networking/tls-offload-reorder-bad.svg b/Documentation/networking/tls-offload-reorder-bad.svg new file mode 100644 index 000000000000..d107aaf0f71e --- /dev/null +++ b/Documentation/networking/tls-offload-reorder-bad.svg @@ -0,0 +1 @@ +<svg version="1.1" viewBox="0.0 0.0 672.0 68.0" fill="none" stroke="none" stroke-linecap="square" stroke-miterlimit="10" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"><clipPath id="p.0"><path d="m0 0l960.0 0l0 720.0l-960.0 0l0 -720.0z" clip-rule="nonzero"/></clipPath><g clip-path="url(#p.0)"><path fill="#000000" fill-opacity="0.0" d="m0 0l960.0 0l0 720.0l-960.0 0z" fill-rule="evenodd"/><path fill="#b6d7a8" d="m0 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m0 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m15.953125 52.942722l-1.640625 0l0 -10.453125q-0.59375 0.5625 -1.5625 1.140625q-0.953125 0.5625 -1.71875 0.84375l0 -1.59375q1.375 -0.640625 2.40625 -1.5625q1.03125 -0.921875 1.453125 -1.78125l1.0625 0l0 13.40625z" fill-rule="nonzero"/><path fill="#c9daf8" d="m340.69897 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m340.69897 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m355.73022 52.942722l0 -3.203125l-5.796875 0l0 -1.5l6.09375 -8.65625l1.34375 0l0 8.65625l1.796875 0l0 1.5l-1.796875 0l0 3.203125l-1.640625 0zm0 -4.703125l0 -6.015625l-4.1875 6.015625l4.1875 0z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m225.37527 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m225.37527 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m235.15652 49.411472l1.640625 -0.21875q0.28125 1.40625 0.953125 2.015625q0.6875 0.609375 1.65625 0.609375q1.15625 0 1.953125 -0.796875q0.796875 -0.796875 0.796875 -1.984375q0 -1.125 -0.734375 -1.859375q-0.734375 -0.734375 -1.875 -0.734375q-0.46875 0 -1.15625 0.171875l0.1875 -1.4375q0.15625 0.015625 0.265625 0.015625q1.046875 0 1.875 -0.546875q0.84375 -0.546875 0.84375 -1.671875q0 -0.90625 -0.609375 -1.5q-0.609375 -0.59375 -1.578125 -0.59375q-0.953125 0 -1.59375 0.609375q-0.640625 0.59375 -0.8125 1.796875l-1.640625 -0.296875q0.296875 -1.640625 1.359375 -2.546875q1.0625 -0.90625 2.65625 -0.90625q1.09375 0 2.0 0.46875q0.921875 0.46875 1.40625 1.28125q0.5 0.8125 0.5 1.71875q0 0.859375 -0.46875 1.578125q-0.46875 0.703125 -1.375 1.125q1.1875 0.28125 1.84375 1.140625q0.65625 0.859375 0.65625 2.15625q0 1.734375 -1.28125 2.953125q-1.265625 1.21875 -3.21875 1.21875q-1.765625 0 -2.921875 -1.046875q-1.15625 -1.046875 -1.328125 -2.71875z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m572.3295 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m572.3295 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m590.72015 51.364597l0 1.578125l-8.828125 0q-0.015625 -0.59375 0.1875 -1.140625q0.34375 -0.90625 1.078125 -1.78125q0.75 -0.875 2.15625 -2.015625q2.171875 -1.78125 2.9375 -2.828125q0.765625 -1.046875 0.765625 -1.96875q0 -0.984375 -0.703125 -1.640625q-0.6875 -0.671875 -1.8125 -0.671875q-1.1875 0 -1.90625 0.71875q-0.703125 0.703125 -0.703125 1.953125l-1.6875 -0.171875q0.171875 -1.890625 1.296875 -2.875q1.140625 -0.984375 3.03125 -0.984375q1.921875 0 3.046875 1.0625q1.125 1.0625 1.125 2.640625q0 0.796875 -0.328125 1.578125q-0.328125 0.78125 -1.09375 1.640625q-0.75 0.84375 -2.53125 2.34375q-1.46875 1.234375 -1.890625 1.6875q-0.421875 0.4375 -0.6875 0.875l6.546875 0z" fill-rule="nonzero"/><path fill="#e06666" d="m615.56793 24.999102l6.5512085 0l0 42.04725l-6.5512085 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m615.56793 24.999102l6.5512085 0l0 42.04725l-6.5512085 0z" fill-rule="evenodd"/><path fill="#c9daf8" d="m456.51425 24.999102l99.02365 0l0 42.04725l-99.02365 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m456.51425 24.999102l99.02365 0l0 42.04725l-99.02365 0z" fill-rule="evenodd"/><path fill="#000000" d="m466.2955 49.442722l1.71875 -0.140625q0.1875 1.25 0.875 1.890625q0.703125 0.625 1.6875 0.625q1.1875 0 2.0 -0.890625q0.828125 -0.890625 0.828125 -2.359375q0 -1.40625 -0.796875 -2.21875q-0.78125 -0.8125 -2.0625 -0.8125q-0.78125 0 -1.421875 0.359375q-0.640625 0.359375 -1.0 0.9375l-1.546875 -0.203125l1.296875 -6.859375l6.640625 0l0 1.5625l-5.328125 0l-0.71875 3.59375q1.203125 -0.84375 2.515625 -0.84375q1.75 0 2.953125 1.21875q1.203125 1.203125 1.203125 3.109375q0 1.8125 -1.046875 3.140625q-1.296875 1.625 -3.515625 1.625q-1.8125 0 -2.96875 -1.015625q-1.15625 -1.03125 -1.3125 -2.71875z" fill-rule="nonzero"/><path fill="#e06666" d="m391.1985 24.999102l6.551178 0l0 42.04725l-6.551178 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m391.1985 24.999102l6.551178 0l0 42.04725l-6.551178 0z" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m114.43843 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" stroke-dasharray="4.0,3.0" d="m114.43843 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m163.95024 24.999102c0 -12.5 114.47246 -25.007874 228.9449 -25.0c114.47241 0.007874016 228.94489 12.531496 228.94489 25.062992" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m163.95024 24.9991c0 -12.499998 114.47246 -25.007872 228.9449 -24.999998c57.236206 0.003937008 114.47244 3.136811 157.3996 7.835138c21.463562 2.3491635 39.349915 5.0896897 51.8703 8.026144c3.130127 0.7341137 5.9248657 1.4804726 8.356262 2.236023c0.30395508 0.09444237 0.6022339 0.1890316 0.89471436 0.28375435l0.37457275 0.12388611" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m609.98517 21.270555l9.406311 2.1936665l-5.7955933 -7.7266836z" fill-rule="evenodd"/><path fill="#e06666" d="m47.56793 24.999102l6.551182 0l0 42.04725l-6.551182 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m47.56793 24.999102l6.551182 0l0 42.04725l-6.551182 0z" fill-rule="evenodd"/></g></svg> diff --git a/Documentation/networking/tls-offload-reorder-good.svg b/Documentation/networking/tls-offload-reorder-good.svg new file mode 100644 index 000000000000..10e17d91f70c --- /dev/null +++ b/Documentation/networking/tls-offload-reorder-good.svg @@ -0,0 +1 @@ +<svg version="1.1" viewBox="0.0 0.0 672.0 68.0" fill="none" stroke="none" stroke-linecap="square" stroke-miterlimit="10" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"><clipPath id="p.0"><path d="m0 0l960.0 0l0 720.0l-960.0 0l0 -720.0z" clip-rule="nonzero"/></clipPath><g clip-path="url(#p.0)"><path fill="#000000" fill-opacity="0.0" d="m0 0l960.0 0l0 720.0l-960.0 0z" fill-rule="evenodd"/><path fill="#b6d7a8" d="m0 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m0 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m15.953125 52.942722l-1.640625 0l0 -10.453125q-0.59375 0.5625 -1.5625 1.140625q-0.953125 0.5625 -1.71875 0.84375l0 -1.59375q1.375 -0.640625 2.40625 -1.5625q1.03125 -0.921875 1.453125 -1.78125l1.0625 0l0 13.40625z" fill-rule="nonzero"/><path fill="#b6d7a8" d="m340.69897 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m340.69897 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m355.73022 52.942722l0 -3.203125l-5.796875 0l0 -1.5l6.09375 -8.65625l1.34375 0l0 8.65625l1.796875 0l0 1.5l-1.796875 0l0 3.203125l-1.640625 0zm0 -4.703125l0 -6.015625l-4.1875 6.015625l4.1875 0z" fill-rule="nonzero"/><path fill="#cfe2f3" d="m225.37527 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m225.37527 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m235.15652 49.411472l1.640625 -0.21875q0.28125 1.40625 0.953125 2.015625q0.6875 0.609375 1.65625 0.609375q1.15625 0 1.953125 -0.796875q0.796875 -0.796875 0.796875 -1.984375q0 -1.125 -0.734375 -1.859375q-0.734375 -0.734375 -1.875 -0.734375q-0.46875 0 -1.15625 0.171875l0.1875 -1.4375q0.15625 0.015625 0.265625 0.015625q1.046875 0 1.875 -0.546875q0.84375 -0.546875 0.84375 -1.671875q0 -0.90625 -0.609375 -1.5q-0.609375 -0.59375 -1.578125 -0.59375q-0.953125 0 -1.59375 0.609375q-0.640625 0.59375 -0.8125 1.796875l-1.640625 -0.296875q0.296875 -1.640625 1.359375 -2.546875q1.0625 -0.90625 2.65625 -0.90625q1.09375 0 2.0 0.46875q0.921875 0.46875 1.40625 1.28125q0.5 0.8125 0.5 1.71875q0 0.859375 -0.46875 1.578125q-0.46875 0.703125 -1.375 1.125q1.1875 0.28125 1.84375 1.140625q0.65625 0.859375 0.65625 2.15625q0 1.734375 -1.28125 2.953125q-1.265625 1.21875 -3.21875 1.21875q-1.765625 0 -2.921875 -1.046875q-1.15625 -1.046875 -1.328125 -2.71875z" fill-rule="nonzero"/><path fill="#e06666" d="m271.56793 24.999102l6.551178 0l0 42.04725l-6.551178 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m271.56793 24.999102l6.551178 0l0 42.04725l-6.551178 0z" fill-rule="evenodd"/><path fill="#cfe2f3" d="m572.3295 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m572.3295 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" d="m590.72015 51.364597l0 1.578125l-8.828125 0q-0.015625 -0.59375 0.1875 -1.140625q0.34375 -0.90625 1.078125 -1.78125q0.75 -0.875 2.15625 -2.015625q2.171875 -1.78125 2.9375 -2.828125q0.765625 -1.046875 0.765625 -1.96875q0 -0.984375 -0.703125 -1.640625q-0.6875 -0.671875 -1.8125 -0.671875q-1.1875 0 -1.90625 0.71875q-0.703125 0.703125 -0.703125 1.953125l-1.6875 -0.171875q0.171875 -1.890625 1.296875 -2.875q1.140625 -0.984375 3.03125 -0.984375q1.921875 0 3.046875 1.0625q1.125 1.0625 1.125 2.640625q0 0.796875 -0.328125 1.578125q-0.328125 0.78125 -1.09375 1.640625q-0.75 0.84375 -2.53125 2.34375q-1.46875 1.234375 -1.890625 1.6875q-0.421875 0.4375 -0.6875 0.875l6.546875 0z" fill-rule="nonzero"/><path fill="#b6d7a8" d="m456.51425 24.999102l99.02365 0l0 42.04725l-99.02365 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m456.51425 24.999102l99.02365 0l0 42.04725l-99.02365 0z" fill-rule="evenodd"/><path fill="#000000" d="m466.2955 49.442722l1.71875 -0.140625q0.1875 1.25 0.875 1.890625q0.703125 0.625 1.6875 0.625q1.1875 0 2.0 -0.890625q0.828125 -0.890625 0.828125 -2.359375q0 -1.40625 -0.796875 -2.21875q-0.78125 -0.8125 -2.0625 -0.8125q-0.78125 0 -1.421875 0.359375q-0.640625 0.359375 -1.0 0.9375l-1.546875 -0.203125l1.296875 -6.859375l6.640625 0l0 1.5625l-5.328125 0l-0.71875 3.59375q1.203125 -0.84375 2.515625 -0.84375q1.75 0 2.953125 1.21875q1.203125 1.203125 1.203125 3.109375q0 1.8125 -1.046875 3.140625q-1.296875 1.625 -3.515625 1.625q-1.8125 0 -2.96875 -1.015625q-1.15625 -1.03125 -1.3125 -2.71875z" fill-rule="nonzero"/><path fill="#e06666" d="m503.1985 24.999102l6.551178 0l0 42.04725l-6.551178 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m503.1985 24.999102l6.551178 0l0 42.04725l-6.551178 0z" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m114.43843 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" stroke-dasharray="4.0,3.0" d="m114.43843 24.999102l99.02362 0l0 42.04725l-99.02362 0z" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m163.95024 24.999102c0 -12.5 114.47246 -25.007874 228.9449 -25.0c114.47241 0.007874016 228.94489 12.531496 228.94489 25.062992" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m163.95024 24.9991c0 -12.499998 114.47246 -25.007872 228.9449 -24.999998c57.236206 0.003937008 114.47244 3.136811 157.3996 7.835138c21.463562 2.3491635 39.349915 5.0896897 51.8703 8.026144c3.130127 0.7341137 5.9248657 1.4804726 8.356262 2.236023c0.30395508 0.09444237 0.6022339 0.1890316 0.89471436 0.28375435l0.37457275 0.12388611" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m609.98517 21.270555l9.406311 2.1936665l-5.7955933 -7.7266836z" fill-rule="evenodd"/><path fill="#e06666" d="m47.56793 24.999102l6.551182 0l0 42.04725l-6.551182 0z" fill-rule="evenodd"/><path stroke="#000000" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m47.56793 24.999102l6.551182 0l0 42.04725l-6.551182 0z" fill-rule="evenodd"/></g></svg> diff --git a/Documentation/networking/tls-offload.rst b/Documentation/networking/tls-offload.rst new file mode 100644 index 000000000000..cb85af559dff --- /dev/null +++ b/Documentation/networking/tls-offload.rst @@ -0,0 +1,482 @@ +.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +================== +Kernel TLS offload +================== + +Kernel TLS operation +==================== + +Linux kernel provides TLS connection offload infrastructure. Once a TCP +connection is in ``ESTABLISHED`` state user space can enable the TLS Upper +Layer Protocol (ULP) and install the cryptographic connection state. +For details regarding the user-facing interface refer to the TLS +documentation in :ref:`Documentation/networking/tls.rst <kernel_tls>`. + +``ktls`` can operate in three modes: + + * Software crypto mode (``TLS_SW``) - CPU handles the cryptography. + In most basic cases only crypto operations synchronous with the CPU + can be used, but depending on calling context CPU may utilize + asynchronous crypto accelerators. The use of accelerators introduces extra + latency on socket reads (decryption only starts when a read syscall + is made) and additional I/O load on the system. + * Packet-based NIC offload mode (``TLS_HW``) - the NIC handles crypto + on a packet by packet basis, provided the packets arrive in order. + This mode integrates best with the kernel stack and is described in detail + in the remaining part of this document + (``ethtool`` flags ``tls-hw-tx-offload`` and ``tls-hw-rx-offload``). + * Full TCP NIC offload mode (``TLS_HW_RECORD``) - mode of operation where + NIC driver and firmware replace the kernel networking stack + with its own TCP handling, it is not usable in production environments + making use of the Linux networking stack for example any firewalling + abilities or QoS and packet scheduling (``ethtool`` flag ``tls-hw-record``). + +The operation mode is selected automatically based on device configuration, +offload opt-in or opt-out on per-connection basis is not currently supported. + +TX +-- + +At a high level user write requests are turned into a scatter list, the TLS ULP +intercepts them, inserts record framing, performs encryption (in ``TLS_SW`` +mode) and then hands the modified scatter list to the TCP layer. From this +point on the TCP stack proceeds as normal. + +In ``TLS_HW`` mode the encryption is not performed in the TLS ULP. +Instead packets reach a device driver, the driver will mark the packets +for crypto offload based on the socket the packet is attached to, +and send them to the device for encryption and transmission. + +RX +-- + +On the receive side if the device handled decryption and authentication +successfully, the driver will set the decrypted bit in the associated +:c:type:`struct sk_buff <sk_buff>`. The packets reach the TCP stack and +are handled normally. ``ktls`` is informed when data is queued to the socket +and the ``strparser`` mechanism is used to delineate the records. Upon read +request, records are retrieved from the socket and passed to decryption routine. +If device decrypted all the segments of the record the decryption is skipped, +otherwise software path handles decryption. + +.. kernel-figure:: tls-offload-layers.svg + :alt: TLS offload layers + :align: center + :figwidth: 28em + + Layers of Kernel TLS stack + +Device configuration +==================== + +During driver initialization device sets the ``NETIF_F_HW_TLS_RX`` and +``NETIF_F_HW_TLS_TX`` features and installs its +:c:type:`struct tlsdev_ops <tlsdev_ops>` +pointer in the :c:member:`tlsdev_ops` member of the +:c:type:`struct net_device <net_device>`. + +When TLS cryptographic connection state is installed on a ``ktls`` socket +(note that it is done twice, once for RX and once for TX direction, +and the two are completely independent), the kernel checks if the underlying +network device is offload-capable and attempts the offload. In case offload +fails the connection is handled entirely in software using the same mechanism +as if the offload was never tried. + +Offload request is performed via the :c:member:`tls_dev_add` callback of +:c:type:`struct tlsdev_ops <tlsdev_ops>`: + +.. code-block:: c + + int (*tls_dev_add)(struct net_device *netdev, struct sock *sk, + enum tls_offload_ctx_dir direction, + struct tls_crypto_info *crypto_info, + u32 start_offload_tcp_sn); + +``direction`` indicates whether the cryptographic information is for +the received or transmitted packets. Driver uses the ``sk`` parameter +to retrieve the connection 5-tuple and socket family (IPv4 vs IPv6). +Cryptographic information in ``crypto_info`` includes the key, iv, salt +as well as TLS record sequence number. ``start_offload_tcp_sn`` indicates +which TCP sequence number corresponds to the beginning of the record with +sequence number from ``crypto_info``. The driver can add its state +at the end of kernel structures (see :c:member:`driver_state` members +in ``include/net/tls.h``) to avoid additional allocations and pointer +dereferences. + +TX +-- + +After TX state is installed, the stack guarantees that the first segment +of the stream will start exactly at the ``start_offload_tcp_sn`` sequence +number, simplifying TCP sequence number matching. + +TX offload being fully initialized does not imply that all segments passing +through the driver and which belong to the offloaded socket will be after +the expected sequence number and will have kernel record information. +In particular, already encrypted data may have been queued to the socket +before installing the connection state in the kernel. + +RX +-- + +In RX direction local networking stack has little control over the segmentation, +so the initial records' TCP sequence number may be anywhere inside the segment. + +Normal operation +================ + +At the minimum the device maintains the following state for each connection, in +each direction: + + * crypto secrets (key, iv, salt) + * crypto processing state (partial blocks, partial authentication tag, etc.) + * record metadata (sequence number, processing offset and length) + * expected TCP sequence number + +There are no guarantees on record length or record segmentation. In particular +segments may start at any point of a record and contain any number of records. +Assuming segments are received in order, the device should be able to perform +crypto operations and authentication regardless of segmentation. For this +to be possible device has to keep small amount of segment-to-segment state. +This includes at least: + + * partial headers (if a segment carried only a part of the TLS header) + * partial data block + * partial authentication tag (all data had been seen but part of the + authentication tag has to be written or read from the subsequent segment) + +Record reassembly is not necessary for TLS offload. If the packets arrive +in order the device should be able to handle them separately and make +forward progress. + +TX +-- + +The kernel stack performs record framing reserving space for the authentication +tag and populating all other TLS header and tailer fields. + +Both the device and the driver maintain expected TCP sequence numbers +due to the possibility of retransmissions and the lack of software fallback +once the packet reaches the device. +For segments passed in order, the driver marks the packets with +a connection identifier (note that a 5-tuple lookup is insufficient to identify +packets requiring HW offload, see the :ref:`5tuple_problems` section) +and hands them to the device. The device identifies the packet as requiring +TLS handling and confirms the sequence number matches its expectation. +The device performs encryption and authentication of the record data. +It replaces the authentication tag and TCP checksum with correct values. + +RX +-- + +Before a packet is DMAed to the host (but after NIC's embedded switching +and packet transformation functions) the device validates the Layer 4 +checksum and performs a 5-tuple lookup to find any TLS connection the packet +may belong to (technically a 4-tuple +lookup is sufficient - IP addresses and TCP port numbers, as the protocol +is always TCP). If connection is matched device confirms if the TCP sequence +number is the expected one and proceeds to TLS handling (record delineation, +decryption, authentication for each record in the packet). The device leaves +the record framing unmodified, the stack takes care of record decapsulation. +Device indicates successful handling of TLS offload in the per-packet context +(descriptor) passed to the host. + +Upon reception of a TLS offloaded packet, the driver sets +the :c:member:`decrypted` mark in :c:type:`struct sk_buff <sk_buff>` +corresponding to the segment. Networking stack makes sure decrypted +and non-decrypted segments do not get coalesced (e.g. by GRO or socket layer) +and takes care of partial decryption. + +Resync handling +=============== + +In presence of packet drops or network packet reordering, the device may lose +synchronization with the TLS stream, and require a resync with the kernel's +TCP stack. + +Note that resync is only attempted for connections which were successfully +added to the device table and are in TLS_HW mode. For example, +if the table was full when cryptographic state was installed in the kernel, +such connection will never get offloaded. Therefore the resync request +does not carry any cryptographic connection state. + +TX +-- + +Segments transmitted from an offloaded socket can get out of sync +in similar ways to the receive side-retransmissions - local drops +are possible, though network reorders are not. + +Whenever an out of order segment is transmitted the driver provides +the device with enough information to perform cryptographic operations. +This means most likely that the part of the record preceding the current +segment has to be passed to the device as part of the packet context, +together with its TCP sequence number and TLS record number. The device +can then initialize its crypto state, process and discard the preceding +data (to be able to insert the authentication tag) and move onto handling +the actual packet. + +In this mode depending on the implementation the driver can either ask +for a continuation with the crypto state and the new sequence number +(next expected segment is the one after the out of order one), or continue +with the previous stream state - assuming that the out of order segment +was just a retransmission. The former is simpler, and does not require +retransmission detection therefore it is the recommended method until +such time it is proven inefficient. + +RX +-- + +A small amount of RX reorder events may not require a full resynchronization. +In particular the device should not lose synchronization +when record boundary can be recovered: + +.. kernel-figure:: tls-offload-reorder-good.svg + :alt: reorder of non-header segment + :align: center + + Reorder of non-header segment + +Green segments are successfully decrypted, blue ones are passed +as received on wire, red stripes mark start of new records. + +In above case segment 1 is received and decrypted successfully. +Segment 2 was dropped so 3 arrives out of order. The device knows +the next record starts inside 3, based on record length in segment 1. +Segment 3 is passed untouched, because due to lack of data from segment 2 +the remainder of the previous record inside segment 3 cannot be handled. +The device can, however, collect the authentication algorithm's state +and partial block from the new record in segment 3 and when 4 and 5 +arrive continue decryption. Finally when 2 arrives it's completely outside +of expected window of the device so it's passed as is without special +handling. ``ktls`` software fallback handles the decryption of record +spanning segments 1, 2 and 3. The device did not get out of sync, +even though two segments did not get decrypted. + +Kernel synchronization may be necessary if the lost segment contained +a record header and arrived after the next record header has already passed: + +.. kernel-figure:: tls-offload-reorder-bad.svg + :alt: reorder of header segment + :align: center + + Reorder of segment with a TLS header + +In this example segment 2 gets dropped, and it contains a record header. +Device can only detect that segment 4 also contains a TLS header +if it knows the length of the previous record from segment 2. In this case +the device will lose synchronization with the stream. + +When the device gets out of sync and the stream reaches TCP sequence +numbers more than a max size record past the expected TCP sequence number, +the device starts scanning for a known header pattern. For example +for TLS 1.2 and TLS 1.3 subsequent bytes of value ``0x03 0x03`` occur +in the SSL/TLS version field of the header. Once pattern is matched +the device continues attempting parsing headers at expected locations +(based on the length fields at guessed locations). +Whenever the expected location does not contain a valid header the scan +is restarted. + +When the header is matched the device sends a confirmation request +to the kernel, asking if the guessed location is correct (if a TLS record +really starts there), and which record sequence number the given header had. +The kernel confirms the guessed location was correct and tells the device +the record sequence number. Meanwhile, the device had been parsing +and counting all records since the just-confirmed one, it adds the number +of records it had seen to the record number provided by the kernel. +At this point the device is in sync and can resume decryption at next +segment boundary. + +In a pathological case the device may latch onto a sequence of matching +headers and never hear back from the kernel (there is no negative +confirmation from the kernel). The implementation may choose to periodically +restart scan. Given how unlikely falsely-matching stream is, however, +periodic restart is not deemed necessary. + +Special care has to be taken if the confirmation request is passed +asynchronously to the packet stream and record may get processed +by the kernel before the confirmation request. + +Error handling +============== + +TX +-- + +Packets may be redirected or rerouted by the stack to a different +device than the selected TLS offload device. The stack will handle +such condition using the :c:func:`sk_validate_xmit_skb` helper +(TLS offload code installs :c:func:`tls_validate_xmit_skb` at this hook). +Offload maintains information about all records until the data is +fully acknowledged, so if skbs reach the wrong device they can be handled +by software fallback. + +Any device TLS offload handling error on the transmission side must result +in the packet being dropped. For example if a packet got out of order +due to a bug in the stack or the device, reached the device and can't +be encrypted such packet must be dropped. + +RX +-- + +If the device encounters any problems with TLS offload on the receive +side it should pass the packet to the host's networking stack as it was +received on the wire. + +For example authentication failure for any record in the segment should +result in passing the unmodified packet to the software fallback. This means +packets should not be modified "in place". Splitting segments to handle partial +decryption is not advised. In other words either all records in the packet +had been handled successfully and authenticated or the packet has to be passed +to the host's stack as it was on the wire (recovering original packet in the +driver if device provides precise error is sufficient). + +The Linux networking stack does not provide a way of reporting per-packet +decryption and authentication errors, packets with errors must simply not +have the :c:member:`decrypted` mark set. + +A packet should also not be handled by the TLS offload if it contains +incorrect checksums. + +Performance metrics +=================== + +TLS offload can be characterized by the following basic metrics: + + * max connection count + * connection installation rate + * connection installation latency + * total cryptographic performance + +Note that each TCP connection requires a TLS session in both directions, +the performance may be reported treating each direction separately. + +Max connection count +-------------------- + +The number of connections device can support can be exposed via +``devlink resource`` API. + +Total cryptographic performance +------------------------------- + +Offload performance may depend on segment and record size. + +Overload of the cryptographic subsystem of the device should not have +significant performance impact on non-offloaded streams. + +Statistics +========== + +Following minimum set of TLS-related statistics should be reported +by the driver: + + * ``rx_tls_decrypted`` - number of successfully decrypted TLS segments + * ``tx_tls_encrypted`` - number of in-order TLS segments passed to device + for encryption + * ``tx_tls_ooo`` - number of TX packets which were part of a TLS stream + but did not arrive in the expected order + * ``tx_tls_drop_no_sync_data`` - number of TX packets dropped because + they arrived out of order and associated record could not be found + (see also :ref:`pre_tls_data`) + +Notable corner cases, exceptions and additional requirements +============================================================ + +.. _5tuple_problems: + +5-tuple matching limitations +---------------------------- + +The device can only recognize received packets based on the 5-tuple +of the socket. Current ``ktls`` implementation will not offload sockets +routed through software interfaces such as those used for tunneling +or virtual networking. However, many packet transformations performed +by the networking stack (most notably any BPF logic) do not require +any intermediate software device, therefore a 5-tuple match may +consistently miss at the device level. In such cases the device +should still be able to perform TX offload (encryption) and should +fallback cleanly to software decryption (RX). + +Out of order +------------ + +Introducing extra processing in NICs should not cause packets to be +transmitted or received out of order, for example pure ACK packets +should not be reordered with respect to data segments. + +Ingress reorder +--------------- + +A device is permitted to perform packet reordering for consecutive +TCP segments (i.e. placing packets in the correct order) but any form +of additional buffering is disallowed. + +Coexistence with standard networking offload features +----------------------------------------------------- + +Offloaded ``ktls`` sockets should support standard TCP stack features +transparently. Enabling device TLS offload should not cause any difference +in packets as seen on the wire. + +Transport layer transparency +---------------------------- + +The device should not modify any packet headers for the purpose +of the simplifying TLS offload. + +The device should not depend on any packet headers beyond what is strictly +necessary for TLS offload. + +Segment drops +------------- + +Dropping packets is acceptable only in the event of catastrophic +system errors and should never be used as an error handling mechanism +in cases arising from normal operation. In other words, reliance +on TCP retransmissions to handle corner cases is not acceptable. + +TLS device features +------------------- + +Drivers should ignore the changes to TLS the device feature flags. +These flags will be acted upon accordingly by the core ``ktls`` code. +TLS device feature flags only control adding of new TLS connection +offloads, old connections will remain active after flags are cleared. + +Known bugs +========== + +skb_orphan() leaks clear text +----------------------------- + +Currently drivers depend on the :c:member:`sk` member of +:c:type:`struct sk_buff <sk_buff>` to identify segments requiring +encryption. Any operation which removes or does not preserve the socket +association such as :c:func:`skb_orphan` or :c:func:`skb_clone` +will cause the driver to miss the packets and lead to clear text leaks. + +Redirects leak clear text +------------------------- + +In the RX direction, if segment has already been decrypted by the device +and it gets redirected or mirrored - clear text will be transmitted out. + +.. _pre_tls_data: + +Transmission of pre-TLS data +---------------------------- + +User can enqueue some already encrypted and framed records before enabling +``ktls`` on the socket. Those records have to get sent as they are. This is +perfectly easy to handle in the software case - such data will be waiting +in the TCP layer, TLS ULP won't see it. In the offloaded case when pre-queued +segment reaches transmission point it appears to be out of order (before the +expected TCP sequence number) and the stack does not have a record information +associated. + +All segments without record information cannot, however, be assumed to be +pre-queued data, because a race condition exists between TCP stack queuing +a retransmission, the driver seeing the retransmission and TCP ACK arriving +for the retransmitted data. diff --git a/Documentation/networking/tls.txt b/Documentation/networking/tls.rst index 58b5ef75f1b7..5bcbf75e2025 100644 --- a/Documentation/networking/tls.txt +++ b/Documentation/networking/tls.rst @@ -1,3 +1,9 @@ +.. _kernel_tls: + +========== +Kernel TLS +========== + Overview ======== @@ -12,6 +18,8 @@ Creating a TLS connection First create a new TCP socket and set the TLS ULP. +.. code-block:: c + sock = socket(AF_INET, SOCK_STREAM, 0); setsockopt(sock, SOL_TCP, TCP_ULP, "tls", sizeof("tls")); @@ -21,6 +29,8 @@ handshake is complete, we have all the parameters required to move the data-path to the kernel. There is a separate socket option for moving the transmit and the receive into the kernel. +.. code-block:: c + /* From linux/tls.h */ struct tls_crypto_info { unsigned short version; @@ -58,6 +68,8 @@ After setting the TLS_TX socket option all application data sent over this socket is encrypted using TLS and the parameters provided in the socket option. For example, we can send an encrypted hello world record as follows: +.. code-block:: c + const char *msg = "hello world\n"; send(sock, msg, strlen(msg)); @@ -67,6 +79,8 @@ to the encrypted kernel send buffer if possible. The sendfile system call will send the file's data over TLS records of maximum length (2^14). +.. code-block:: c + file = open(filename, O_RDONLY); fstat(file, &stat); sendfile(sock, file, &offset, stat.st_size); @@ -89,6 +103,8 @@ After setting the TLS_RX socket option, all recv family socket calls are decrypted using TLS parameters provided. A full TLS record must be received before decryption can happen. +.. code-block:: c + char buffer[16384]; recv(sock, buffer, 16384); @@ -97,12 +113,12 @@ large enough, and no additional allocations occur. If the userspace buffer is too small, data is decrypted in the kernel and copied to userspace. -EINVAL is returned if the TLS version in the received message does not +``EINVAL`` is returned if the TLS version in the received message does not match the version passed in setsockopt. -EMSGSIZE is returned if the received message is too big. +``EMSGSIZE`` is returned if the received message is too big. -EBADMSG is returned if decryption failed for any other reason. +``EBADMSG`` is returned if decryption failed for any other reason. Send TLS control messages ------------------------- @@ -113,9 +129,11 @@ These messages can be sent over the socket by providing the TLS record type via a CMSG. For example the following function sends @data of @length bytes using a record of type @record_type. -/* send TLS control message using record_type */ +.. code-block:: c + + /* send TLS control message using record_type */ static int klts_send_ctrl_message(int sock, unsigned char record_type, - void *data, size_t length) + void *data, size_t length) { struct msghdr msg = {0}; int cmsg_len = sizeof(record_type); @@ -151,6 +169,8 @@ type passed via cmsg. If no cmsg buffer is provided, an error is returned if a control message is received. Data messages may be received without a cmsg buffer set. +.. code-block:: c + char buffer[16384]; char cmsg[CMSG_SPACE(sizeof(unsigned char))]; struct msghdr msg = {0}; @@ -186,12 +206,10 @@ Integrating in to userspace TLS library At a high level, the kernel TLS ULP is a replacement for the record layer of a userspace TLS library. -A patchset to OpenSSL to use ktls as the record layer is here: - -https://github.com/Mellanox/openssl/commits/tls_rx2 - -An example of calling send directly after a handshake using -gnutls. Since it doesn't implement a full record layer, control -messages are not supported: +A patchset to OpenSSL to use ktls as the record layer is +`here <https://github.com/Mellanox/openssl/commits/tls_rx2>`_. -https://github.com/ktls/af_ktls-tool/commits/RX +`An example <https://github.com/ktls/af_ktls-tool/commits/RX>`_ +of calling send directly after a handshake using gnutls. +Since it doesn't implement a full record layer, control +messages are not supported. diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 18735dc460a0..0a18075c485e 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -31,7 +31,7 @@ you probably needn't concern yourself with isdn4k-utils. ====================== =============== ======================================== GNU C 4.6 gcc --version GNU make 3.81 make --version -binutils 2.20 ld -v +binutils 2.21 ld -v flex 2.5.35 flex --version bison 2.0 bison --version util-linux 2.10o fdformat --version @@ -77,9 +77,7 @@ You will need GNU make 3.81 or later to build the kernel. Binutils -------- -The build system has, as of 4.13, switched to using thin archives (`ar T`) -rather than incremental linking (`ld -r`) for built-in.a intermediate steps. -This requires binutils 2.20 or newer. +Binutils 2.21 or newer is needed to build the kernel. pkg-config ---------- diff --git a/Documentation/s390/3270.txt b/Documentation/s390/3270.rst index 7c715de99774..e09e77954238 100644 --- a/Documentation/s390/3270.txt +++ b/Documentation/s390/3270.rst @@ -1,13 +1,17 @@ +=============================== IBM 3270 Display System support +=============================== This file describes the driver that supports local channel attachment of IBM 3270 devices. It consists of three sections: + * Introduction * Installation * Operation -INTRODUCTION. +Introduction +============ This paper describes installing and operating 3270 devices under Linux/390. A 3270 device is a block-mode rows-and-columns terminal of @@ -17,12 +21,12 @@ twenty and thirty years ago. You may have 3270s in-house and not know it. If you're using the VM-ESA operating system, define a 3270 to your virtual machine by using the command "DEF GRAF <hex-address>" This paper presumes you will be -defining four 3270s with the CP/CMS commands +defining four 3270s with the CP/CMS commands: - DEF GRAF 620 - DEF GRAF 621 - DEF GRAF 622 - DEF GRAF 623 + - DEF GRAF 620 + - DEF GRAF 621 + - DEF GRAF 622 + - DEF GRAF 623 Your network connection from VM-ESA allows you to use x3270, tn3270, or another 3270 emulator, started from an xterm window on your PC or @@ -34,7 +38,8 @@ This paper covers installation of the driver and operation of a dialed-in x3270. -INSTALLATION. +Installation +============ You install the driver by installing a patch, doing a kernel build, and running the configuration script (config3270.sh, in this directory). @@ -59,13 +64,15 @@ Use #CP TERM CONMODE 3270 to change it to 3270. If you generate only at boot time to a 3270 if it is a 3215. In brief, these are the steps: + 1. Install the tub3270 patch - 2. (If a module) add a line to a file in /etc/modprobe.d/*.conf + 2. (If a module) add a line to a file in `/etc/modprobe.d/*.conf` 3. (If VM) define devices with DEF GRAF 4. Reboot 5. Configure To test that everything works, assuming VM and x3270, + 1. Bring up an x3270 window. 2. Use the DIAL command in that window. 3. You should immediately see a Linux login screen. @@ -74,7 +81,8 @@ Here are the installation steps in detail: 1. The 3270 driver is a part of the official Linux kernel source. Build a tree with the kernel source and any necessary - patches. Then do + patches. Then do:: + make oldconfig (If you wish to disable 3215 console support, edit .config; change CONFIG_TN3215's value to "n"; @@ -84,20 +92,22 @@ Here are the installation steps in detail: make modules_install 2. (Perform this step only if you have configured tub3270 as a - module.) Add a line to a file /etc/modprobe.d/*.conf to automatically + module.) Add a line to a file `/etc/modprobe.d/*.conf` to automatically load the driver when it's needed. With this line added, you will see login prompts appear on your 3270s as soon as boot is complete (or with emulated 3270s, as soon as you dial into your vm guest using the command "DIAL <vmguestname>"). Since the line-mode major number is - 227, the line to add should be: + 227, the line to add should be:: + alias char-major-227 tub3270 3. Define graphic devices to your vm guest machine, if you haven't already. Define them before you reboot (reipl): - DEFINE GRAF 620 - DEFINE GRAF 621 - DEFINE GRAF 622 - DEFINE GRAF 623 + + - DEFINE GRAF 620 + - DEFINE GRAF 621 + - DEFINE GRAF 622 + - DEFINE GRAF 623 4. Reboot. The reboot process scans hardware devices, including 3270s, and this enables the tub3270 driver once loaded to respond @@ -107,21 +117,23 @@ Here are the installation steps in detail: 5. Run the 3270 configuration script config3270. It is distributed in this same directory, Documentation/s390, as - config3270.sh. Inspect the output script it produces, + config3270.sh. Inspect the output script it produces, /tmp/mkdev3270, and then run that script. This will create the necessary character special device files and make the necessary changes to /etc/inittab. Then notify /sbin/init that /etc/inittab has changed, by issuing - the telinit command with the q operand: + the telinit command with the q operand:: + cd Documentation/s390 sh config3270.sh sh /tmp/mkdev3270 telinit q - This should be sufficient for your first time. If your 3270 + This should be sufficient for your first time. If your 3270 configuration has changed and you're reusing config3270, you - should follow these steps: + should follow these steps:: + Change 3270 configuration Reboot Run config3270 and /tmp/mkdev3270 @@ -132,8 +144,10 @@ Here are the testing steps in detail: 1. Bring up an x3270 window, or use an actual hardware 3278 or 3279, or use the 3270 emulator of your choice. You would be running the emulator on your PC or workstation. You would use - the command, for example, + the command, for example:: + x3270 vm-esa-domain-name & + if you wanted a 3278 Model 4 with 43 rows of 80 columns, the default model number. The driver does not take advantage of extended attributes. @@ -144,7 +158,8 @@ Here are the testing steps in detail: 2. Use the DIAL command instead of the LOGIN command to connect to one of the virtual 3270s you defined with the DEF GRAF - commands: + commands:: + dial my-vm-guest-name 3. You should immediately see a login prompt from your @@ -171,14 +186,17 @@ Here are the testing steps in detail: Wrong major number? Wrong minor number? There's your problem! - D. Do you get the message + D. Do you get the message:: + "HCPDIA047E my-vm-guest-name 0620 does not exist"? + If so, you must issue the command "DEF GRAF 620" from your VM 3215 console and then reboot the system. OPERATION. +========== The driver defines three areas on the 3270 screen: the log area, the input area, and the status area. @@ -203,8 +221,10 @@ which indicates no scrolling will occur. (If you hit ENTER with "Linux Running" and nothing typed, the application receives a newline.) You may change the scrolling timeout value. For example, the following -command line: +command line:: + echo scrolltime=60 > /proc/tty/driver/tty3270 + changes the scrolling timeout value to 60 sec. Set scrolltime to 0 if you wish to prevent scrolling entirely. @@ -228,7 +248,8 @@ cause an EOF also by typing "^D" and hitting ENTER. No PF key is preassigned to cause a job suspension, but you may cause a job suspension by typing "^Z" and hitting ENTER. You may wish to assign this function to a PF key. To make PF7 cause job suspension, -execute the command: +execute the command:: + echo pf7=^z > /proc/tty/driver/tty3270 If the input you type does not end with the two characters "^n", the @@ -243,8 +264,10 @@ command is entered into the stack only when the input area is not made invisible (such as for password entry) and it is not identical to the current top entry. PF10 rotates backward through the command stack; PF11 rotates forward. You may assign the backward function to any PF -key (or PA key, for that matter), say, PA3, with the command: +key (or PA key, for that matter), say, PA3, with the command:: + echo -e pa3=\\033k > /proc/tty/driver/tty3270 + This assigns the string ESC-k to PA3. Similarly, the string ESC-j performs the forward function. (Rationale: In bash with vi-mode line editing, ESC-k and ESC-j retrieve backward and forward history. @@ -252,15 +275,19 @@ Suggestions welcome.) Is a stack size of twenty commands not to your liking? Change it on the fly. To change to saving the last 100 commands, execute the -command: +command:: + echo recallsize=100 > /proc/tty/driver/tty3270 Have a command you issue frequently? Assign it to a PF or PA key! Use -the command - echo pf24="mkdir foobar; cd foobar" > /proc/tty/driver/tty3270 +the command:: + + echo pf24="mkdir foobar; cd foobar" > /proc/tty/driver/tty3270 + to execute the commands mkdir foobar and cd foobar immediately when you hit PF24. Want to see the command line first, before you execute it? -Use the -n option of the echo command: +Use the -n option of the echo command:: + echo -n pf24="mkdir foo; cd foo" > /proc/tty/driver/tty3270 diff --git a/Documentation/s390/Debugging390.txt b/Documentation/s390/Debugging390.txt deleted file mode 100644 index 5ae7f868a007..000000000000 --- a/Documentation/s390/Debugging390.txt +++ /dev/null @@ -1,2142 +0,0 @@ - - Debugging on Linux for s/390 & z/Architecture - by - Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com) - Copyright (C) 2000-2001 IBM Deutschland Entwicklung GmbH, IBM Corporation - Best viewed with fixed width fonts - -Overview of Document: -===================== -This document is intended to give a good overview of how to debug Linux for -s/390 and z/Architecture. It is not intended as a complete reference and not a -tutorial on the fundamentals of C & assembly. It doesn't go into -390 IO in any detail. It is intended to complement the documents in the -reference section below & any other worthwhile references you get. - -It is intended like the Enterprise Systems Architecture/390 Reference Summary -to be printed out & used as a quick cheat sheet self help style reference when -problems occur. - -Contents -======== -Register Set -Address Spaces on Intel Linux -Address Spaces on Linux for s/390 & z/Architecture -The Linux for s/390 & z/Architecture Kernel Task Structure -Register Usage & Stackframes on Linux for s/390 & z/Architecture -A sample program with comments -Compiling programs for debugging on Linux for s/390 & z/Architecture -Debugging under VM -s/390 & z/Architecture IO Overview -Debugging IO on s/390 & z/Architecture under VM -GDB on s/390 & z/Architecture -Stack chaining in gdb by hand -Examining core dumps -ldd -Debugging modules -The proc file system -SysRq -References -Special Thanks - -Register Set -============ -The current architectures have the following registers. - -16 General propose registers, 32 bit on s/390 and 64 bit on z/Architecture, -r0-r15 (or gpr0-gpr15), used for arithmetic and addressing. - -16 Control registers, 32 bit on s/390 and 64 bit on z/Architecture, cr0-cr15, -kernel usage only, used for memory management, interrupt control, debugging -control etc. - -16 Access registers (ar0-ar15), 32 bit on both s/390 and z/Architecture, -normally not used by normal programs but potentially could be used as -temporary storage. These registers have a 1:1 association with general -purpose registers and are designed to be used in the so-called access -register mode to select different address spaces. -Access register 0 (and access register 1 on z/Architecture, which needs a -64 bit pointer) is currently used by the pthread library as a pointer to -the current running threads private area. - -16 64 bit floating point registers (fp0-fp15 ) IEEE & HFP floating -point format compliant on G5 upwards & a Floating point control reg (FPC) -4 64 bit registers (fp0,fp2,fp4 & fp6) HFP only on older machines. -Note: -Linux (currently) always uses IEEE & emulates G5 IEEE format on older machines, -( provided the kernel is configured for this ). - - -The PSW is the most important register on the machine it -is 64 bit on s/390 & 128 bit on z/Architecture & serves the roles of -a program counter (pc), condition code register,memory space designator. -In IBM standard notation I am counting bit 0 as the MSB. -It has several advantages over a normal program counter -in that you can change address translation & program counter -in a single instruction. To change address translation, -e.g. switching address translation off requires that you -have a logical=physical mapping for the address you are -currently running at. - - Bit Value -s/390 z/Architecture -0 0 Reserved ( must be 0 ) otherwise specification exception occurs. - -1 1 Program Event Recording 1 PER enabled, - PER is used to facilitate debugging e.g. single stepping. - -2-4 2-4 Reserved ( must be 0 ). - -5 5 Dynamic address translation 1=DAT on. - -6 6 Input/Output interrupt Mask - -7 7 External interrupt Mask used primarily for interprocessor - signalling and clock interrupts. - -8-11 8-11 PSW Key used for complex memory protection mechanism - (not used under linux) - -12 12 1 on s/390 0 on z/Architecture - -13 13 Machine Check Mask 1=enable machine check interrupts - -14 14 Wait State. Set this to 1 to stop the processor except for - interrupts and give time to other LPARS. Used in CPU idle in - the kernel to increase overall usage of processor resources. - -15 15 Problem state ( if set to 1 certain instructions are disabled ) - all linux user programs run with this bit 1 - ( useful info for debugging under VM ). - -16-17 16-17 Address Space Control - - 00 Primary Space Mode: - The register CR1 contains the primary address-space control ele- - ment (PASCE), which points to the primary space region/segment - table origin. - - 01 Access register mode - - 10 Secondary Space Mode: - The register CR7 contains the secondary address-space control - element (SASCE), which points to the secondary space region or - segment table origin. - - 11 Home Space Mode: - The register CR13 contains the home space address-space control - element (HASCE), which points to the home space region/segment - table origin. - - See "Address Spaces on Linux for s/390 & z/Architecture" below - for more information about address space usage in Linux. - -18-19 18-19 Condition codes (CC) - -20 20 Fixed point overflow mask if 1=FPU exceptions for this event - occur ( normally 0 ) - -21 21 Decimal overflow mask if 1=FPU exceptions for this event occur - ( normally 0 ) - -22 22 Exponent underflow mask if 1=FPU exceptions for this event occur - ( normally 0 ) - -23 23 Significance Mask if 1=FPU exceptions for this event occur - ( normally 0 ) - -24-31 24-30 Reserved Must be 0. - - 31 Extended Addressing Mode - 32 Basic Addressing Mode - Used to set addressing mode - PSW 31 PSW 32 - 0 0 24 bit - 0 1 31 bit - 1 1 64 bit - -32 1=31 bit addressing mode 0=24 bit addressing mode (for backward - compatibility), linux always runs with this bit set to 1 - -33-64 Instruction address. - 33-63 Reserved must be 0 - 64-127 Address - In 24 bits mode bits 64-103=0 bits 104-127 Address - In 31 bits mode bits 64-96=0 bits 97-127 Address - Note: unlike 31 bit mode on s/390 bit 96 must be zero - when loading the address with LPSWE otherwise a - specification exception occurs, LPSW is fully backward - compatible. - - -Prefix Page(s) --------------- -This per cpu memory area is too intimately tied to the processor not to mention. -It exists between the real addresses 0-4096 on s/390 and between 0-8192 on -z/Architecture and is exchanged with one page on s/390 or two pages on -z/Architecture in absolute storage by the set prefix instruction during Linux -startup. -This page is mapped to a different prefix for each processor in an SMP -configuration (assuming the OS designer is sane of course). -Bytes 0-512 (200 hex) on s/390 and 0-512, 4096-4544, 4604-5119 currently on -z/Architecture are used by the processor itself for holding such information -as exception indications and entry points for exceptions. -Bytes after 0xc00 hex are used by linux for per processor globals on s/390 and -z/Architecture (there is a gap on z/Architecture currently between 0xc00 and -0x1000, too, which is used by Linux). -The closest thing to this on traditional architectures is the interrupt -vector table. This is a good thing & does simplify some of the kernel coding -however it means that we now cannot catch stray NULL pointers in the -kernel without hard coded checks. - - - -Address Spaces on Intel Linux -============================= - -The traditional Intel Linux is approximately mapped as follows forgive -the ascii art. -0xFFFFFFFF 4GB Himem ***************** - * * - * Kernel Space * - * * - ***************** **************** -User Space Himem * User Stack * * * -(typically 0xC0000000 3GB ) ***************** * * - * Shared Libs * * Next Process * - ***************** * to * - * * <== * Run * <== - * User Program * * * - * Data BSS * * * - * Text * * * - * Sections * * * -0x00000000 ***************** **************** - -Now it is easy to see that on Intel it is quite easy to recognise a kernel -address as being one greater than user space himem (in this case 0xC0000000), -and addresses of less than this are the ones in the current running program on -this processor (if an smp box). -If using the virtual machine ( VM ) as a debugger it is quite difficult to -know which user process is running as the address space you are looking at -could be from any process in the run queue. - -The limitation of Intels addressing technique is that the linux -kernel uses a very simple real address to virtual addressing technique -of Real Address=Virtual Address-User Space Himem. -This means that on Intel the kernel linux can typically only address -Himem=0xFFFFFFFF-0xC0000000=1GB & this is all the RAM these machines -can typically use. -They can lower User Himem to 2GB or lower & thus be -able to use 2GB of RAM however this shrinks the maximum size -of User Space from 3GB to 2GB they have a no win limit of 4GB unless -they go to 64 Bit. - - -On 390 our limitations & strengths make us slightly different. -For backward compatibility we are only allowed use 31 bits (2GB) -of our 32 bit addresses, however, we use entirely separate address -spaces for the user & kernel. - -This means we can support 2GB of non Extended RAM on s/390, & more -with the Extended memory management swap device & -currently 4TB of physical memory currently on z/Architecture. - - -Address Spaces on Linux for s/390 & z/Architecture -================================================== - -Our addressing scheme is basically as follows: - - Primary Space Home Space -Himem 0x7fffffff 2GB on s/390 ***************** **************** -currently 0x3ffffffffff (2^42)-1 * User Stack * * * -on z/Architecture. ***************** * * - * Shared Libs * * * - ***************** * * - * * * Kernel * - * User Program * * * - * Data BSS * * * - * Text * * * - * Sections * * * -0x00000000 ***************** **************** - -This also means that we need to look at the PSW problem state bit and the -addressing mode to decide whether we are looking at user or kernel space. - -User space runs in primary address mode (or access register mode within -the vdso code). - -The kernel usually also runs in home space mode, however when accessing -user space the kernel switches to primary or secondary address mode if -the mvcos instruction is not available or if a compare-and-swap (futex) -instruction on a user space address is performed. - -When also looking at the ASCE control registers, this means: - -User space: -- runs in primary or access register mode -- cr1 contains the user asce -- cr7 contains the user asce -- cr13 contains the kernel asce - -Kernel space: -- runs in home space mode -- cr1 contains the user or kernel asce - -> the kernel asce is loaded when a uaccess requires primary or - secondary address mode -- cr7 contains the user or kernel asce, (changed with set_fs()) -- cr13 contains the kernel asce - -In case of uaccess the kernel changes to: -- primary space mode in case of a uaccess (copy_to_user) and uses - e.g. the mvcp instruction to access user space. However the kernel - will stay in home space mode if the mvcos instruction is available -- secondary space mode in case of futex atomic operations, so that the - instructions come from primary address space and data from secondary - space - -In case of KVM, the kernel runs in home space mode, but cr1 gets switched -to contain the gmap asce before the SIE instruction gets executed. When -the SIE instruction is finished, cr1 will be switched back to contain the -user asce. - - -Virtual Addresses on s/390 & z/Architecture -=========================================== - -A virtual address on s/390 is made up of 3 parts -The SX (segment index, roughly corresponding to the PGD & PMD in Linux -terminology) being bits 1-11. -The PX (page index, corresponding to the page table entry (pte) in Linux -terminology) being bits 12-19. -The remaining bits BX (the byte index are the offset in the page ) -i.e. bits 20 to 31. - -On z/Architecture in linux we currently make up an address from 4 parts. -The region index bits (RX) 0-32 we currently use bits 22-32 -The segment index (SX) being bits 33-43 -The page index (PX) being bits 44-51 -The byte index (BX) being bits 52-63 - -Notes: -1) s/390 has no PMD so the PMD is really the PGD also. -A lot of this stuff is defined in pgtable.h. - -2) Also seeing as s/390's page indexes are only 1k in size -(bits 12-19 x 4 bytes per pte ) we use 1 ( page 4k ) -to make the best use of memory by updating 4 segment indices -entries each time we mess with a PMD & use offsets -0,1024,2048 & 3072 in this page as for our segment indexes. -On z/Architecture our page indexes are now 2k in size -( bits 12-19 x 8 bytes per pte ) we do a similar trick -but only mess with 2 segment indices each time we mess with -a PMD. - -3) As z/Architecture supports up to a massive 5-level page table lookup we -can only use 3 currently on Linux ( as this is all the generic kernel -currently supports ) however this may change in future -this allows us to access ( according to my sums ) -4TB of virtual storage per process i.e. -4096*512(PTES)*1024(PMDS)*2048(PGD) = 4398046511104 bytes, -enough for another 2 or 3 of years I think :-). -to do this we use a region-third-table designation type in -our address space control registers. - - -The Linux for s/390 & z/Architecture Kernel Task Structure -========================================================== -Each process/thread under Linux for S390 has its own kernel task_struct -defined in linux/include/linux/sched.h -The S390 on initialisation & resuming of a process on a cpu sets -the __LC_KERNEL_STACK variable in the spare prefix area for this cpu -(which we use for per-processor globals). - -The kernel stack pointer is intimately tied with the task structure for -each processor as follows. - - s/390 - ************************ - * 1 page kernel stack * - * ( 4K ) * - ************************ - * 1 page task_struct * - * ( 4K ) * -8K aligned ************************ - - z/Architecture - ************************ - * 2 page kernel stack * - * ( 8K ) * - ************************ - * 2 page task_struct * - * ( 8K ) * -16K aligned ************************ - -What this means is that we don't need to dedicate any register or global -variable to point to the current running process & can retrieve it with the -following very simple construct for s/390 & one very similar for z/Architecture. - -static inline struct task_struct * get_current(void) -{ - struct task_struct *current; - __asm__("lhi %0,-8192\n\t" - "nr %0,15" - : "=r" (current) ); - return current; -} - -i.e. just anding the current kernel stack pointer with the mask -8192. -Thankfully because Linux doesn't have support for nested IO interrupts -& our devices have large buffers can survive interrupts being shut for -short amounts of time we don't need a separate stack for interrupts. - - - - -Register Usage & Stackframes on Linux for s/390 & z/Architecture -================================================================= -Overview: ---------- -This is the code that gcc produces at the top & the bottom of -each function. It usually is fairly consistent & similar from -function to function & if you know its layout you can probably -make some headway in finding the ultimate cause of a problem -after a crash without a source level debugger. - -Note: To follow stackframes requires a knowledge of C or Pascal & -limited knowledge of one assembly language. - -It should be noted that there are some differences between the -s/390 and z/Architecture stack layouts as the z/Architecture stack layout -didn't have to maintain compatibility with older linkage formats. - -Glossary: ---------- -alloca: -This is a built in compiler function for runtime allocation -of extra space on the callers stack which is obviously freed -up on function exit ( e.g. the caller may choose to allocate nothing -of a buffer of 4k if required for temporary purposes ), it generates -very efficient code ( a few cycles ) when compared to alternatives -like malloc. - -automatics: These are local variables on the stack, -i.e they aren't in registers & they aren't static. - -back-chain: -This is a pointer to the stack pointer before entering a -framed functions ( see frameless function ) prologue got by -dereferencing the address of the current stack pointer, - i.e. got by accessing the 32 bit value at the stack pointers -current location. - -base-pointer: -This is a pointer to the back of the literal pool which -is an area just behind each procedure used to store constants -in each function. - -call-clobbered: The caller probably needs to save these registers if there -is something of value in them, on the stack or elsewhere before making a -call to another procedure so that it can restore it later. - -epilogue: -The code generated by the compiler to return to the caller. - -frameless-function -A frameless function in Linux for s390 & z/Architecture is one which doesn't -need more than the register save area (96 bytes on s/390, 160 on z/Architecture) -given to it by the caller. -A frameless function never: -1) Sets up a back chain. -2) Calls alloca. -3) Calls other normal functions -4) Has automatics. - -GOT-pointer: -This is a pointer to the global-offset-table in ELF -( Executable Linkable Format, Linux'es most common executable format ), -all globals & shared library objects are found using this pointer. - -lazy-binding -ELF shared libraries are typically only loaded when routines in the shared -library are actually first called at runtime. This is lazy binding. - -procedure-linkage-table -This is a table found from the GOT which contains pointers to routines -in other shared libraries which can't be called to by easier means. - -prologue: -The code generated by the compiler to set up the stack frame. - -outgoing-args: -This is extra area allocated on the stack of the calling function if the -parameters for the callee's cannot all be put in registers, the same -area can be reused by each function the caller calls. - -routine-descriptor: -A COFF executable format based concept of a procedure reference -actually being 8 bytes or more as opposed to a simple pointer to the routine. -This is typically defined as follows -Routine Descriptor offset 0=Pointer to Function -Routine Descriptor offset 4=Pointer to Table of Contents -The table of contents/TOC is roughly equivalent to a GOT pointer. -& it means that shared libraries etc. can be shared between several -environments each with their own TOC. - - -static-chain: This is used in nested functions a concept adopted from pascal -by gcc not used in ansi C or C++ ( although quite useful ), basically it -is a pointer used to reference local variables of enclosing functions. -You might come across this stuff once or twice in your lifetime. - -e.g. -The function below should return 11 though gcc may get upset & toss warnings -about unused variables. -int FunctionA(int a) -{ - int b; - FunctionC(int c) - { - b=c+1; - } - FunctionC(10); - return(b); -} - - -s/390 & z/Architecture Register usage -===================================== -r0 used by syscalls/assembly call-clobbered -r1 used by syscalls/assembly call-clobbered -r2 argument 0 / return value 0 call-clobbered -r3 argument 1 / return value 1 (if long long) call-clobbered -r4 argument 2 call-clobbered -r5 argument 3 call-clobbered -r6 argument 4 saved -r7 pointer-to arguments 5 to ... saved -r8 this & that saved -r9 this & that saved -r10 static-chain ( if nested function ) saved -r11 frame-pointer ( if function used alloca ) saved -r12 got-pointer saved -r13 base-pointer saved -r14 return-address saved -r15 stack-pointer saved - -f0 argument 0 / return value ( float/double ) call-clobbered -f2 argument 1 call-clobbered -f4 z/Architecture argument 2 saved -f6 z/Architecture argument 3 saved -The remaining floating points -f1,f3,f5 f7-f15 are call-clobbered. - -Notes: ------- -1) The only requirement is that registers which are used -by the callee are saved, e.g. the compiler is perfectly -capable of using r11 for purposes other than a frame a -frame pointer if a frame pointer is not needed. -2) In functions with variable arguments e.g. printf the calling procedure -is identical to one without variable arguments & the same number of -parameters. However, the prologue of this function is somewhat more -hairy owing to it having to move these parameters to the stack to -get va_start, va_arg & va_end to work. -3) Access registers are currently unused by gcc but are used in -the kernel. Possibilities exist to use them at the moment for -temporary storage but it isn't recommended. -4) Only 4 of the floating point registers are used for -parameter passing as older machines such as G3 only have only 4 -& it keeps the stack frame compatible with other compilers. -However with IEEE floating point emulation under linux on the -older machines you are free to use the other 12. -5) A long long or double parameter cannot be have the -first 4 bytes in a register & the second four bytes in the -outgoing args area. It must be purely in the outgoing args -area if crossing this boundary. -6) Floating point parameters are mixed with outgoing args -on the outgoing args area in the order the are passed in as parameters. -7) Floating point arguments 2 & 3 are saved in the outgoing args area for -z/Architecture - - -Stack Frame Layout ------------------- -s/390 z/Architecture -0 0 back chain ( a 0 here signifies end of back chain ) -4 8 eos ( end of stack, not used on Linux for S390 used in other linkage formats ) -8 16 glue used in other s/390 linkage formats for saved routine descriptors etc. -12 24 glue used in other s/390 linkage formats for saved routine descriptors etc. -16 32 scratch area -20 40 scratch area -24 48 saved r6 of caller function -28 56 saved r7 of caller function -32 64 saved r8 of caller function -36 72 saved r9 of caller function -40 80 saved r10 of caller function -44 88 saved r11 of caller function -48 96 saved r12 of caller function -52 104 saved r13 of caller function -56 112 saved r14 of caller function -60 120 saved r15 of caller function -64 128 saved f4 of caller function -72 132 saved f6 of caller function -80 undefined -96 160 outgoing args passed from caller to callee -96+x 160+x possible stack alignment ( 8 bytes desirable ) -96+x+y 160+x+y alloca space of caller ( if used ) -96+x+y+z 160+x+y+z automatics of caller ( if used ) -0 back-chain - -A sample program with comments. -=============================== - -Comments on the function test ------------------------------ -1) It didn't need to set up a pointer to the constant pool gpr13 as it is not -used ( :-( ). -2) This is a frameless function & no stack is bought. -3) The compiler was clever enough to recognise that it could return the -value in r2 as well as use it for the passed in parameter ( :-) ). -4) The basr ( branch relative & save ) trick works as follows the instruction -has a special case with r0,r0 with some instruction operands is understood as -the literal value 0, some risc architectures also do this ). So now -we are branching to the next address & the address new program counter is -in r13,so now we subtract the size of the function prologue we have executed -+ the size of the literal pool to get to the top of the literal pool -0040037c int test(int b) -{ # Function prologue below - 40037c: 90 de f0 34 stm %r13,%r14,52(%r15) # Save registers r13 & r14 - 400380: 0d d0 basr %r13,%r0 # Set up pointer to constant pool using - 400382: a7 da ff fa ahi %r13,-6 # basr trick - return(5+b); - # Huge main program - 400386: a7 2a 00 05 ahi %r2,5 # add 5 to r2 - - # Function epilogue below - 40038a: 98 de f0 34 lm %r13,%r14,52(%r15) # restore registers r13 & 14 - 40038e: 07 fe br %r14 # return -} - -Comments on the function main ------------------------------ -1) The compiler did this function optimally ( 8-) ) - -Literal pool for main. -400390: ff ff ff ec .long 0xffffffec -main(int argc,char *argv[]) -{ # Function prologue below - 400394: 90 bf f0 2c stm %r11,%r15,44(%r15) # Save necessary registers - 400398: 18 0f lr %r0,%r15 # copy stack pointer to r0 - 40039a: a7 fa ff a0 ahi %r15,-96 # Make area for callee saving - 40039e: 0d d0 basr %r13,%r0 # Set up r13 to point to - 4003a0: a7 da ff f0 ahi %r13,-16 # literal pool - 4003a4: 50 00 f0 00 st %r0,0(%r15) # Save backchain - - return(test(5)); # Main Program Below - 4003a8: 58 e0 d0 00 l %r14,0(%r13) # load relative address of test from - # literal pool - 4003ac: a7 28 00 05 lhi %r2,5 # Set first parameter to 5 - 4003b0: 4d ee d0 00 bas %r14,0(%r14,%r13) # jump to test setting r14 as return - # address using branch & save instruction. - - # Function Epilogue below - 4003b4: 98 bf f0 8c lm %r11,%r15,140(%r15)# Restore necessary registers. - 4003b8: 07 fe br %r14 # return to do program exit -} - - -Compiler updates ----------------- - -main(int argc,char *argv[]) -{ - 4004fc: 90 7f f0 1c stm %r7,%r15,28(%r15) - 400500: a7 d5 00 04 bras %r13,400508 <main+0xc> - 400504: 00 40 04 f4 .long 0x004004f4 - # compiler now puts constant pool in code to so it saves an instruction - 400508: 18 0f lr %r0,%r15 - 40050a: a7 fa ff a0 ahi %r15,-96 - 40050e: 50 00 f0 00 st %r0,0(%r15) - return(test(5)); - 400512: 58 10 d0 00 l %r1,0(%r13) - 400516: a7 28 00 05 lhi %r2,5 - 40051a: 0d e1 basr %r14,%r1 - # compiler adds 1 extra instruction to epilogue this is done to - # avoid processor pipeline stalls owing to data dependencies on g5 & - # above as register 14 in the old code was needed directly after being loaded - # by the lm %r11,%r15,140(%r15) for the br %14. - 40051c: 58 40 f0 98 l %r4,152(%r15) - 400520: 98 7f f0 7c lm %r7,%r15,124(%r15) - 400524: 07 f4 br %r4 -} - - -Hartmut ( our compiler developer ) also has been threatening to take out the -stack backchain in optimised code as this also causes pipeline stalls, you -have been warned. - -64 bit z/Architecture code disassembly --------------------------------------- - -If you understand the stuff above you'll understand the stuff -below too so I'll avoid repeating myself & just say that -some of the instructions have g's on the end of them to indicate -they are 64 bit & the stack offsets are a bigger, -the only other difference you'll find between 32 & 64 bit is that -we now use f4 & f6 for floating point arguments on 64 bit. -00000000800005b0 <test>: -int test(int b) -{ - return(5+b); - 800005b0: a7 2a 00 05 ahi %r2,5 - 800005b4: b9 14 00 22 lgfr %r2,%r2 # downcast to integer - 800005b8: 07 fe br %r14 - 800005ba: 07 07 bcr 0,%r7 - - -} - -00000000800005bc <main>: -main(int argc,char *argv[]) -{ - 800005bc: eb bf f0 58 00 24 stmg %r11,%r15,88(%r15) - 800005c2: b9 04 00 1f lgr %r1,%r15 - 800005c6: a7 fb ff 60 aghi %r15,-160 - 800005ca: e3 10 f0 00 00 24 stg %r1,0(%r15) - return(test(5)); - 800005d0: a7 29 00 05 lghi %r2,5 - # brasl allows jumps > 64k & is overkill here bras would do fune - 800005d4: c0 e5 ff ff ff ee brasl %r14,800005b0 <test> - 800005da: e3 40 f1 10 00 04 lg %r4,272(%r15) - 800005e0: eb bf f0 f8 00 04 lmg %r11,%r15,248(%r15) - 800005e6: 07 f4 br %r4 -} - - - -Compiling programs for debugging on Linux for s/390 & z/Architecture -==================================================================== --gdwarf-2 now works it should be considered the default debugging -format for s/390 & z/Architecture as it is more reliable for debugging -shared libraries, normal -g debugging works much better now -Thanks to the IBM java compiler developers bug reports. - -This is typically done adding/appending the flags -g or -gdwarf-2 to the -CFLAGS & LDFLAGS variables Makefile of the program concerned. - -If using gdb & you would like accurate displays of registers & - stack traces compile without optimisation i.e make sure -that there is no -O2 or similar on the CFLAGS line of the Makefile & -the emitted gcc commands, obviously this will produce worse code -( not advisable for shipment ) but it is an aid to the debugging process. - -This aids debugging because the compiler will copy parameters passed in -in registers onto the stack so backtracing & looking at passed in -parameters will work, however some larger programs which use inline functions -will not compile without optimisation. - -Debugging with optimisation has since much improved after fixing -some bugs, please make sure you are using gdb-5.0 or later developed -after Nov'2000. - - - -Debugging under VM -================== - -Notes ------ -Addresses & values in the VM debugger are always hex never decimal -Address ranges are of the format <HexValue1>-<HexValue2> or -<HexValue1>.<HexValue2> -For example, the address range 0x2000 to 0x3000 can be described as 2000-3000 -or 2000.1000 - -The VM Debugger is case insensitive. - -VM's strengths are usually other debuggers weaknesses you can get at any -resource no matter how sensitive e.g. memory management resources, change -address translation in the PSW. For kernel hacking you will reap dividends if -you get good at it. - -The VM Debugger displays operators but not operands, and also the debugger -displays useful information on the same line as the author of the code probably -felt that it was a good idea not to go over the 80 columns on the screen. -This isn't as unintuitive as it may seem as the s/390 instructions are easy to -decode mentally and you can make a good guess at a lot of them as all the -operands are nibble (half byte aligned). -So if you have an objdump listing by hand, it is quite easy to follow, and if -you don't have an objdump listing keep a copy of the s/390 Reference Summary -or alternatively the s/390 principles of operation next to you. -e.g. even I can guess that -0001AFF8' LR 180F CC 0 -is a ( load register ) lr r0,r15 - -Also it is very easy to tell the length of a 390 instruction from the 2 most -significant bits in the instruction (not that this info is really useful except -if you are trying to make sense of a hexdump of code). -Here is a table -Bits Instruction Length ------------------------------------------- -00 2 Bytes -01 4 Bytes -10 4 Bytes -11 6 Bytes - -The debugger also displays other useful info on the same line such as the -addresses being operated on destination addresses of branches & condition codes. -e.g. -00019736' AHI A7DAFF0E CC 1 -000198BA' BRC A7840004 -> 000198C2' CC 0 -000198CE' STM 900EF068 >> 0FA95E78 CC 2 - - - -Useful VM debugger commands ---------------------------- - -I suppose I'd better mention this before I start -to list the current active traces do -Q TR -there can be a maximum of 255 of these per set -( more about trace sets later ). -To stop traces issue a -TR END. -To delete a particular breakpoint issue -TR DEL <breakpoint number> - -The PA1 key drops to CP mode so you can issue debugger commands, -Doing alt c (on my 3270 console at least ) clears the screen. -hitting b <enter> comes back to the running operating system -from cp mode ( in our case linux ). -It is typically useful to add shortcuts to your profile.exec file -if you have one ( this is roughly equivalent to autoexec.bat in DOS ). -file here are a few from mine. -/* this gives me command history on issuing f12 */ -set pf12 retrieve -/* this continues */ -set pf8 imm b -/* goes to trace set a */ -set pf1 imm tr goto a -/* goes to trace set b */ -set pf2 imm tr goto b -/* goes to trace set c */ -set pf3 imm tr goto c - - - -Instruction Tracing -------------------- -Setting a simple breakpoint -TR I PSWA <address> -To debug a particular function try -TR I R <function address range> -TR I on its own will single step. -TR I DATA <MNEMONIC> <OPTIONAL RANGE> will trace for particular mnemonics -e.g. -TR I DATA 4D R 0197BC.4000 -will trace for BAS'es ( opcode 4D ) in the range 0197BC.4000 -if you were inclined you could add traces for all branch instructions & -suffix them with the run prefix so you would have a backtrace on screen -when a program crashes. -TR BR <INTO OR FROM> will trace branches into or out of an address. -e.g. -TR BR INTO 0 is often quite useful if a program is getting awkward & deciding -to branch to 0 & crashing as this will stop at the address before in jumps to 0. -TR I R <address range> RUN cmd d g -single steps a range of addresses but stays running & -displays the gprs on each step. - - - -Displaying & modifying Registers --------------------------------- -D G will display all the gprs -Adding a extra G to all the commands is necessary to access the full 64 bit -content in VM on z/Architecture. Obviously this isn't required for access -registers as these are still 32 bit. -e.g. DGG instead of DG -D X will display all the control registers -D AR will display all the access registers -D AR4-7 will display access registers 4 to 7 -CPU ALL D G will display the GRPS of all CPUS in the configuration -D PSW will display the current PSW -st PSW 2000 will put the value 2000 into the PSW & -cause crash your machine. -D PREFIX displays the prefix offset - - -Displaying Memory ------------------ -To display memory mapped using the current PSW's mapping try -D <range> -To make VM display a message each time it hits a particular address and -continue try -D I<range> will disassemble/display a range of instructions. -ST addr 32 bit word will store a 32 bit aligned address -D T<range> will display the EBCDIC in an address (if you are that way inclined) -D R<range> will display real addresses ( without DAT ) but with prefixing. -There are other complex options to display if you need to get at say home space -but are in primary space the easiest thing to do is to temporarily -modify the PSW to the other addressing mode, display the stuff & then -restore it. - - - -Hints ------ -If you want to issue a debugger command without halting your virtual machine -with the PA1 key try prefixing the command with #CP e.g. -#cp tr i pswa 2000 -also suffixing most debugger commands with RUN will cause them not -to stop just display the mnemonic at the current instruction on the console. -If you have several breakpoints you want to put into your program & -you get fed up of cross referencing with System.map -you can do the following trick for several symbols. -grep do_signal System.map -which emits the following among other things -0001f4e0 T do_signal -now you can do - -TR I PSWA 0001f4e0 cmd msg * do_signal -This sends a message to your own console each time do_signal is entered. -( As an aside I wrote a perl script once which automatically generated a REXX -script with breakpoints on every kernel procedure, this isn't a good idea -because there are thousands of these routines & VM can only set 255 breakpoints -at a time so you nearly had to spend as long pruning the file down as you would -entering the msgs by hand), however, the trick might be useful for a single -object file. In the 3270 terminal emulator x3270 there is a very useful option -in the file menu called "Save Screen In File" - this is very good for keeping a -copy of traces. - -From CMS help <command name> will give you online help on a particular command. -e.g. -HELP DISPLAY - -Also CP has a file called profile.exec which automatically gets called -on startup of CMS ( like autoexec.bat ), keeping on a DOS analogy session -CP has a feature similar to doskey, it may be useful for you to -use profile.exec to define some keystrokes. -e.g. -SET PF9 IMM B -This does a single step in VM on pressing F8. -SET PF10 ^ -This sets up the ^ key. -which can be used for ^c (ctrl-c),^z (ctrl-z) which can't be typed directly -into some 3270 consoles. -SET PF11 ^- -This types the starting keystrokes for a sysrq see SysRq below. -SET PF12 RETRIEVE -This retrieves command history on pressing F12. - - -Sometimes in VM the display is set up to scroll automatically this -can be very annoying if there are messages you wish to look at -to stop this do -TERM MORE 255 255 -This will nearly stop automatic screen updates, however it will -cause a denial of service if lots of messages go to the 3270 console, -so it would be foolish to use this as the default on a production machine. - - -Tracing particular processes ----------------------------- -The kernel's text segment is intentionally at an address in memory that it will -very seldom collide with text segments of user programs ( thanks Martin ), -this simplifies debugging the kernel. -However it is quite common for user processes to have addresses which collide -this can make debugging a particular process under VM painful under normal -circumstances as the process may change when doing a -TR I R <address range>. -Thankfully after reading VM's online help I figured out how to debug -I particular process. - -Your first problem is to find the STD ( segment table designation ) -of the program you wish to debug. -There are several ways you can do this here are a few -1) objdump --syms <program to be debugged> | grep main -To get the address of main in the program. -tr i pswa <address of main> -Start the program, if VM drops to CP on what looks like the entry -point of the main function this is most likely the process you wish to debug. -Now do a D X13 or D XG13 on z/Architecture. -On 31 bit the STD is bits 1-19 ( the STO segment table origin ) -& 25-31 ( the STL segment table length ) of CR13. -now type -TR I R STD <CR13's value> 0.7fffffff -e.g. -TR I R STD 8F32E1FF 0.7fffffff -Another very useful variation is -TR STORE INTO STD <CR13's value> <address range> -for finding out when a particular variable changes. - -An alternative way of finding the STD of a currently running process -is to do the following, ( this method is more complex but -could be quite convenient if you aren't updating the kernel much & -so your kernel structures will stay constant for a reasonable period of -time ). - -grep task /proc/<pid>/status -from this you should see something like -task: 0f160000 ksp: 0f161de8 pt_regs: 0f161f68 -This now gives you a pointer to the task structure. -Now make CC:="s390-gcc -g" kernel/sched.s -To get the task_struct stabinfo. -( task_struct is defined in include/linux/sched.h ). -Now we want to look at -task->active_mm->pgd -on my machine the active_mm in the task structure stab is -active_mm:(4,12),672,32 -its offset is 672/8=84=0x54 -the pgd member in the mm_struct stab is -pgd:(4,6)=*(29,5),96,32 -so its offset is 96/8=12=0xc - -so we'll -hexdump -s 0xf160054 /dev/mem | more -i.e. task_struct+active_mm offset -to look at the active_mm member -f160054 0fee cc60 0019 e334 0000 0000 0000 0011 -hexdump -s 0x0feecc6c /dev/mem | more -i.e. active_mm+pgd offset -feecc6c 0f2c 0000 0000 0001 0000 0001 0000 0010 -we get something like -now do -TR I R STD <pgd|0x7f> 0.7fffffff -i.e. the 0x7f is added because the pgd only -gives the page table origin & we need to set the low bits -to the maximum possible segment table length. -TR I R STD 0f2c007f 0.7fffffff -on z/Architecture you'll probably need to do -TR I R STD <pgd|0x7> 0.ffffffffffffffff -to set the TableType to 0x1 & the Table length to 3. - - - -Tracing Program Exceptions --------------------------- -If you get a crash which says something like -illegal operation or specification exception followed by a register dump -You can restart linux & trace these using the tr prog <range or value> trace -option. - - -The most common ones you will normally be tracing for is -1=operation exception -2=privileged operation exception -4=protection exception -5=addressing exception -6=specification exception -10=segment translation exception -11=page translation exception - -The full list of these is on page 22 of the current s/390 Reference Summary. -e.g. -tr prog 10 will trace segment translation exceptions. -tr prog on its own will trace all program interruption codes. - -Trace Sets ----------- -On starting VM you are initially in the INITIAL trace set. -You can do a Q TR to verify this. -If you have a complex tracing situation where you wish to wait for instance -till a driver is open before you start tracing IO, but know in your -heart that you are going to have to make several runs through the code till you -have a clue whats going on. - -What you can do is -TR I PSWA <Driver open address> -hit b to continue till breakpoint -reach the breakpoint -now do your -TR GOTO B -TR IO 7c08-7c09 inst int run -or whatever the IO channels you wish to trace are & hit b - -To got back to the initial trace set do -TR GOTO INITIAL -& the TR I PSWA <Driver open address> will be the only active breakpoint again. - - -Tracing linux syscalls under VM -------------------------------- -Syscalls are implemented on Linux for S390 by the Supervisor call instruction -(SVC). There 256 possibilities of these as the instruction is made up of a 0xA -opcode and the second byte being the syscall number. They are traced using the -simple command: -TR SVC <Optional value or range> -the syscalls are defined in linux/arch/s390/include/asm/unistd.h -e.g. to trace all file opens just do -TR SVC 5 ( as this is the syscall number of open ) - - -SMP Specific commands ---------------------- -To find out how many cpus you have -Q CPUS displays all the CPU's available to your virtual machine -To find the cpu that the current cpu VM debugger commands are being directed at -do Q CPU to change the current cpu VM debugger commands are being directed at do -CPU <desired cpu no> - -On a SMP guest issue a command to all CPUs try prefixing the command with cpu -all. To issue a command to a particular cpu try cpu <cpu number> e.g. -CPU 01 TR I R 2000.3000 -If you are running on a guest with several cpus & you have a IO related problem -& cannot follow the flow of code but you know it isn't smp related. -from the bash prompt issue -shutdown -h now or halt. -do a Q CPUS to find out how many cpus you have -detach each one of them from cp except cpu 0 -by issuing a -DETACH CPU 01-(number of cpus in configuration) -& boot linux again. -TR SIGP will trace inter processor signal processor instructions. -DEFINE CPU 01-(number in configuration) -will get your guests cpus back. - - -Help for displaying ascii textstrings -------------------------------------- -On the very latest VM Nucleus'es VM can now display ascii -( thanks Neale for the hint ) by doing -D TX<lowaddr>.<len> -e.g. -D TX0.100 - -Alternatively -============= -Under older VM debuggers (I love EBDIC too) you can use following little -program which converts a command line of hex digits to ascii text. It can be -compiled under linux and you can copy the hex digits from your x3270 terminal -to your xterm if you are debugging from a linuxbox. - -This is quite useful when looking at a parameter passed in as a text string -under VM ( unless you are good at decoding ASCII in your head ). - -e.g. consider tracing an open syscall -TR SVC 5 -We have stopped at a breakpoint -000151B0' SVC 0A05 -> 0001909A' CC 0 - -D 20.8 to check the SVC old psw in the prefix area and see was it from userspace -(for the layout of the prefix area consult the "Fixed Storage Locations" -chapter of the s/390 Reference Summary if you have it available). -V00000020 070C2000 800151B2 -The problem state bit wasn't set & it's also too early in the boot sequence -for it to be a userspace SVC if it was we would have to temporarily switch the -psw to user space addressing so we could get at the first parameter of the open -in gpr2. -Next do a -D G2 -GPR 2 = 00014CB4 -Now display what gpr2 is pointing to -D 00014CB4.20 -V00014CB4 2F646576 2F636F6E 736F6C65 00001BF5 -V00014CC4 FC00014C B4001001 E0001000 B8070707 -Now copy the text till the first 00 hex ( which is the end of the string -to an xterm & do hex2ascii on it. -hex2ascii 2F646576 2F636F6E 736F6C65 00 -outputs -Decoded Hex:=/ d e v / c o n s o l e 0x00 -We were opening the console device, - -You can compile the code below yourself for practice :-), -/* - * hex2ascii.c - * a useful little tool for converting a hexadecimal command line to ascii - * - * Author(s): Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com) - * (C) 2000 IBM Deutschland Entwicklung GmbH, IBM Corporation. - */ -#include <stdio.h> - -int main(int argc,char *argv[]) -{ - int cnt1,cnt2,len,toggle=0; - int startcnt=1; - unsigned char c,hex; - - if(argc>1&&(strcmp(argv[1],"-a")==0)) - startcnt=2; - printf("Decoded Hex:="); - for(cnt1=startcnt;cnt1<argc;cnt1++) - { - len=strlen(argv[cnt1]); - for(cnt2=0;cnt2<len;cnt2++) - { - c=argv[cnt1][cnt2]; - if(c>='0'&&c<='9') - c=c-'0'; - if(c>='A'&&c<='F') - c=c-'A'+10; - if(c>='a'&&c<='f') - c=c-'a'+10; - switch(toggle) - { - case 0: - hex=c<<4; - toggle=1; - break; - case 1: - hex+=c; - if(hex<32||hex>127) - { - if(startcnt==1) - printf("0x%02X ",(int)hex); - else - printf("."); - } - else - { - printf("%c",hex); - if(startcnt==1) - printf(" "); - } - toggle=0; - break; - } - } - } - printf("\n"); -} - - - - -Stack tracing under VM ----------------------- -A basic backtrace ------------------ - -Here are the tricks I use 9 out of 10 times it works pretty well, - -When your backchain reaches a dead end --------------------------------------- -This can happen when an exception happens in the kernel and the kernel is -entered twice. If you reach the NULL pointer at the end of the back chain you -should be able to sniff further back if you follow the following tricks. -1) A kernel address should be easy to recognise since it is in -primary space & the problem state bit isn't set & also -The Hi bit of the address is set. -2) Another backchain should also be easy to recognise since it is an -address pointing to another address approximately 100 bytes or 0x70 hex -behind the current stackpointer. - - -Here is some practice. -boot the kernel & hit PA1 at some random time -d g to display the gprs, this should display something like -GPR 0 = 00000001 00156018 0014359C 00000000 -GPR 4 = 00000001 001B8888 000003E0 00000000 -GPR 8 = 00100080 00100084 00000000 000FE000 -GPR 12 = 00010400 8001B2DC 8001B36A 000FFED8 -Note that GPR14 is a return address but as we are real men we are going to -trace the stack. -display 0x40 bytes after the stack pointer. - -V000FFED8 000FFF38 8001B838 80014C8E 000FFF38 -V000FFEE8 00000000 00000000 000003E0 00000000 -V000FFEF8 00100080 00100084 00000000 000FE000 -V000FFF08 00010400 8001B2DC 8001B36A 000FFED8 - - -Ah now look at whats in sp+56 (sp+0x38) this is 8001B36A our saved r14 if -you look above at our stackframe & also agrees with GPR14. - -now backchain -d 000FFF38.40 -we now are taking the contents of SP to get our first backchain. - -V000FFF38 000FFFA0 00000000 00014995 00147094 -V000FFF48 00147090 001470A0 000003E0 00000000 -V000FFF58 00100080 00100084 00000000 001BF1D0 -V000FFF68 00010400 800149BA 80014CA6 000FFF38 - -This displays a 2nd return address of 80014CA6 - -now do d 000FFFA0.40 for our 3rd backchain - -V000FFFA0 04B52002 0001107F 00000000 00000000 -V000FFFB0 00000000 00000000 FF000000 0001107F -V000FFFC0 00000000 00000000 00000000 00000000 -V000FFFD0 00010400 80010802 8001085A 000FFFA0 - - -our 3rd return address is 8001085A - -as the 04B52002 looks suspiciously like rubbish it is fair to assume that the -kernel entry routines for the sake of optimisation don't set up a backchain. - -now look at System.map to see if the addresses make any sense. - -grep -i 0001b3 System.map -outputs among other things -0001b304 T cpu_idle -so 8001B36A -is cpu_idle+0x66 ( quiet the cpu is asleep, don't wake it ) - - -grep -i 00014 System.map -produces among other things -00014a78 T start_kernel -so 0014CA6 is start_kernel+some hex number I can't add in my head. - -grep -i 00108 System.map -this produces -00010800 T _stext -so 8001085A is _stext+0x5a - -Congrats you've done your first backchain. - - - -s/390 & z/Architecture IO Overview -================================== - -I am not going to give a course in 390 IO architecture as this would take me -quite a while and I'm no expert. Instead I'll give a 390 IO architecture -summary for Dummies. If you have the s/390 principles of operation available -read this instead. If nothing else you may find a few useful keywords in here -and be able to use them on a web search engine to find more useful information. - -Unlike other bus architectures modern 390 systems do their IO using mostly -fibre optics and devices such as tapes and disks can be shared between several -mainframes. Also S390 can support up to 65536 devices while a high end PC based -system might be choking with around 64. - -Here is some of the common IO terminology: - -Subchannel: -This is the logical number most IO commands use to talk to an IO device. There -can be up to 0x10000 (65536) of these in a configuration, typically there are a -few hundred. Under VM for simplicity they are allocated contiguously, however -on the native hardware they are not. They typically stay consistent between -boots provided no new hardware is inserted or removed. -Under Linux for s390 we use these as IRQ's and also when issuing an IO command -(CLEAR SUBCHANNEL, HALT SUBCHANNEL, MODIFY SUBCHANNEL, RESUME SUBCHANNEL, -START SUBCHANNEL, STORE SUBCHANNEL and TEST SUBCHANNEL). We use this as the ID -of the device we wish to talk to. The most important of these instructions are -START SUBCHANNEL (to start IO), TEST SUBCHANNEL (to check whether the IO -completed successfully) and HALT SUBCHANNEL (to kill IO). A subchannel can have -up to 8 channel paths to a device, this offers redundancy if one is not -available. - -Device Number: -This number remains static and is closely tied to the hardware. There are 65536 -of these, made up of a CHPID (Channel Path ID, the most significant 8 bits) and -another lsb 8 bits. These remain static even if more devices are inserted or -removed from the hardware. There is a 1 to 1 mapping between subchannels and -device numbers, provided devices aren't inserted or removed. - -Channel Control Words: -CCWs are linked lists of instructions initially pointed to by an operation -request block (ORB), which is initially given to Start Subchannel (SSCH) -command along with the subchannel number for the IO subsystem to process -while the CPU continues executing normal code. -CCWs come in two flavours, Format 0 (24 bit for backward compatibility) and -Format 1 (31 bit). These are typically used to issue read and write (and many -other) instructions. They consist of a length field and an absolute address -field. -Each IO typically gets 1 or 2 interrupts, one for channel end (primary status) -when the channel is idle, and the second for device end (secondary status). -Sometimes you get both concurrently. You check how the IO went on by issuing a -TEST SUBCHANNEL at each interrupt, from which you receive an Interruption -response block (IRB). If you get channel and device end status in the IRB -without channel checks etc. your IO probably went okay. If you didn't you -probably need to examine the IRB, extended status word etc. -If an error occurs, more sophisticated control units have a facility known as -concurrent sense. This means that if an error occurs Extended sense information -will be presented in the Extended status word in the IRB. If not you have to -issue a subsequent SENSE CCW command after the test subchannel. - - -TPI (Test pending interrupt) can also be used for polled IO, but in -multitasking multiprocessor systems it isn't recommended except for -checking special cases (i.e. non looping checks for pending IO etc.). - -Store Subchannel and Modify Subchannel can be used to examine and modify -operating characteristics of a subchannel (e.g. channel paths). - -Other IO related Terms: -Sysplex: S390's Clustering Technology -QDIO: S390's new high speed IO architecture to support devices such as gigabit -ethernet, this architecture is also designed to be forward compatible with -upcoming 64 bit machines. - - -General Concepts - -Input Output Processors (IOP's) are responsible for communicating between -the mainframe CPU's & the channel & relieve the mainframe CPU's from the -burden of communicating with IO devices directly, this allows the CPU's to -concentrate on data processing. - -IOP's can use one or more links ( known as channel paths ) to talk to each -IO device. It first checks for path availability & chooses an available one, -then starts ( & sometimes terminates IO ). -There are two types of channel path: ESCON & the Parallel IO interface. - -IO devices are attached to control units, control units provide the -logic to interface the channel paths & channel path IO protocols to -the IO devices, they can be integrated with the devices or housed separately -& often talk to several similar devices ( typical examples would be raid -controllers or a control unit which connects to 1000 3270 terminals ). - - - +---------------------------------------------------------------+ - | +-----+ +-----+ +-----+ +-----+ +----------+ +----------+ | - | | CPU | | CPU | | CPU | | CPU | | Main | | Expanded | | - | | | | | | | | | | Memory | | Storage | | - | +-----+ +-----+ +-----+ +-----+ +----------+ +----------+ | - |---------------------------------------------------------------+ - | IOP | IOP | IOP | - |--------------------------------------------------------------- - | C | C | C | C | C | C | C | C | C | C | C | C | C | C | C | C | - ---------------------------------------------------------------- - || || - || Bus & Tag Channel Path || ESCON - || ====================== || Channel - || || || || Path - +----------+ +----------+ +----------+ - | | | | | | - | CU | | CU | | CU | - | | | | | | - +----------+ +----------+ +----------+ - | | | | | -+----------+ +----------+ +----------+ +----------+ +----------+ -|I/O Device| |I/O Device| |I/O Device| |I/O Device| |I/O Device| -+----------+ +----------+ +----------+ +----------+ +----------+ - CPU = Central Processing Unit - C = Channel - IOP = IP Processor - CU = Control Unit - -The 390 IO systems come in 2 flavours the current 390 machines support both - -The Older 360 & 370 Interface,sometimes called the Parallel I/O interface, -sometimes called Bus-and Tag & sometimes Original Equipment Manufacturers -Interface (OEMI). - -This byte wide Parallel channel path/bus has parity & data on the "Bus" cable -and control lines on the "Tag" cable. These can operate in byte multiplex mode -for sharing between several slow devices or burst mode and monopolize the -channel for the whole burst. Up to 256 devices can be addressed on one of these -cables. These cables are about one inch in diameter. The maximum unextended -length supported by these cables is 125 Meters but this can be extended up to -2km with a fibre optic channel extended such as a 3044. The maximum burst speed -supported is 4.5 megabytes per second. However, some really old processors -support only transfer rates of 3.0, 2.0 & 1.0 MB/sec. -One of these paths can be daisy chained to up to 8 control units. - - -ESCON if fibre optic it is also called FICON -Was introduced by IBM in 1990. Has 2 fibre optic cables and uses either leds or -lasers for communication at a signaling rate of up to 200 megabits/sec. As -10bits are transferred for every 8 bits info this drops to 160 megabits/sec -and to 18.6 Megabytes/sec once control info and CRC are added. ESCON only -operates in burst mode. - -ESCONs typical max cable length is 3km for the led version and 20km for the -laser version known as XDF (extended distance facility). This can be further -extended by using an ESCON director which triples the above mentioned ranges. -Unlike Bus & Tag as ESCON is serial it uses a packet switching architecture, -the standard Bus & Tag control protocol is however present within the packets. -Up to 256 devices can be attached to each control unit that uses one of these -interfaces. - -Common 390 Devices include: -Network adapters typically OSA2,3172's,2116's & OSA-E gigabit ethernet adapters, -Consoles 3270 & 3215 (a teletype emulated under linux for a line mode console). -DASD's direct access storage devices ( otherwise known as hard disks ). -Tape Drives. -CTC ( Channel to Channel Adapters ), -ESCON or Parallel Cables used as a very high speed serial link -between 2 machines. - - -Debugging IO on s/390 & z/Architecture under VM -=============================================== - -Now we are ready to go on with IO tracing commands under VM - -A few self explanatory queries: -Q OSA -Q CTC -Q DISK ( This command is CMS specific ) -Q DASD - - - - - - -Q OSA on my machine returns -OSA 7C08 ON OSA 7C08 SUBCHANNEL = 0000 -OSA 7C09 ON OSA 7C09 SUBCHANNEL = 0001 -OSA 7C14 ON OSA 7C14 SUBCHANNEL = 0002 -OSA 7C15 ON OSA 7C15 SUBCHANNEL = 0003 - -If you have a guest with certain privileges you may be able to see devices -which don't belong to you. To avoid this, add the option V. -e.g. -Q V OSA - -Now using the device numbers returned by this command we will -Trace the io starting up on the first device 7c08 & 7c09 -In our simplest case we can trace the -start subchannels -like TR SSCH 7C08-7C09 -or the halt subchannels -or TR HSCH 7C08-7C09 -MSCH's ,STSCH's I think you can guess the rest - -A good trick is tracing all the IO's and CCWS and spooling them into the reader -of another VM guest so he can ftp the logfile back to his own machine. I'll do -a small bit of this and give you a look at the output. - -1) Spool stdout to VM reader -SP PRT TO (another vm guest ) or * for the local vm guest -2) Fill the reader with the trace -TR IO 7c08-7c09 INST INT CCW PRT RUN -3) Start up linux -i 00c -4) Finish the trace -TR END -5) close the reader -C PRT -6) list reader contents -RDRLIST -7) copy it to linux4's minidisk -RECEIVE / LOG TXT A1 ( replace -8) -filel & press F11 to look at it -You should see something like: - -00020942' SSCH B2334000 0048813C CC 0 SCH 0000 DEV 7C08 - CPA 000FFDF0 PARM 00E2C9C4 KEY 0 FPI C0 LPM 80 - CCW 000FFDF0 E4200100 00487FE8 0000 E4240100 ........ - IDAL 43D8AFE8 - IDAL 0FB76000 -00020B0A' I/O DEV 7C08 -> 000197BC' SCH 0000 PARM 00E2C9C4 -00021628' TSCH B2354000 >> 00488164 CC 0 SCH 0000 DEV 7C08 - CCWA 000FFDF8 DEV STS 0C SCH STS 00 CNT 00EC - KEY 0 FPI C0 CC 0 CTLS 4007 -00022238' STSCH B2344000 >> 00488108 CC 0 SCH 0000 DEV 7C08 - -If you don't like messing up your readed ( because you possibly booted from it ) -you can alternatively spool it to another readers guest. - - -Other common VM device related commands ---------------------------------------------- -These commands are listed only because they have -been of use to me in the past & may be of use to -you too. For more complete info on each of the commands -use type HELP <command> from CMS. -detaching devices -DET <devno range> -ATT <devno range> <guest> -attach a device to guest * for your own guest -READY <devno> cause VM to issue a fake interrupt. - -The VARY command is normally only available to VM administrators. -VARY ON PATH <path> TO <devno range> -VARY OFF PATH <PATH> FROM <devno range> -This is used to switch on or off channel paths to devices. - -Q CHPID <channel path ID> -This displays state of devices using this channel path -D SCHIB <subchannel> -This displays the subchannel information SCHIB block for the device. -this I believe is also only available to administrators. -DEFINE CTC <devno> -defines a virtual CTC channel to channel connection -2 need to be defined on each guest for the CTC driver to use. -COUPLE devno userid remote devno -Joins a local virtual device to a remote virtual device -( commonly used for the CTC driver ). - -Building a VM ramdisk under CMS which linux can use -def vfb-<blocksize> <subchannel> <number blocks> -blocksize is commonly 4096 for linux. -Formatting it -format <subchannel> <driver letter e.g. x> (blksize <blocksize> - -Sharing a disk between multiple guests -LINK userid devno1 devno2 mode password - - - -GDB on S390 -=========== -N.B. if compiling for debugging gdb works better without optimisation -( see Compiling programs for debugging ) - -invocation ----------- -gdb <victim program> <optional corefile> - -Online help ------------ -help: gives help on commands -e.g. -help -help display -Note gdb's online help is very good use it. - - -Assembly --------- -info registers: displays registers other than floating point. -info all-registers: displays floating points as well. -disassemble: disassembles -e.g. -disassemble without parameters will disassemble the current function -disassemble $pc $pc+10 - -Viewing & modifying variables ------------------------------ -print or p: displays variable or register -e.g. p/x $sp will display the stack pointer - -display: prints variable or register each time program stops -e.g. -display/x $pc will display the program counter -display argc - -undisplay : undo's display's - -info breakpoints: shows all current breakpoints - -info stack: shows stack back trace (if this doesn't work too well, I'll show -you the stacktrace by hand below). - -info locals: displays local variables. - -info args: display current procedure arguments. - -set args: will set argc & argv each time the victim program is invoked. - -set <variable>=value -set argc=100 -set $pc=0 - - - -Modifying execution -------------------- -step: steps n lines of sourcecode -step steps 1 line. -step 100 steps 100 lines of code. - -next: like step except this will not step into subroutines - -stepi: steps a single machine code instruction. -e.g. stepi 100 - -nexti: steps a single machine code instruction but will not step into -subroutines. - -finish: will run until exit of the current routine - -run: (re)starts a program - -cont: continues a program - -quit: exits gdb. - - -breakpoints ------------- - -break -sets a breakpoint -e.g. - -break main - -break *$pc - -break *0x400618 - -Here's a really useful one for large programs -rbr -Set a breakpoint for all functions matching REGEXP -e.g. -rbr 390 -will set a breakpoint with all functions with 390 in their name. - -info breakpoints -lists all breakpoints - -delete: delete breakpoint by number or delete them all -e.g. -delete 1 will delete the first breakpoint -delete will delete them all - -watch: This will set a watchpoint ( usually hardware assisted ), -This will watch a variable till it changes -e.g. -watch cnt, will watch the variable cnt till it changes. -As an aside unfortunately gdb's, architecture independent watchpoint code -is inconsistent & not very good, watchpoints usually work but not always. - -info watchpoints: Display currently active watchpoints - -condition: ( another useful one ) -Specify breakpoint number N to break only if COND is true. -Usage is `condition N COND', where N is an integer and COND is an -expression to be evaluated whenever breakpoint N is reached. - - - -User defined functions/macros ------------------------------ -define: ( Note this is very very useful,simple & powerful ) -usage define <name> <list of commands> end - -examples which you should consider putting into .gdbinit in your home directory -define d -stepi -disassemble $pc $pc+10 -end - -define e -nexti -disassemble $pc $pc+10 -end - - -Other hard to classify stuff ----------------------------- -signal n: -sends the victim program a signal. -e.g. signal 3 will send a SIGQUIT. - -info signals: -what gdb does when the victim receives certain signals. - -list: -e.g. -list lists current function source -list 1,10 list first 10 lines of current file. -list test.c:1,10 - - -directory: -Adds directories to be searched for source if gdb cannot find the source. -(note it is a bit sensitive about slashes) -e.g. To add the root of the filesystem to the searchpath do -directory // - - -call <function> -This calls a function in the victim program, this is pretty powerful -e.g. -(gdb) call printf("hello world") -outputs: -$1 = 11 - -You might now be thinking that the line above didn't work, something extra had -to be done. -(gdb) call fflush(stdout) -hello world$2 = 0 -As an aside the debugger also calls malloc & free under the hood -to make space for the "hello world" string. - - - -hints ------ -1) command completion works just like bash -( if you are a bad typist like me this really helps ) -e.g. hit br <TAB> & cursor up & down :-). - -2) if you have a debugging problem that takes a few steps to recreate -put the steps into a file called .gdbinit in your current working directory -if you have defined a few extra useful user defined commands put these in -your home directory & they will be read each time gdb is launched. - -A typical .gdbinit file might be. -break main -run -break runtime_exception -cont - - -stack chaining in gdb by hand ------------------------------ -This is done using a the same trick described for VM -p/x (*($sp+56))&0x7fffffff get the first backchain. - -For z/Architecture -Replace 56 with 112 & ignore the &0x7fffffff -in the macros below & do nasty casts to longs like the following -as gdb unfortunately deals with printed arguments as ints which -messes up everything. -i.e. here is a 3rd backchain dereference -p/x *(long *)(***(long ***)$sp+112) - - -this outputs -$5 = 0x528f18 -on my machine. -Now you can use -info symbol (*($sp+56))&0x7fffffff -you might see something like. -rl_getc + 36 in section .text telling you what is located at address 0x528f18 -Now do. -p/x (*(*$sp+56))&0x7fffffff -This outputs -$6 = 0x528ed0 -Now do. -info symbol (*(*$sp+56))&0x7fffffff -rl_read_key + 180 in section .text -now do -p/x (*(**$sp+56))&0x7fffffff -& so on. - -Disassembling instructions without debug info ---------------------------------------------- -gdb typically complains if there is a lack of debugging -symbols in the disassemble command with -"No function contains specified address." To get around -this do -x/<number lines to disassemble>xi <address> -e.g. -x/20xi 0x400730 - - - -Note: Remember gdb has history just like bash you don't need to retype the -whole line just use the up & down arrows. - - - -For more info -------------- -From your linuxbox do -man gdb or info gdb. - -core dumps ----------- -What a core dump ?, -A core dump is a file generated by the kernel (if allowed) which contains the -registers and all active pages of the program which has crashed. -From this file gdb will allow you to look at the registers, stack trace and -memory of the program as if it just crashed on your system. It is usually -called core and created in the current working directory. -This is very useful in that a customer can mail a core dump to a technical -support department and the technical support department can reconstruct what -happened. Provided they have an identical copy of this program with debugging -symbols compiled in and the source base of this build is available. -In short it is far more useful than something like a crash log could ever hope -to be. - -Why have I never seen one ?. -Probably because you haven't used the command -ulimit -c unlimited in bash -to allow core dumps, now do -ulimit -a -to verify that the limit was accepted. - -A sample core dump -To create this I'm going to do -ulimit -c unlimited -gdb -to launch gdb (my victim app. ) now be bad & do the following from another -telnet/xterm session to the same machine -ps -aux | grep gdb -kill -SIGSEGV <gdb's pid> -or alternatively use killall -SIGSEGV gdb if you have the killall command. -Now look at the core dump. -./gdb core -Displays the following -GNU gdb 4.18 -Copyright 1998 Free Software Foundation, Inc. -GDB is free software, covered by the GNU General Public License, and you are -welcome to change it and/or distribute copies of it under certain conditions. -Type "show copying" to see the conditions. -There is absolutely no warranty for GDB. Type "show warranty" for details. -This GDB was configured as "s390-ibm-linux"... -Core was generated by `./gdb'. -Program terminated with signal 11, Segmentation fault. -Reading symbols from /usr/lib/libncurses.so.4...done. -Reading symbols from /lib/libm.so.6...done. -Reading symbols from /lib/libc.so.6...done. -Reading symbols from /lib/ld-linux.so.2...done. -#0 0x40126d1a in read () from /lib/libc.so.6 -Setting up the environment for debugging gdb. -Breakpoint 1 at 0x4dc6f8: file utils.c, line 471. -Breakpoint 2 at 0x4d87a4: file top.c, line 2609. -(top-gdb) info stack -#0 0x40126d1a in read () from /lib/libc.so.6 -#1 0x528f26 in rl_getc (stream=0x7ffffde8) at input.c:402 -#2 0x528ed0 in rl_read_key () at input.c:381 -#3 0x5167e6 in readline_internal_char () at readline.c:454 -#4 0x5168ee in readline_internal_charloop () at readline.c:507 -#5 0x51692c in readline_internal () at readline.c:521 -#6 0x5164fe in readline (prompt=0x7ffff810) - at readline.c:349 -#7 0x4d7a8a in command_line_input (prompt=0x564420 "(gdb) ", repeat=1, - annotation_suffix=0x4d6b44 "prompt") at top.c:2091 -#8 0x4d6cf0 in command_loop () at top.c:1345 -#9 0x4e25bc in main (argc=1, argv=0x7ffffdf4) at main.c:635 - - -LDD -=== -This is a program which lists the shared libraries which a library needs, -Note you also get the relocations of the shared library text segments which -help when using objdump --source. -e.g. - ldd ./gdb -outputs -libncurses.so.4 => /usr/lib/libncurses.so.4 (0x40018000) -libm.so.6 => /lib/libm.so.6 (0x4005e000) -libc.so.6 => /lib/libc.so.6 (0x40084000) -/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) - - -Debugging shared libraries -========================== -Most programs use shared libraries, however it can be very painful -when you single step instruction into a function like printf for the -first time & you end up in functions like _dl_runtime_resolve this is -the ld.so doing lazy binding, lazy binding is a concept in ELF where -shared library functions are not loaded into memory unless they are -actually used, great for saving memory but a pain to debug. -To get around this either relink the program -static or exit gdb type -export LD_BIND_NOW=true this will stop lazy binding & restart the gdb'ing -the program in question. - - - -Debugging modules -================= -As modules are dynamically loaded into the kernel their address can be -anywhere to get around this use the -m option with insmod to emit a load -map which can be piped into a file if required. - -The proc file system -==================== -What is it ?. -It is a filesystem created by the kernel with files which are created on demand -by the kernel if read, or can be used to modify kernel parameters, -it is a powerful concept. - -e.g. - -cat /proc/sys/net/ipv4/ip_forward -On my machine outputs -0 -telling me ip_forwarding is not on to switch it on I can do -echo 1 > /proc/sys/net/ipv4/ip_forward -cat it again -cat /proc/sys/net/ipv4/ip_forward -On my machine now outputs -1 -IP forwarding is on. -There is a lot of useful info in here best found by going in and having a look -around, so I'll take you through some entries I consider important. - -All the processes running on the machine have their own entry defined by -/proc/<pid> -So lets have a look at the init process -cd /proc/1 - -cat cmdline -emits -init [2] - -cd /proc/1/fd -This contains numerical entries of all the open files, -some of these you can cat e.g. stdout (2) - -cat /proc/29/maps -on my machine emits - -00400000-00478000 r-xp 00000000 5f:00 4103 /bin/bash -00478000-0047e000 rw-p 00077000 5f:00 4103 /bin/bash -0047e000-00492000 rwxp 00000000 00:00 0 -40000000-40015000 r-xp 00000000 5f:00 14382 /lib/ld-2.1.2.so -40015000-40016000 rw-p 00014000 5f:00 14382 /lib/ld-2.1.2.so -40016000-40017000 rwxp 00000000 00:00 0 -40017000-40018000 rw-p 00000000 00:00 0 -40018000-4001b000 r-xp 00000000 5f:00 14435 /lib/libtermcap.so.2.0.8 -4001b000-4001c000 rw-p 00002000 5f:00 14435 /lib/libtermcap.so.2.0.8 -4001c000-4010d000 r-xp 00000000 5f:00 14387 /lib/libc-2.1.2.so -4010d000-40111000 rw-p 000f0000 5f:00 14387 /lib/libc-2.1.2.so -40111000-40114000 rw-p 00000000 00:00 0 -40114000-4011e000 r-xp 00000000 5f:00 14408 /lib/libnss_files-2.1.2.so -4011e000-4011f000 rw-p 00009000 5f:00 14408 /lib/libnss_files-2.1.2.so -7fffd000-80000000 rwxp ffffe000 00:00 0 - - -Showing us the shared libraries init uses where they are in memory -& memory access permissions for each virtual memory area. - -/proc/1/cwd is a softlink to the current working directory. -/proc/1/root is the root of the filesystem for this process. - -/proc/1/mem is the current running processes memory which you -can read & write to like a file. -strace uses this sometimes as it is a bit faster than the -rather inefficient ptrace interface for peeking at DATA. - - -cat status - -Name: init -State: S (sleeping) -Pid: 1 -PPid: 0 -Uid: 0 0 0 0 -Gid: 0 0 0 0 -Groups: -VmSize: 408 kB -VmLck: 0 kB -VmRSS: 208 kB -VmData: 24 kB -VmStk: 8 kB -VmExe: 368 kB -VmLib: 0 kB -SigPnd: 0000000000000000 -SigBlk: 0000000000000000 -SigIgn: 7fffffffd7f0d8fc -SigCgt: 00000000280b2603 -CapInh: 00000000fffffeff -CapPrm: 00000000ffffffff -CapEff: 00000000fffffeff - -User PSW: 070de000 80414146 -task: 004b6000 tss: 004b62d8 ksp: 004b7ca8 pt_regs: 004b7f68 -User GPRS: -00000400 00000000 0000000b 7ffffa90 -00000000 00000000 00000000 0045d9f4 -0045cafc 7ffffa90 7fffff18 0045cb08 -00010400 804039e8 80403af8 7ffff8b0 -User ACRS: -00000000 00000000 00000000 00000000 -00000001 00000000 00000000 00000000 -00000000 00000000 00000000 00000000 -00000000 00000000 00000000 00000000 -Kernel BackChain CallChain BackChain CallChain - 004b7ca8 8002bd0c 004b7d18 8002b92c - 004b7db8 8005cd50 004b7e38 8005d12a - 004b7f08 80019114 -Showing among other things memory usage & status of some signals & -the processes'es registers from the kernel task_structure -as well as a backchain which may be useful if a process crashes -in the kernel for some unknown reason. - -Some driver debugging techniques -================================ -debug feature -------------- -Some of our drivers now support a "debug feature" in -/proc/s390dbf see s390dbf.txt in the linux/Documentation directory -for more info. -e.g. -to switch on the lcs "debug feature" -echo 5 > /proc/s390dbf/lcs/level -& then after the error occurred. -cat /proc/s390dbf/lcs/sprintf >/logfile -the logfile now contains some information which may help -tech support resolve a problem in the field. - - - -high level debugging network drivers ------------------------------------- -ifconfig is a quite useful command -it gives the current state of network drivers. - -If you suspect your network device driver is dead -one way to check is type -ifconfig <network device> -e.g. tr0 -You should see something like -tr0 Link encap:16/4 Mbps Token Ring (New) HWaddr 00:04:AC:20:8E:48 - inet addr:9.164.185.132 Bcast:9.164.191.255 Mask:255.255.224.0 - UP BROADCAST RUNNING MULTICAST MTU:2000 Metric:1 - RX packets:246134 errors:0 dropped:0 overruns:0 frame:0 - TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 - collisions:0 txqueuelen:100 - -if the device doesn't say up -try -/etc/rc.d/init.d/network start -( this starts the network stack & hopefully calls ifconfig tr0 up ). -ifconfig looks at the output of /proc/net/dev and presents it in a more -presentable form. -Now ping the device from a machine in the same subnet. -if the RX packets count & TX packets counts don't increment you probably -have problems. -next -cat /proc/net/arp -Do you see any hardware addresses in the cache if not you may have problems. -Next try -ping -c 5 <broadcast_addr> i.e. the Bcast field above in the output of -ifconfig. Do you see any replies from machines other than the local machine -if not you may have problems. also if the TX packets count in ifconfig -hasn't incremented either you have serious problems in your driver -(e.g. the txbusy field of the network device being stuck on ) -or you may have multiple network devices connected. - - -chandev -------- -There is a new device layer for channel devices, some -drivers e.g. lcs are registered with this layer. -If the device uses the channel device layer you'll be -able to find what interrupts it uses & the current state -of the device. -See the manpage chandev.8 &type cat /proc/chandev for more info. - - -SysRq -===== -This is now supported by linux for s/390 & z/Architecture. -To enable it do compile the kernel with -Kernel Hacking -> Magic SysRq Key Enabled -echo "1" > /proc/sys/kernel/sysrq -also type -echo "8" >/proc/sys/kernel/printk -To make printk output go to console. -On 390 all commands are prefixed with -^- -e.g. -^-t will show tasks. -^-? or some unknown command will display help. -The sysrq key reading is very picky ( I have to type the keys in an - xterm session & paste them into the x3270 console ) -& it may be wise to predefine the keys as described in the VM hints above - -This is particularly useful for syncing disks unmounting & rebooting -if the machine gets partially hung. - -Read Documentation/admin-guide/sysrq.rst for more info - -References: -=========== -Enterprise Systems Architecture Reference Summary -Enterprise Systems Architecture Principles of Operation -Hartmut Penners s390 stack frame sheet. -IBM Mainframe Channel Attachment a technology brief from a CISCO webpage -Various bits of man & info pages of Linux. -Linux & GDB source. -Various info & man pages. -CMS Help on tracing commands. -Linux for s/390 Elf Application Binary Interface -Linux for z/Series Elf Application Binary Interface ( Both Highly Recommended ) -z/Architecture Principles of Operation SA22-7832-00 -Enterprise Systems Architecture/390 Reference Summary SA22-7209-01 & the -Enterprise Systems Architecture/390 Principles of Operation SA22-7201-05 - -Special Thanks -============== -Special thanks to Neale Ferguson who maintains a much -prettier HTML version of this page at -http://linuxvm.org/penguinvm/ -Bob Grainger Stefan Bader & others for reporting bugs diff --git a/Documentation/s390/cds.txt b/Documentation/s390/cds.rst index 480a78ef5a1e..7006d8209d2e 100644 --- a/Documentation/s390/cds.txt +++ b/Documentation/s390/cds.rst @@ -1,14 +1,18 @@ +=========================== Linux for S/390 and zSeries +=========================== Common Device Support (CDS) Device Driver I/O Support Routines -Authors : Ingo Adlung - Cornelia Huck +Authors: + - Ingo Adlung + - Cornelia Huck Copyright, IBM Corp. 1999-2002 Introduction +============ This document describes the common device support routines for Linux/390. Different than other hardware architectures, ESA/390 has defined a unified @@ -27,18 +31,20 @@ Operation manual (IBM Form. No. SA22-7201). In order to build common device support for ESA/390 I/O interfaces, a functional layer was introduced that provides generic I/O access methods to -the hardware. +the hardware. -The common device support layer comprises the I/O support routines defined -below. Some of them implement common Linux device driver interfaces, while +The common device support layer comprises the I/O support routines defined +below. Some of them implement common Linux device driver interfaces, while some of them are ESA/390 platform specific. Note: -In order to write a driver for S/390, you also need to look into the interface -described in Documentation/s390/driver-model.txt. + In order to write a driver for S/390, you also need to look into the interface + described in Documentation/s390/driver-model.rst. Note for porting drivers from 2.4: + The major changes are: + * The functions use a ccw_device instead of an irq (subchannel). * All drivers must define a ccw_driver (see driver-model.txt) and the associated functions. @@ -57,19 +63,16 @@ The major changes are: ccw_device_get_ciw() get commands from extended sense data. -ccw_device_start() -ccw_device_start_timeout() -ccw_device_start_key() -ccw_device_start_key_timeout() +ccw_device_start(), ccw_device_start_timeout(), ccw_device_start_key(), ccw_device_start_key_timeout() initiate an I/O request. ccw_device_resume() resume channel program execution. -ccw_device_halt() +ccw_device_halt() terminate the current I/O request processed on the device. -do_IRQ() +do_IRQ() generic interrupt routine. This function is called by the interrupt entry routine whenever an I/O interrupt is presented to the system. The do_IRQ() routine determines the interrupt status and calls the device specific @@ -82,12 +85,15 @@ first level interrupt handler only and does not comprise a device driver callable interface. Instead, the functional description of do_IO() also describes the input to the device specific interrupt handler. -Note: All explanations apply also to the 64 bit architecture s390x. +Note: + All explanations apply also to the 64 bit architecture s390x. Common Device Support (CDS) for Linux/390 Device Drivers +======================================================== General Information +------------------- The following chapters describe the I/O related interface routines the Linux/390 common device support (CDS) provides to allow for device specific @@ -101,6 +107,7 @@ can be found in the architecture specific C header file linux/arch/s390/include/asm/irq.h. Overview of CDS interface concepts +---------------------------------- Different to other hardware platforms, the ESA/390 architecture doesn't define interrupt lines managed by a specific interrupt controller and bus systems @@ -126,7 +133,7 @@ has to call every single device driver registered on this IRQ in order to determine the device driver owning the device that raised the interrupt. Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel). -For internal use of the common I/O layer, these are still there. However, +For internal use of the common I/O layer, these are still there. However, device drivers should use the new calling interface via the ccw_device only. During its startup the Linux/390 system checks for peripheral devices. Each @@ -134,7 +141,7 @@ of those devices is uniquely defined by a so called subchannel by the ESA/390 channel subsystem. While the subchannel numbers are system generated, each subchannel also takes a user defined attribute, the so called device number. Both subchannel number and device number cannot exceed 65535. During sysfs -initialisation, the information about control unit type and device types that +initialisation, the information about control unit type and device types that imply specific I/O commands (channel command words - CCWs) in order to operate the device are gathered. Device drivers can retrieve this set of hardware information during their initialization step to recognize the devices they @@ -164,18 +171,26 @@ get_ciw() - get command information word This call enables a device driver to get information about supported commands from the extended SenseID data. -struct ciw * -ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd); +:: -cdev - The ccw_device for which the command is to be retrieved. -cmd - The command type to be retrieved. + struct ciw * + ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd); + +==== ======================================================== +cdev The ccw_device for which the command is to be retrieved. +cmd The command type to be retrieved. +==== ======================================================== ccw_device_get_ciw() returns: -NULL - No extended data available, invalid device or command not found. -!NULL - The command requested. +===== ================================================================ + NULL No extended data available, invalid device or command not found. +!NULL The command requested. +===== ================================================================ + +:: -ccw_device_start() - Initiate I/O Request + ccw_device_start() - Initiate I/O Request The ccw_device_start() routines is the I/O request front-end processor. All device driver I/O requests must be issued using this routine. A device driver @@ -186,93 +201,105 @@ This description also covers the status information passed to the device driver's interrupt handler as this is related to the rules (flags) defined with the associated I/O request when calling ccw_device_start(). -int ccw_device_start(struct ccw_device *cdev, - struct ccw1 *cpa, - unsigned long intparm, - __u8 lpm, - unsigned long flags); -int ccw_device_start_timeout(struct ccw_device *cdev, - struct ccw1 *cpa, - unsigned long intparm, - __u8 lpm, - unsigned long flags, - int expires); -int ccw_device_start_key(struct ccw_device *cdev, - struct ccw1 *cpa, - unsigned long intparm, - __u8 lpm, - __u8 key, - unsigned long flags); -int ccw_device_start_key_timeout(struct ccw_device *cdev, - struct ccw1 *cpa, - unsigned long intparm, - __u8 lpm, - __u8 key, - unsigned long flags, - int expires); - -cdev : ccw_device the I/O is destined for -cpa : logical start address of channel program -user_intparm : user specific interrupt information; will be presented - back to the device driver's interrupt handler. Allows a - device driver to associate the interrupt with a - particular I/O request. -lpm : defines the channel path to be used for a specific I/O - request. A value of 0 will make cio use the opm. -key : the storage key to use for the I/O (useful for operating on a - storage with a storage key != default key) -flag : defines the action to be performed for I/O processing -expires : timeout value in jiffies. The common I/O layer will terminate - the running program after this and call the interrupt handler - with ERR_PTR(-ETIMEDOUT) as irb. - -Possible flag values are : - -DOIO_ALLOW_SUSPEND - channel program may become suspended -DOIO_DENY_PREFETCH - don't allow for CCW prefetch; usually - this implies the channel program might - become modified -DOIO_SUPPRESS_INTER - don't call the handler on intermediate status - -The cpa parameter points to the first format 1 CCW of a channel program : - -struct ccw1 { - __u8 cmd_code;/* command code */ - __u8 flags; /* flags, like IDA addressing, etc. */ - __u16 count; /* byte count */ - __u32 cda; /* data address */ -} __attribute__ ((packed,aligned(8))); - -with the following CCW flags values defined : - -CCW_FLAG_DC - data chaining -CCW_FLAG_CC - command chaining -CCW_FLAG_SLI - suppress incorrect length -CCW_FLAG_SKIP - skip -CCW_FLAG_PCI - PCI -CCW_FLAG_IDA - indirect addressing -CCW_FLAG_SUSPEND - suspend +:: + + int ccw_device_start(struct ccw_device *cdev, + struct ccw1 *cpa, + unsigned long intparm, + __u8 lpm, + unsigned long flags); + int ccw_device_start_timeout(struct ccw_device *cdev, + struct ccw1 *cpa, + unsigned long intparm, + __u8 lpm, + unsigned long flags, + int expires); + int ccw_device_start_key(struct ccw_device *cdev, + struct ccw1 *cpa, + unsigned long intparm, + __u8 lpm, + __u8 key, + unsigned long flags); + int ccw_device_start_key_timeout(struct ccw_device *cdev, + struct ccw1 *cpa, + unsigned long intparm, + __u8 lpm, + __u8 key, + unsigned long flags, + int expires); + +============= ============================================================= +cdev ccw_device the I/O is destined for +cpa logical start address of channel program +user_intparm user specific interrupt information; will be presented + back to the device driver's interrupt handler. Allows a + device driver to associate the interrupt with a + particular I/O request. +lpm defines the channel path to be used for a specific I/O + request. A value of 0 will make cio use the opm. +key the storage key to use for the I/O (useful for operating on a + storage with a storage key != default key) +flag defines the action to be performed for I/O processing +expires timeout value in jiffies. The common I/O layer will terminate + the running program after this and call the interrupt handler + with ERR_PTR(-ETIMEDOUT) as irb. +============= ============================================================= + +Possible flag values are: + +========================= ============================================= +DOIO_ALLOW_SUSPEND channel program may become suspended +DOIO_DENY_PREFETCH don't allow for CCW prefetch; usually + this implies the channel program might + become modified +DOIO_SUPPRESS_INTER don't call the handler on intermediate status +========================= ============================================= + +The cpa parameter points to the first format 1 CCW of a channel program:: + + struct ccw1 { + __u8 cmd_code;/* command code */ + __u8 flags; /* flags, like IDA addressing, etc. */ + __u16 count; /* byte count */ + __u32 cda; /* data address */ + } __attribute__ ((packed,aligned(8))); + +with the following CCW flags values defined: + +=================== ========================= +CCW_FLAG_DC data chaining +CCW_FLAG_CC command chaining +CCW_FLAG_SLI suppress incorrect length +CCW_FLAG_SKIP skip +CCW_FLAG_PCI PCI +CCW_FLAG_IDA indirect addressing +CCW_FLAG_SUSPEND suspend +=================== ========================= Via ccw_device_set_options(), the device driver may specify the following options for the device: -DOIO_EARLY_NOTIFICATION - allow for early interrupt notification -DOIO_REPORT_ALL - report all interrupt conditions +========================= ====================================== +DOIO_EARLY_NOTIFICATION allow for early interrupt notification +DOIO_REPORT_ALL report all interrupt conditions +========================= ====================================== -The ccw_device_start() function returns : +The ccw_device_start() function returns: - 0 - successful completion or request successfully initiated --EBUSY - The device is currently processing a previous I/O request, or there is - a status pending at the device. --ENODEV - cdev is invalid, the device is not operational or the ccw_device is - not online. +======== ====================================================================== + 0 successful completion or request successfully initiated + -EBUSY The device is currently processing a previous I/O request, or there is + a status pending at the device. +-ENODEV cdev is invalid, the device is not operational or the ccw_device is + not online. +======== ====================================================================== When the I/O request completes, the CDS first level interrupt handler will accumulate the status in a struct irb and then call the device interrupt handler. -The intparm field will contain the value the device driver has associated with a -particular I/O request. If a pending device status was recognized, +The intparm field will contain the value the device driver has associated with a +particular I/O request. If a pending device status was recognized, intparm will be set to 0 (zero). This may happen during I/O initiation or delayed by an alert status notification. In any case this status is not related to the current (last) I/O request. In case of a delayed status notification no special @@ -282,9 +309,11 @@ never started, even though ccw_device_start() returned with successful completio The irb may contain an error value, and the device driver should check for this first: --ETIMEDOUT: the common I/O layer terminated the request after the specified - timeout value --EIO: the common I/O layer terminated the request due to an error state +========== ================================================================= +-ETIMEDOUT the common I/O layer terminated the request after the specified + timeout value +-EIO the common I/O layer terminated the request due to an error state +========== ================================================================= If the concurrent sense flag in the extended status word (esw) in the irb is set, the field erw.scnt in the esw describes the number of device specific @@ -294,6 +323,7 @@ sensing by the device driver itself is required. The device interrupt handler can use the following definitions to investigate the primary unit check source coded in sense byte 0 : +======================= ==== SNS0_CMD_REJECT 0x80 SNS0_INTERVENTION_REQ 0x40 SNS0_BUS_OUT_CHECK 0x20 @@ -301,36 +331,41 @@ SNS0_EQUIPMENT_CHECK 0x10 SNS0_DATA_CHECK 0x08 SNS0_OVERRUN 0x04 SNS0_INCOMPL_DOMAIN 0x01 +======================= ==== Depending on the device status, multiple of those values may be set together. Please refer to the device specific documentation for details. The irb->scsw.cstat field provides the (accumulated) subchannel status : -SCHN_STAT_PCI - program controlled interrupt -SCHN_STAT_INCORR_LEN - incorrect length -SCHN_STAT_PROG_CHECK - program check -SCHN_STAT_PROT_CHECK - protection check -SCHN_STAT_CHN_DATA_CHK - channel data check -SCHN_STAT_CHN_CTRL_CHK - channel control check -SCHN_STAT_INTF_CTRL_CHK - interface control check -SCHN_STAT_CHAIN_CHECK - chaining check +========================= ============================ +SCHN_STAT_PCI program controlled interrupt +SCHN_STAT_INCORR_LEN incorrect length +SCHN_STAT_PROG_CHECK program check +SCHN_STAT_PROT_CHECK protection check +SCHN_STAT_CHN_DATA_CHK channel data check +SCHN_STAT_CHN_CTRL_CHK channel control check +SCHN_STAT_INTF_CTRL_CHK interface control check +SCHN_STAT_CHAIN_CHECK chaining check +========================= ============================ The irb->scsw.dstat field provides the (accumulated) device status : -DEV_STAT_ATTENTION - attention -DEV_STAT_STAT_MOD - status modifier -DEV_STAT_CU_END - control unit end -DEV_STAT_BUSY - busy -DEV_STAT_CHN_END - channel end -DEV_STAT_DEV_END - device end -DEV_STAT_UNIT_CHECK - unit check -DEV_STAT_UNIT_EXCEP - unit exception +===================== ================= +DEV_STAT_ATTENTION attention +DEV_STAT_STAT_MOD status modifier +DEV_STAT_CU_END control unit end +DEV_STAT_BUSY busy +DEV_STAT_CHN_END channel end +DEV_STAT_DEV_END device end +DEV_STAT_UNIT_CHECK unit check +DEV_STAT_UNIT_EXCEP unit exception +===================== ================= Please see the ESA/390 Principles of Operation manual for details on the individual flag meanings. -Usage Notes : +Usage Notes: ccw_device_start() must be called disabled and with the ccw device lock held. @@ -374,32 +409,39 @@ secondary status without error (alert status) is presented, this indicates successful completion for all overlapping ccw_device_start() requests that have been issued since the last secondary (final) status. -Channel programs that intend to set the suspend flag on a channel command word -(CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the -suspend flag will cause a channel program check. At the time the channel program -becomes suspended an intermediate interrupt will be generated by the channel +Channel programs that intend to set the suspend flag on a channel command word +(CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the +suspend flag will cause a channel program check. At the time the channel program +becomes suspended an intermediate interrupt will be generated by the channel subsystem. -ccw_device_resume() - Resume Channel Program Execution +ccw_device_resume() - Resume Channel Program Execution -If a device driver chooses to suspend the current channel program execution by -setting the CCW suspend flag on a particular CCW, the channel program execution -is suspended. In order to resume channel program execution the CIO layer -provides the ccw_device_resume() routine. +If a device driver chooses to suspend the current channel program execution by +setting the CCW suspend flag on a particular CCW, the channel program execution +is suspended. In order to resume channel program execution the CIO layer +provides the ccw_device_resume() routine. -int ccw_device_resume(struct ccw_device *cdev); +:: -cdev - ccw_device the resume operation is requested for + int ccw_device_resume(struct ccw_device *cdev); + +==== ================================================ +cdev ccw_device the resume operation is requested for +==== ================================================ The ccw_device_resume() function returns: - 0 - suspended channel program is resumed --EBUSY - status pending --ENODEV - cdev invalid or not-operational subchannel --EINVAL - resume function not applicable --ENOTCONN - there is no I/O request pending for completion +========= ============================================== + 0 suspended channel program is resumed + -EBUSY status pending + -ENODEV cdev invalid or not-operational subchannel + -EINVAL resume function not applicable +-ENOTCONN there is no I/O request pending for completion +========= ============================================== Usage Notes: + Please have a look at the ccw_device_start() usage notes for more details on suspended channel programs. @@ -412,22 +454,28 @@ command is provided. ccw_device_halt() must be called disabled and with the ccw device lock held. -int ccw_device_halt(struct ccw_device *cdev, - unsigned long intparm); +:: + + int ccw_device_halt(struct ccw_device *cdev, + unsigned long intparm); -cdev : ccw_device the halt operation is requested for -intparm : interruption parameter; value is only used if no I/O - is outstanding, otherwise the intparm associated with - the I/O request is returned +======= ===================================================== +cdev ccw_device the halt operation is requested for +intparm interruption parameter; value is only used if no I/O + is outstanding, otherwise the intparm associated with + the I/O request is returned +======= ===================================================== -The ccw_device_halt() function returns : +The ccw_device_halt() function returns: - 0 - request successfully initiated --EBUSY - the device is currently busy, or status pending. --ENODEV - cdev invalid. --EINVAL - The device is not operational or the ccw device is not online. +======= ============================================================== + 0 request successfully initiated +-EBUSY the device is currently busy, or status pending. +-ENODEV cdev invalid. +-EINVAL The device is not operational or the ccw device is not online. +======= ============================================================== -Usage Notes : +Usage Notes: A device driver may write a never-ending channel program by writing a channel program that at its end loops back to its beginning by means of a transfer in @@ -438,25 +486,34 @@ can then perform an appropriate action. Prior to interrupt of an outstanding read to a network device (with or without PCI flag) a ccw_device_halt() is required to end the pending operation. -ccw_device_clear() - Terminage I/O Request Processing +:: + + ccw_device_clear() - Terminage I/O Request Processing In order to terminate all I/O processing at the subchannel, the clear subchannel (CSCH) command is used. It can be issued via ccw_device_clear(). ccw_device_clear() must be called disabled and with the ccw device lock held. -int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm); +:: + + int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm); -cdev: ccw_device the clear operation is requested for -intparm: interruption parameter (see ccw_device_halt()) +======= =============================================== +cdev ccw_device the clear operation is requested for +intparm interruption parameter (see ccw_device_halt()) +======= =============================================== The ccw_device_clear() function returns: - 0 - request successfully initiated --ENODEV - cdev invalid --EINVAL - The device is not operational or the ccw device is not online. +======= ============================================================== + 0 request successfully initiated +-ENODEV cdev invalid +-EINVAL The device is not operational or the ccw device is not online. +======= ============================================================== Miscellaneous Support Routines +------------------------------ This chapter describes various routines to be used in a Linux/390 device driver programming environment. @@ -466,7 +523,8 @@ get_ccwdev_lock() Get the address of the device specific lock. This is then used in spin_lock() / spin_unlock() calls. +:: -__u8 ccw_device_get_path_mask(struct ccw_device *cdev); + __u8 ccw_device_get_path_mask(struct ccw_device *cdev); Get the mask of the path currently available for cdev. diff --git a/Documentation/s390/CommonIO b/Documentation/s390/common_io.rst index 6e0f63f343b4..846485681ce7 100644 --- a/Documentation/s390/CommonIO +++ b/Documentation/s390/common_io.rst @@ -1,5 +1,9 @@ -S/390 common I/O-Layer - command line parameters, procfs and debugfs entries -============================================================================ +====================== +S/390 common I/O-Layer +====================== + +command line parameters, procfs and debugfs entries +=================================================== Command line parameters ----------------------- @@ -13,7 +17,7 @@ Command line parameters device := {all | [!]ipldev | [!]condev | [!]<devno> | [!]<devno>-<devno>} The given devices will be ignored by the common I/O-layer; no detection - and device sensing will be done on any of those devices. The subchannel to + and device sensing will be done on any of those devices. The subchannel to which the device in question is attached will be treated as if no device was attached. @@ -28,14 +32,20 @@ Command line parameters keywords can be used to refer to the CCW based boot device and CCW console device respectively (these are probably useful only when combined with the '!' operator). The '!' operator will cause the I/O-layer to _not_ ignore a device. - The command line is parsed from left to right. + The command line + is parsed from left to right. + + For example:: - For example, cio_ignore=0.0.0023-0.0.0042,0.0.4711 + will ignore all devices ranging from 0.0.0023 to 0.0.0042 and the device 0.0.4711, if detected. - As another example, + + As another example:: + cio_ignore=all,!0.0.4711,!0.0.fd00-0.0.fd02 + will ignore all devices but 0.0.4711, 0.0.fd00, 0.0.fd01, 0.0.fd02. By default, no devices are ignored. @@ -48,40 +58,45 @@ Command line parameters Lists the ranges of devices (by bus id) which are ignored by common I/O. - You can un-ignore certain or all devices by piping to /proc/cio_ignore. - "free all" will un-ignore all ignored devices, + You can un-ignore certain or all devices by piping to /proc/cio_ignore. + "free all" will un-ignore all ignored devices, "free <device range>, <device range>, ..." will un-ignore the specified devices. For example, if devices 0.0.0023 to 0.0.0042 and 0.0.4711 are ignored, + - echo free 0.0.0030-0.0.0032 > /proc/cio_ignore will un-ignore devices 0.0.0030 to 0.0.0032 and will leave devices 0.0.0023 to 0.0.002f, 0.0.0033 to 0.0.0042 and 0.0.4711 ignored; - echo free 0.0.0041 > /proc/cio_ignore will furthermore un-ignore device 0.0.0041; - - echo free all > /proc/cio_ignore will un-ignore all remaining ignored + - echo free all > /proc/cio_ignore will un-ignore all remaining ignored devices. - When a device is un-ignored, device recognition and sensing is performed and + When a device is un-ignored, device recognition and sensing is performed and the device driver will be notified if possible, so the device will become available to the system. Note that un-ignoring is performed asynchronously. - You can also add ranges of devices to be ignored by piping to + You can also add ranges of devices to be ignored by piping to /proc/cio_ignore; "add <device range>, <device range>, ..." will ignore the specified devices. Note: While already known devices can be added to the list of devices to be - ignored, there will be no effect on then. However, if such a device + ignored, there will be no effect on then. However, if such a device disappears and then reappears, it will then be ignored. To make known devices go away, you need the "purge" command (see below). - For example, + For example:: + "echo add 0.0.a000-0.0.accc, 0.0.af00-0.0.afff > /proc/cio_ignore" + will add 0.0.a000-0.0.accc and 0.0.af00-0.0.afff to the list of ignored devices. - You can remove already known but now ignored devices via + You can remove already known but now ignored devices via:: + "echo purge > /proc/cio_ignore" + All devices ignored but still registered and not online (= not in use) will be deregistered and thus removed from the system. @@ -115,11 +130,11 @@ debugfs entries Various debug messages from the common I/O-layer. - /sys/kernel/debug/s390dbf/cio_trace/hex_ascii - Logs the calling of functions in the common I/O-layer and, if applicable, + Logs the calling of functions in the common I/O-layer and, if applicable, which subchannel they were called for, as well as dumps of some data structures (like irb in an error case). - The level of logging can be changed to be more or less verbose by piping to + The level of logging can be changed to be more or less verbose by piping to /sys/kernel/debug/s390dbf/cio_*/level a number between 0 and 6; see the - documentation on the S/390 debug feature (Documentation/s390/s390dbf.txt) + documentation on the S/390 debug feature (Documentation/s390/s390dbf.rst) for details. diff --git a/Documentation/s390/DASD b/Documentation/s390/dasd.rst index 9963f1e9c98a..9e22247285c8 100644 --- a/Documentation/s390/DASD +++ b/Documentation/s390/dasd.rst @@ -1,4 +1,6 @@ +================== DASD device driver +================== S/390's disk devices (DASDs) are managed by Linux via the DASD device driver. It is valid for all types of DASDs and represents them to @@ -14,14 +16,14 @@ parameters are to be given in hexadecimal notation without a leading If you supply kernel parameters the different instances are processed in order of appearance and a minor number is reserved for any device covered by the supplied range up to 64 volumes. Additional DASDs are -ignored. If you do not supply the 'dasd=' kernel parameter at all, the +ignored. If you do not supply the 'dasd=' kernel parameter at all, the DASD driver registers all supported DASDs of your system to a minor number in ascending order of the subchannel number. The driver currently supports ECKD-devices and there are stubs for support of the FBA and CKD architectures. For the FBA architecture only some smart data structures are missing to make the support -complete. +complete. We performed our testing on 3380 and 3390 type disks of different sizes, under VM and on the bare hardware (LPAR), using internal disks of the multiprise as well as a RAMAC virtual array. Disks exported by @@ -34,19 +36,22 @@ accessibility of the DASD from other OSs. In a later stage we will provide support of partitions, maybe VTOC oriented or using a kind of partition table in the label record. -USAGE +Usage +===== -Low-level format (?CKD only) For using an ECKD-DASD as a Linux harddisk you have to low-level format the tracks by issuing the BLKDASDFORMAT-ioctl on that device. This will erase any data on that volume including IBM volume -labels, VTOCs etc. The ioctl may take a 'struct format_data *' or -'NULL' as an argument. -typedef struct { +labels, VTOCs etc. The ioctl may take a `struct format_data *` or +'NULL' as an argument:: + + typedef struct { int start_unit; int stop_unit; int blksize; -} format_data_t; + } format_data_t; + When a NULL argument is passed to the BLKDASDFORMAT ioctl the whole disk is formatted to a blocksize of 1024 bytes. Otherwise start_unit and stop_unit are the first and last track to be formatted. If @@ -56,17 +61,23 @@ up to the last track. blksize can be any power of two between 512 and 1kB blocks anyway and you gain approx. 50% of capacity increasing your blksize from 512 byte to 1kB. --Make a filesystem +Make a filesystem +================= + Then you can mk??fs the filesystem of your choice on that volume or partition. For reasons of sanity you should build your filesystem on -the partition /dev/dd?1 instead of the whole volume. You only lose 3kB +the partition /dev/dd?1 instead of the whole volume. You only lose 3kB but may be sure that you can reuse your data after introduction of a real partition table. -BUGS: +Bugs +==== + - Performance sometimes is rather low because we don't fully exploit clustering -TODO-List: +TODO-List +========= + - Add IBM'S Disk layout to genhd - Enhance driver to use more than one major number - Enable usage as a module diff --git a/Documentation/s390/debugging390.rst b/Documentation/s390/debugging390.rst new file mode 100644 index 000000000000..d49305fd5e1a --- /dev/null +++ b/Documentation/s390/debugging390.rst @@ -0,0 +1,2613 @@ +============================================= +Debugging on Linux for s/390 & z/Architecture +============================================= + +Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com) + +Copyright (C) 2000-2001 IBM Deutschland Entwicklung GmbH, IBM Corporation + +.. Best viewed with fixed width fonts + +Overview of Document: +===================== +This document is intended to give a good overview of how to debug Linux for +s/390 and z/Architecture. It is not intended as a complete reference and not a +tutorial on the fundamentals of C & assembly. It doesn't go into +390 IO in any detail. It is intended to complement the documents in the +reference section below & any other worthwhile references you get. + +It is intended like the Enterprise Systems Architecture/390 Reference Summary +to be printed out & used as a quick cheat sheet self help style reference when +problems occur. + +.. Contents + ======== + Register Set + Address Spaces on Intel Linux + Address Spaces on Linux for s/390 & z/Architecture + The Linux for s/390 & z/Architecture Kernel Task Structure + Register Usage & Stackframes on Linux for s/390 & z/Architecture + A sample program with comments + Compiling programs for debugging on Linux for s/390 & z/Architecture + Debugging under VM + s/390 & z/Architecture IO Overview + Debugging IO on s/390 & z/Architecture under VM + GDB on s/390 & z/Architecture + Stack chaining in gdb by hand + Examining core dumps + ldd + Debugging modules + The proc file system + SysRq + References + Special Thanks + +Register Set +============ +The current architectures have the following registers. + +16 General propose registers, 32 bit on s/390 and 64 bit on z/Architecture, +r0-r15 (or gpr0-gpr15), used for arithmetic and addressing. + +16 Control registers, 32 bit on s/390 and 64 bit on z/Architecture, cr0-cr15, +kernel usage only, used for memory management, interrupt control, debugging +control etc. + +16 Access registers (ar0-ar15), 32 bit on both s/390 and z/Architecture, +normally not used by normal programs but potentially could be used as +temporary storage. These registers have a 1:1 association with general +purpose registers and are designed to be used in the so-called access +register mode to select different address spaces. +Access register 0 (and access register 1 on z/Architecture, which needs a +64 bit pointer) is currently used by the pthread library as a pointer to +the current running threads private area. + +16 64-bit floating point registers (fp0-fp15 ) IEEE & HFP floating +point format compliant on G5 upwards & a Floating point control reg (FPC) + +4 64-bit registers (fp0,fp2,fp4 & fp6) HFP only on older machines. + +Note: + Linux (currently) always uses IEEE & emulates G5 IEEE format on older + machines, ( provided the kernel is configured for this ). + + +The PSW is the most important register on the machine it +is 64 bit on s/390 & 128 bit on z/Architecture & serves the roles of +a program counter (pc), condition code register,memory space designator. +In IBM standard notation I am counting bit 0 as the MSB. +It has several advantages over a normal program counter +in that you can change address translation & program counter +in a single instruction. To change address translation, +e.g. switching address translation off requires that you +have a logical=physical mapping for the address you are +currently running at. + ++-------------------------+-------------------------------------------------+ +| Bit | | ++--------+----------------+ Value | +| s/390 | z/Architecture | | ++========+================+=================================================+ +| 0 | 0 | Reserved (must be 0) otherwise specification | +| | | exception occurs. | ++--------+----------------+-------------------------------------------------+ +| 1 | 1 | Program Event Recording 1 PER enabled, | +| | | PER is used to facilitate debugging e.g. | +| | | single stepping. | ++--------+----------------+-------------------------------------------------+ +| 2-4 | 2-4 | Reserved (must be 0). | ++--------+----------------+-------------------------------------------------+ +| 5 | 5 | Dynamic address translation 1=DAT on. | ++--------+----------------+-------------------------------------------------+ +| 6 | 6 | Input/Output interrupt Mask | ++--------+----------------+-------------------------------------------------+ +| 7 | 7 | External interrupt Mask used primarily for | +| | | interprocessor signalling and clock interrupts. | ++--------+----------------+-------------------------------------------------+ +| 8-11 | 8-11 | PSW Key used for complex memory protection | +| | | mechanism (not used under linux) | ++--------+----------------+-------------------------------------------------+ +| 12 | 12 | 1 on s/390 0 on z/Architecture | ++--------+----------------+-------------------------------------------------+ +| 13 | 13 | Machine Check Mask 1=enable machine check | +| | | interrupts | ++--------+----------------+-------------------------------------------------+ +| 14 | 14 | Wait State. Set this to 1 to stop the processor | +| | | except for interrupts and give time to other | +| | | LPARS. Used in CPU idle in the kernel to | +| | | increase overall usage of processor resources. | ++--------+----------------+-------------------------------------------------+ +| 15 | 15 | Problem state (if set to 1 certain instructions | +| | | are disabled). All linux user programs run with | +| | | this bit 1 (useful info for debugging under VM).| ++--------+----------------+-------------------------------------------------+ +| 16-17 | 16-17 | Address Space Control | +| | | | +| | | 00 Primary Space Mode: | +| | | | +| | | The register CR1 contains the primary | +| | | address-space control element (PASCE), which | +| | | points to the primary space region/segment | +| | | table origin. | +| | | | +| | | 01 Access register mode | +| | | | +| | | 10 Secondary Space Mode: | +| | | | +| | | The register CR7 contains the secondary | +| | | address-space control element (SASCE), which | +| | | points to the secondary space region or | +| | | segment table origin. | +| | | | +| | | 11 Home Space Mode: | +| | | | +| | | The register CR13 contains the home space | +| | | address-space control element (HASCE), which | +| | | points to the home space region/segment | +| | | table origin. | +| | | | +| | | See "Address Spaces on Linux for s/390 & | +| | | z/Architecture" below for more information | +| | | about address space usage in Linux. | ++--------+----------------+-------------------------------------------------+ +| 18-19 | 18-19 | Condition codes (CC) | ++--------+----------------+-------------------------------------------------+ +| 20 | 20 | Fixed point overflow mask if 1=FPU exceptions | +| | | for this event occur (normally 0) | ++--------+----------------+-------------------------------------------------+ +| 21 | 21 | Decimal overflow mask if 1=FPU exceptions for | +| | | this event occur (normally 0) | ++--------+----------------+-------------------------------------------------+ +| 22 | 22 | Exponent underflow mask if 1=FPU exceptions | +| | | for this event occur (normally 0) | ++--------+----------------+-------------------------------------------------+ +| 23 | 23 | Significance Mask if 1=FPU exceptions for this | +| | | event occur (normally 0) | ++--------+----------------+-------------------------------------------------+ +| 24-31 | 24-30 | Reserved Must be 0. | +| +----------------+-------------------------------------------------+ +| | 31 | Extended Addressing Mode | +| +----------------+-------------------------------------------------+ +| | 32 | Basic Addressing Mode | +| | | | +| | | Used to set addressing mode | +| | | | +| | | +---------+----------+----------+ | +| | | | PSW 31 | PSW 32 | | | +| | | +---------+----------+----------+ | +| | | | 0 | 0 | 24 bit | | +| | | +---------+----------+----------+ | +| | | | 0 | 1 | 31 bit | | +| | | +---------+----------+----------+ | +| | | | 1 | 1 | 64 bit | | +| | | +---------+----------+----------+ | ++--------+----------------+-------------------------------------------------+ +| 32 | | 1=31 bit addressing mode 0=24 bit addressing | +| | | mode (for backward compatibility), linux | +| | | always runs with this bit set to 1 | ++--------+----------------+-------------------------------------------------+ +| 33-64 | | Instruction address. | +| +----------------+-------------------------------------------------+ +| | 33-63 | Reserved must be 0 | +| +----------------+-------------------------------------------------+ +| | 64-127 | Address | +| | | | +| | | - In 24 bits mode bits 64-103=0 bits 104-127 | +| | | Address | +| | | - In 31 bits mode bits 64-96=0 bits 97-127 | +| | | Address | +| | | | +| | | Note: | +| | | unlike 31 bit mode on s/390 bit 96 must be | +| | | zero when loading the address with LPSWE | +| | | otherwise a specification exception occurs, | +| | | LPSW is fully backward compatible. | ++--------+----------------+-------------------------------------------------+ + +Prefix Page(s) +-------------- +This per cpu memory area is too intimately tied to the processor not to mention. +It exists between the real addresses 0-4096 on s/390 and between 0-8192 on +z/Architecture and is exchanged with one page on s/390 or two pages on +z/Architecture in absolute storage by the set prefix instruction during Linux +startup. + +This page is mapped to a different prefix for each processor in an SMP +configuration (assuming the OS designer is sane of course). + +Bytes 0-512 (200 hex) on s/390 and 0-512, 4096-4544, 4604-5119 currently on +z/Architecture are used by the processor itself for holding such information +as exception indications and entry points for exceptions. + +Bytes after 0xc00 hex are used by linux for per processor globals on s/390 and +z/Architecture (there is a gap on z/Architecture currently between 0xc00 and +0x1000, too, which is used by Linux). + +The closest thing to this on traditional architectures is the interrupt +vector table. This is a good thing & does simplify some of the kernel coding +however it means that we now cannot catch stray NULL pointers in the +kernel without hard coded checks. + + + +Address Spaces on Intel Linux +============================= + +The traditional Intel Linux is approximately mapped as follows forgive +the ascii art:: + + 0xFFFFFFFF 4GB Himem ***************** + * * + * Kernel Space * + * * + ***************** **************** + User Space Himem * User Stack * * * + (typically 0xC0000000 3GB ) ***************** * * + * Shared Libs * * Next Process * + ***************** * to * + * * <== * Run * <== + * User Program * * * + * Data BSS * * * + * Text * * * + * Sections * * * + 0x00000000 ***************** **************** + +Now it is easy to see that on Intel it is quite easy to recognise a kernel +address as being one greater than user space himem (in this case 0xC0000000), +and addresses of less than this are the ones in the current running program on +this processor (if an smp box). + +If using the virtual machine ( VM ) as a debugger it is quite difficult to +know which user process is running as the address space you are looking at +could be from any process in the run queue. + +The limitation of Intels addressing technique is that the linux +kernel uses a very simple real address to virtual addressing technique +of Real Address=Virtual Address-User Space Himem. +This means that on Intel the kernel linux can typically only address +Himem=0xFFFFFFFF-0xC0000000=1GB & this is all the RAM these machines +can typically use. + +They can lower User Himem to 2GB or lower & thus be +able to use 2GB of RAM however this shrinks the maximum size +of User Space from 3GB to 2GB they have a no win limit of 4GB unless +they go to 64 Bit. + + +On 390 our limitations & strengths make us slightly different. +For backward compatibility we are only allowed use 31 bits (2GB) +of our 32 bit addresses, however, we use entirely separate address +spaces for the user & kernel. + +This means we can support 2GB of non Extended RAM on s/390, & more +with the Extended memory management swap device & +currently 4TB of physical memory currently on z/Architecture. + + +Address Spaces on Linux for s/390 & z/Architecture +================================================== + +Our addressing scheme is basically as follows:: + + Primary Space Home Space + Himem 0x7fffffff 2GB on s/390 ***************** **************** + currently 0x3ffffffffff (2^42)-1 * User Stack * * * + on z/Architecture. ***************** * * + * Shared Libs * * * + ***************** * * + * * * Kernel * + * User Program * * * + * Data BSS * * * + * Text * * * + * Sections * * * + 0x00000000 ***************** **************** + +This also means that we need to look at the PSW problem state bit and the +addressing mode to decide whether we are looking at user or kernel space. + +User space runs in primary address mode (or access register mode within +the vdso code). + +The kernel usually also runs in home space mode, however when accessing +user space the kernel switches to primary or secondary address mode if +the mvcos instruction is not available or if a compare-and-swap (futex) +instruction on a user space address is performed. + +When also looking at the ASCE control registers, this means: + +User space: + +- runs in primary or access register mode +- cr1 contains the user asce +- cr7 contains the user asce +- cr13 contains the kernel asce + +Kernel space: + +- runs in home space mode +- cr1 contains the user or kernel asce + + - the kernel asce is loaded when a uaccess requires primary or + secondary address mode + +- cr7 contains the user or kernel asce, (changed with set_fs()) +- cr13 contains the kernel asce + +In case of uaccess the kernel changes to: + +- primary space mode in case of a uaccess (copy_to_user) and uses + e.g. the mvcp instruction to access user space. However the kernel + will stay in home space mode if the mvcos instruction is available +- secondary space mode in case of futex atomic operations, so that the + instructions come from primary address space and data from secondary + space + +In case of KVM, the kernel runs in home space mode, but cr1 gets switched +to contain the gmap asce before the SIE instruction gets executed. When +the SIE instruction is finished, cr1 will be switched back to contain the +user asce. + + +Virtual Addresses on s/390 & z/Architecture +=========================================== + +A virtual address on s/390 is made up of 3 parts +The SX (segment index, roughly corresponding to the PGD & PMD in Linux +terminology) being bits 1-11. + +The PX (page index, corresponding to the page table entry (pte) in Linux +terminology) being bits 12-19. + +The remaining bits BX (the byte index are the offset in the page ) +i.e. bits 20 to 31. + +On z/Architecture in linux we currently make up an address from 4 parts. + +- The region index bits (RX) 0-32 we currently use bits 22-32 +- The segment index (SX) being bits 33-43 +- The page index (PX) being bits 44-51 +- The byte index (BX) being bits 52-63 + +Notes: + 1) s/390 has no PMD so the PMD is really the PGD also. + A lot of this stuff is defined in pgtable.h. + + 2) Also seeing as s/390's page indexes are only 1k in size + (bits 12-19 x 4 bytes per pte ) we use 1 ( page 4k ) + to make the best use of memory by updating 4 segment indices + entries each time we mess with a PMD & use offsets + 0,1024,2048 & 3072 in this page as for our segment indexes. + On z/Architecture our page indexes are now 2k in size + ( bits 12-19 x 8 bytes per pte ) we do a similar trick + but only mess with 2 segment indices each time we mess with + a PMD. + + 3) As z/Architecture supports up to a massive 5-level page table lookup we + can only use 3 currently on Linux ( as this is all the generic kernel + currently supports ) however this may change in future + this allows us to access ( according to my sums ) + 4TB of virtual storage per process i.e. + 4096*512(PTES)*1024(PMDS)*2048(PGD) = 4398046511104 bytes, + enough for another 2 or 3 of years I think :-). + to do this we use a region-third-table designation type in + our address space control registers. + + +The Linux for s/390 & z/Architecture Kernel Task Structure +========================================================== +Each process/thread under Linux for S390 has its own kernel task_struct +defined in linux/include/linux/sched.h +The S390 on initialisation & resuming of a process on a cpu sets +the __LC_KERNEL_STACK variable in the spare prefix area for this cpu +(which we use for per-processor globals). + +The kernel stack pointer is intimately tied with the task structure for +each processor as follows:: + + s/390 + ************************ + * 1 page kernel stack * + * ( 4K ) * + ************************ + * 1 page task_struct * + * ( 4K ) * + 8K aligned ************************ + + z/Architecture + ************************ + * 2 page kernel stack * + * ( 8K ) * + ************************ + * 2 page task_struct * + * ( 8K ) * + 16K aligned ************************ + +What this means is that we don't need to dedicate any register or global +variable to point to the current running process & can retrieve it with the +following very simple construct for s/390 & one very similar for +z/Architecture:: + + static inline struct task_struct * get_current(void) + { + struct task_struct *current; + __asm__("lhi %0,-8192\n\t" + "nr %0,15" + : "=r" (current) ); + return current; + } + +i.e. just anding the current kernel stack pointer with the mask -8192. +Thankfully because Linux doesn't have support for nested IO interrupts +& our devices have large buffers can survive interrupts being shut for +short amounts of time we don't need a separate stack for interrupts. + + + + +Register Usage & Stackframes on Linux for s/390 & z/Architecture +================================================================= +Overview: +--------- +This is the code that gcc produces at the top & the bottom of +each function. It usually is fairly consistent & similar from +function to function & if you know its layout you can probably +make some headway in finding the ultimate cause of a problem +after a crash without a source level debugger. + +Note: To follow stackframes requires a knowledge of C or Pascal & +limited knowledge of one assembly language. + +It should be noted that there are some differences between the +s/390 and z/Architecture stack layouts as the z/Architecture stack layout +didn't have to maintain compatibility with older linkage formats. + +Glossary: +--------- +alloca: + This is a built in compiler function for runtime allocation + of extra space on the callers stack which is obviously freed + up on function exit ( e.g. the caller may choose to allocate nothing + of a buffer of 4k if required for temporary purposes ), it generates + very efficient code ( a few cycles ) when compared to alternatives + like malloc. + +automatics: + These are local variables on the stack, i.e they aren't in registers & + they aren't static. + +back-chain: + This is a pointer to the stack pointer before entering a + framed functions ( see frameless function ) prologue got by + dereferencing the address of the current stack pointer, + i.e. got by accessing the 32 bit value at the stack pointers + current location. + +base-pointer: + This is a pointer to the back of the literal pool which + is an area just behind each procedure used to store constants + in each function. + +call-clobbered: + The caller probably needs to save these registers if there + is something of value in them, on the stack or elsewhere before making a + call to another procedure so that it can restore it later. + +epilogue: + The code generated by the compiler to return to the caller. + +frameless-function: + A frameless function in Linux for s390 & z/Architecture is one which doesn't + need more than the register save area (96 bytes on s/390, 160 on z/Architecture) + given to it by the caller. + + A frameless function never: + + 1) Sets up a back chain. + 2) Calls alloca. + 3) Calls other normal functions + 4) Has automatics. + +GOT-pointer: + This is a pointer to the global-offset-table in ELF + ( Executable Linkable Format, Linux'es most common executable format ), + all globals & shared library objects are found using this pointer. + +lazy-binding + ELF shared libraries are typically only loaded when routines in the shared + library are actually first called at runtime. This is lazy binding. + +procedure-linkage-table + This is a table found from the GOT which contains pointers to routines + in other shared libraries which can't be called to by easier means. + +prologue: + The code generated by the compiler to set up the stack frame. + +outgoing-args: + This is extra area allocated on the stack of the calling function if the + parameters for the callee's cannot all be put in registers, the same + area can be reused by each function the caller calls. + +routine-descriptor: + A COFF executable format based concept of a procedure reference + actually being 8 bytes or more as opposed to a simple pointer to the routine. + This is typically defined as follows: + + - Routine Descriptor offset 0=Pointer to Function + - Routine Descriptor offset 4=Pointer to Table of Contents + + The table of contents/TOC is roughly equivalent to a GOT pointer. + & it means that shared libraries etc. can be shared between several + environments each with their own TOC. + +static-chain: + This is used in nested functions a concept adopted from pascal + by gcc not used in ansi C or C++ ( although quite useful ), basically it + is a pointer used to reference local variables of enclosing functions. + You might come across this stuff once or twice in your lifetime. + + e.g. + + The function below should return 11 though gcc may get upset & toss warnings + about unused variables:: + + int FunctionA(int a) + { + int b; + FunctionC(int c) + { + b=c+1; + } + FunctionC(10); + return(b); + } + + +s/390 & z/Architecture Register usage +===================================== + +======== ========================================== =============== +r0 used by syscalls/assembly call-clobbered +r1 used by syscalls/assembly call-clobbered +r2 argument 0 / return value 0 call-clobbered +r3 argument 1 / return value 1 (if long long) call-clobbered +r4 argument 2 call-clobbered +r5 argument 3 call-clobbered +r6 argument 4 saved +r7 pointer-to arguments 5 to ... saved +r8 this & that saved +r9 this & that saved +r10 static-chain ( if nested function ) saved +r11 frame-pointer ( if function used alloca ) saved +r12 got-pointer saved +r13 base-pointer saved +r14 return-address saved +r15 stack-pointer saved + +f0 argument 0 / return value ( float/double ) call-clobbered +f2 argument 1 call-clobbered +f4 z/Architecture argument 2 saved +f6 z/Architecture argument 3 saved +======== ========================================== =============== + +The remaining floating points +f1,f3,f5 f7-f15 are call-clobbered. + +Notes: +------ +1) The only requirement is that registers which are used + by the callee are saved, e.g. the compiler is perfectly + capable of using r11 for purposes other than a frame a + frame pointer if a frame pointer is not needed. +2) In functions with variable arguments e.g. printf the calling procedure + is identical to one without variable arguments & the same number of + parameters. However, the prologue of this function is somewhat more + hairy owing to it having to move these parameters to the stack to + get va_start, va_arg & va_end to work. +3) Access registers are currently unused by gcc but are used in + the kernel. Possibilities exist to use them at the moment for + temporary storage but it isn't recommended. +4) Only 4 of the floating point registers are used for + parameter passing as older machines such as G3 only have only 4 + & it keeps the stack frame compatible with other compilers. + However with IEEE floating point emulation under linux on the + older machines you are free to use the other 12. +5) A long long or double parameter cannot be have the + first 4 bytes in a register & the second four bytes in the + outgoing args area. It must be purely in the outgoing args + area if crossing this boundary. +6) Floating point parameters are mixed with outgoing args + on the outgoing args area in the order the are passed in as parameters. +7) Floating point arguments 2 & 3 are saved in the outgoing args area for + z/Architecture + + +Stack Frame Layout +------------------ + +========= ============== ====================================================== +s/390 z/Architecture +========= ============== ====================================================== +0 0 back chain ( a 0 here signifies end of back chain ) +4 8 eos ( end of stack, not used on Linux for S390 used + in other linkage formats ) +8 16 glue used in other s/390 linkage formats for saved + routine descriptors etc. +12 24 glue used in other s/390 linkage formats for saved + routine descriptors etc. +16 32 scratch area +20 40 scratch area +24 48 saved r6 of caller function +28 56 saved r7 of caller function +32 64 saved r8 of caller function +36 72 saved r9 of caller function +40 80 saved r10 of caller function +44 88 saved r11 of caller function +48 96 saved r12 of caller function +52 104 saved r13 of caller function +56 112 saved r14 of caller function +60 120 saved r15 of caller function +64 128 saved f4 of caller function +72 132 saved f6 of caller function +80 undefined +96 160 outgoing args passed from caller to callee +96+x 160+x possible stack alignment ( 8 bytes desirable ) +96+x+y 160+x+y alloca space of caller ( if used ) +96+x+y+z 160+x+y+z automatics of caller ( if used ) +0 back-chain +========= ============== ====================================================== + +A sample program with comments. +=============================== + +Comments on the function test +----------------------------- +1) It didn't need to set up a pointer to the constant pool gpr13 as it is not + used ( :-( ). +2) This is a frameless function & no stack is bought. +3) The compiler was clever enough to recognise that it could return the + value in r2 as well as use it for the passed in parameter ( :-) ). +4) The basr ( branch relative & save ) trick works as follows the instruction + has a special case with r0,r0 with some instruction operands is understood as + the literal value 0, some risc architectures also do this ). So now + we are branching to the next address & the address new program counter is + in r13,so now we subtract the size of the function prologue we have executed + the size of the literal pool to get to the top of the literal pool:: + + + 0040037c int test(int b) + { # Function prologue below + 40037c: 90 de f0 34 stm %r13,%r14,52(%r15) # Save registers r13 & r14 + 400380: 0d d0 basr %r13,%r0 # Set up pointer to constant pool using + 400382: a7 da ff fa ahi %r13,-6 # basr trick + return(5+b); + # Huge main program + 400386: a7 2a 00 05 ahi %r2,5 # add 5 to r2 + + # Function epilogue below + 40038a: 98 de f0 34 lm %r13,%r14,52(%r15) # restore registers r13 & 14 + 40038e: 07 fe br %r14 # return + } + +Comments on the function main +----------------------------- +1) The compiler did this function optimally ( 8-) ):: + + Literal pool for main. + 400390: ff ff ff ec .long 0xffffffec + main(int argc,char *argv[]) + { # Function prologue below + 400394: 90 bf f0 2c stm %r11,%r15,44(%r15) # Save necessary registers + 400398: 18 0f lr %r0,%r15 # copy stack pointer to r0 + 40039a: a7 fa ff a0 ahi %r15,-96 # Make area for callee saving + 40039e: 0d d0 basr %r13,%r0 # Set up r13 to point to + 4003a0: a7 da ff f0 ahi %r13,-16 # literal pool + 4003a4: 50 00 f0 00 st %r0,0(%r15) # Save backchain + + return(test(5)); # Main Program Below + 4003a8: 58 e0 d0 00 l %r14,0(%r13) # load relative address of test from + # literal pool + 4003ac: a7 28 00 05 lhi %r2,5 # Set first parameter to 5 + 4003b0: 4d ee d0 00 bas %r14,0(%r14,%r13) # jump to test setting r14 as return + # address using branch & save instruction. + + # Function Epilogue below + 4003b4: 98 bf f0 8c lm %r11,%r15,140(%r15)# Restore necessary registers. + 4003b8: 07 fe br %r14 # return to do program exit + } + + +Compiler updates +---------------- + +:: + + main(int argc,char *argv[]) + { + 4004fc: 90 7f f0 1c stm %r7,%r15,28(%r15) + 400500: a7 d5 00 04 bras %r13,400508 <main+0xc> + 400504: 00 40 04 f4 .long 0x004004f4 + # compiler now puts constant pool in code to so it saves an instruction + 400508: 18 0f lr %r0,%r15 + 40050a: a7 fa ff a0 ahi %r15,-96 + 40050e: 50 00 f0 00 st %r0,0(%r15) + return(test(5)); + 400512: 58 10 d0 00 l %r1,0(%r13) + 400516: a7 28 00 05 lhi %r2,5 + 40051a: 0d e1 basr %r14,%r1 + # compiler adds 1 extra instruction to epilogue this is done to + # avoid processor pipeline stalls owing to data dependencies on g5 & + # above as register 14 in the old code was needed directly after being loaded + # by the lm %r11,%r15,140(%r15) for the br %14. + 40051c: 58 40 f0 98 l %r4,152(%r15) + 400520: 98 7f f0 7c lm %r7,%r15,124(%r15) + 400524: 07 f4 br %r4 + } + + +Hartmut ( our compiler developer ) also has been threatening to take out the +stack backchain in optimised code as this also causes pipeline stalls, you +have been warned. + +64 bit z/Architecture code disassembly +-------------------------------------- + +If you understand the stuff above you'll understand the stuff +below too so I'll avoid repeating myself & just say that +some of the instructions have g's on the end of them to indicate +they are 64 bit & the stack offsets are a bigger, +the only other difference you'll find between 32 & 64 bit is that +we now use f4 & f6 for floating point arguments on 64 bit:: + + 00000000800005b0 <test>: + int test(int b) + { + return(5+b); + 800005b0: a7 2a 00 05 ahi %r2,5 + 800005b4: b9 14 00 22 lgfr %r2,%r2 # downcast to integer + 800005b8: 07 fe br %r14 + 800005ba: 07 07 bcr 0,%r7 + + + } + + 00000000800005bc <main>: + main(int argc,char *argv[]) + { + 800005bc: eb bf f0 58 00 24 stmg %r11,%r15,88(%r15) + 800005c2: b9 04 00 1f lgr %r1,%r15 + 800005c6: a7 fb ff 60 aghi %r15,-160 + 800005ca: e3 10 f0 00 00 24 stg %r1,0(%r15) + return(test(5)); + 800005d0: a7 29 00 05 lghi %r2,5 + # brasl allows jumps > 64k & is overkill here bras would do fune + 800005d4: c0 e5 ff ff ff ee brasl %r14,800005b0 <test> + 800005da: e3 40 f1 10 00 04 lg %r4,272(%r15) + 800005e0: eb bf f0 f8 00 04 lmg %r11,%r15,248(%r15) + 800005e6: 07 f4 br %r4 + } + + + +Compiling programs for debugging on Linux for s/390 & z/Architecture +==================================================================== +-gdwarf-2 now works it should be considered the default debugging +format for s/390 & z/Architecture as it is more reliable for debugging +shared libraries, normal -g debugging works much better now +Thanks to the IBM java compiler developers bug reports. + +This is typically done adding/appending the flags -g or -gdwarf-2 to the +CFLAGS & LDFLAGS variables Makefile of the program concerned. + +If using gdb & you would like accurate displays of registers & +stack traces compile without optimisation i.e make sure +that there is no -O2 or similar on the CFLAGS line of the Makefile & +the emitted gcc commands, obviously this will produce worse code +( not advisable for shipment ) but it is an aid to the debugging process. + +This aids debugging because the compiler will copy parameters passed in +in registers onto the stack so backtracing & looking at passed in +parameters will work, however some larger programs which use inline functions +will not compile without optimisation. + +Debugging with optimisation has since much improved after fixing +some bugs, please make sure you are using gdb-5.0 or later developed +after Nov'2000. + + + +Debugging under VM +================== + +Notes +----- +Addresses & values in the VM debugger are always hex never decimal +Address ranges are of the format <HexValue1>-<HexValue2> or +<HexValue1>.<HexValue2> +For example, the address range 0x2000 to 0x3000 can be described as 2000-3000 +or 2000.1000 + +The VM Debugger is case insensitive. + +VM's strengths are usually other debuggers weaknesses you can get at any +resource no matter how sensitive e.g. memory management resources, change +address translation in the PSW. For kernel hacking you will reap dividends if +you get good at it. + +The VM Debugger displays operators but not operands, and also the debugger +displays useful information on the same line as the author of the code probably +felt that it was a good idea not to go over the 80 columns on the screen. +This isn't as unintuitive as it may seem as the s/390 instructions are easy to +decode mentally and you can make a good guess at a lot of them as all the +operands are nibble (half byte aligned). +So if you have an objdump listing by hand, it is quite easy to follow, and if +you don't have an objdump listing keep a copy of the s/390 Reference Summary +or alternatively the s/390 principles of operation next to you. +e.g. even I can guess that +0001AFF8' LR 180F CC 0 +is a ( load register ) lr r0,r15 + +Also it is very easy to tell the length of a 390 instruction from the 2 most +significant bits in the instruction (not that this info is really useful except +if you are trying to make sense of a hexdump of code). +Here is a table + +======================= ================== +Bits Instruction Length +======================= ================== +00 2 Bytes +01 4 Bytes +10 4 Bytes +11 6 Bytes +======================= ================== + +The debugger also displays other useful info on the same line such as the +addresses being operated on destination addresses of branches & condition codes. +e.g.:: + + 00019736' AHI A7DAFF0E CC 1 + 000198BA' BRC A7840004 -> 000198C2' CC 0 + 000198CE' STM 900EF068 >> 0FA95E78 CC 2 + + + +Useful VM debugger commands +--------------------------- + +I suppose I'd better mention this before I start +to list the current active traces do:: + + Q TR + +there can be a maximum of 255 of these per set +( more about trace sets later ). + +To stop traces issue a:: + + TR END. + +To delete a particular breakpoint issue:: + + TR DEL <breakpoint number> + +The PA1 key drops to CP mode so you can issue debugger commands, +Doing alt c (on my 3270 console at least ) clears the screen. + +hitting b <enter> comes back to the running operating system +from cp mode ( in our case linux ). + +It is typically useful to add shortcuts to your profile.exec file +if you have one ( this is roughly equivalent to autoexec.bat in DOS ). +file here are a few from mine:: + + /* this gives me command history on issuing f12 */ + set pf12 retrieve + /* this continues */ + set pf8 imm b + /* goes to trace set a */ + set pf1 imm tr goto a + /* goes to trace set b */ + set pf2 imm tr goto b + /* goes to trace set c */ + set pf3 imm tr goto c + + + +Instruction Tracing +------------------- +Setting a simple breakpoint:: + + TR I PSWA <address> + +To debug a particular function try:: + + TR I R <function address range> + TR I on its own will single step. + TR I DATA <MNEMONIC> <OPTIONAL RANGE> will trace for particular mnemonics + +e.g.:: + + TR I DATA 4D R 0197BC.4000 + +will trace for BAS'es ( opcode 4D ) in the range 0197BC.4000 + +if you were inclined you could add traces for all branch instructions & +suffix them with the run prefix so you would have a backtrace on screen +when a program crashes:: + + TR BR <INTO OR FROM> will trace branches into or out of an address. + +e.g.:: + + TR BR INTO 0 + +is often quite useful if a program is getting awkward & deciding +to branch to 0 & crashing as this will stop at the address before in jumps to 0. + +:: + + TR I R <address range> RUN cmd d g + +single steps a range of addresses but stays running & +displays the gprs on each step. + + + +Displaying & modifying Registers +-------------------------------- +D G + will display all the gprs + +Adding a extra G to all the commands is necessary to access the full 64 bit +content in VM on z/Architecture. Obviously this isn't required for access +registers as these are still 32 bit. + +e.g. + +DGG + instead of DG + +D X + will display all the control registers +D AR + will display all the access registers +D AR4-7 + will display access registers 4 to 7 +CPU ALL D G + will display the GRPS of all CPUS in the configuration +D PSW + will display the current PSW +st PSW 2000 + will put the value 2000 into the PSW & cause crash your machine. +D PREFIX + displays the prefix offset + + +Displaying Memory +----------------- +To display memory mapped using the current PSW's mapping try:: + + D <range> + +To make VM display a message each time it hits a particular address and +continue try: + +D I<range> + will disassemble/display a range of instructions. + +ST addr 32 bit word + will store a 32 bit aligned address +D T<range> + will display the EBCDIC in an address (if you are that way inclined) +D R<range> + will display real addresses ( without DAT ) but with prefixing. + +There are other complex options to display if you need to get at say home space +but are in primary space the easiest thing to do is to temporarily +modify the PSW to the other addressing mode, display the stuff & then +restore it. + + + +Hints +----- +If you want to issue a debugger command without halting your virtual machine +with the PA1 key try prefixing the command with #CP e.g.:: + + #cp tr i pswa 2000 + +also suffixing most debugger commands with RUN will cause them not +to stop just display the mnemonic at the current instruction on the console. + +If you have several breakpoints you want to put into your program & +you get fed up of cross referencing with System.map +you can do the following trick for several symbols. + +:: + + grep do_signal System.map + +which emits the following among other things:: + + 0001f4e0 T do_signal + +now you can do:: + + TR I PSWA 0001f4e0 cmd msg * do_signal + +This sends a message to your own console each time do_signal is entered. +( As an aside I wrote a perl script once which automatically generated a REXX +script with breakpoints on every kernel procedure, this isn't a good idea +because there are thousands of these routines & VM can only set 255 breakpoints +at a time so you nearly had to spend as long pruning the file down as you would +entering the msgs by hand), however, the trick might be useful for a single +object file. In the 3270 terminal emulator x3270 there is a very useful option +in the file menu called "Save Screen In File" - this is very good for keeping a +copy of traces. + +From CMS help <command name> will give you online help on a particular command. +e.g.:: + + HELP DISPLAY + +Also CP has a file called profile.exec which automatically gets called +on startup of CMS ( like autoexec.bat ), keeping on a DOS analogy session +CP has a feature similar to doskey, it may be useful for you to +use profile.exec to define some keystrokes. + +SET PF9 IMM B + This does a single step in VM on pressing F8. + +SET PF10 ^ + This sets up the ^ key. + which can be used for ^c (ctrl-c),^z (ctrl-z) which can't be typed + directly into some 3270 consoles. + +SET PF11 ^- + This types the starting keystrokes for a sysrq see SysRq below. +SET PF12 RETRIEVE + This retrieves command history on pressing F12. + + +Sometimes in VM the display is set up to scroll automatically this +can be very annoying if there are messages you wish to look at +to stop this do + +TERM MORE 255 255 + This will nearly stop automatic screen updates, however it will + cause a denial of service if lots of messages go to the 3270 console, + so it would be foolish to use this as the default on a production machine. + + +Tracing particular processes +---------------------------- +The kernel's text segment is intentionally at an address in memory that it will +very seldom collide with text segments of user programs ( thanks Martin ), +this simplifies debugging the kernel. +However it is quite common for user processes to have addresses which collide +this can make debugging a particular process under VM painful under normal +circumstances as the process may change when doing a:: + + TR I R <address range>. + +Thankfully after reading VM's online help I figured out how to debug +I particular process. + +Your first problem is to find the STD ( segment table designation ) +of the program you wish to debug. +There are several ways you can do this here are a few + +Run:: + + objdump --syms <program to be debugged> | grep main + +To get the address of main in the program. Then:: + + tr i pswa <address of main> + +Start the program, if VM drops to CP on what looks like the entry +point of the main function this is most likely the process you wish to debug. +Now do a D X13 or D XG13 on z/Architecture. + +On 31 bit the STD is bits 1-19 ( the STO segment table origin ) +& 25-31 ( the STL segment table length ) of CR13. + +now type:: + + TR I R STD <CR13's value> 0.7fffffff + +e.g.:: + + TR I R STD 8F32E1FF 0.7fffffff + +Another very useful variation is:: + + TR STORE INTO STD <CR13's value> <address range> + +for finding out when a particular variable changes. + +An alternative way of finding the STD of a currently running process +is to do the following, ( this method is more complex but +could be quite convenient if you aren't updating the kernel much & +so your kernel structures will stay constant for a reasonable period of +time ). + +:: + + grep task /proc/<pid>/status + +from this you should see something like:: + + task: 0f160000 ksp: 0f161de8 pt_regs: 0f161f68 + +This now gives you a pointer to the task structure. + +Now make:: + + CC:="s390-gcc -g" kernel/sched.s + +To get the task_struct stabinfo. + +( task_struct is defined in include/linux/sched.h ). + +Now we want to look at +task->active_mm->pgd + +on my machine the active_mm in the task structure stab is +active_mm:(4,12),672,32 + +its offset is 672/8=84=0x54 + +the pgd member in the mm_struct stab is +pgd:(4,6)=*(29,5),96,32 +so its offset is 96/8=12=0xc + +so we'll:: + + hexdump -s 0xf160054 /dev/mem | more + +i.e. task_struct+active_mm offset +to look at the active_mm member:: + + f160054 0fee cc60 0019 e334 0000 0000 0000 0011 + +:: + + hexdump -s 0x0feecc6c /dev/mem | more + +i.e. active_mm+pgd offset:: + + feecc6c 0f2c 0000 0000 0001 0000 0001 0000 0010 + +we get something like +now do:: + + TR I R STD <pgd|0x7f> 0.7fffffff + +i.e. the 0x7f is added because the pgd only +gives the page table origin & we need to set the low bits +to the maximum possible segment table length. + +:: + + TR I R STD 0f2c007f 0.7fffffff + +on z/Architecture you'll probably need to do:: + + TR I R STD <pgd|0x7> 0.ffffffffffffffff + +to set the TableType to 0x1 & the Table length to 3. + + + +Tracing Program Exceptions +-------------------------- +If you get a crash which says something like +illegal operation or specification exception followed by a register dump +You can restart linux & trace these using the tr prog <range or value> trace +option. + + +The most common ones you will normally be tracing for is: + +- 1=operation exception +- 2=privileged operation exception +- 4=protection exception +- 5=addressing exception +- 6=specification exception +- 10=segment translation exception +- 11=page translation exception + +The full list of these is on page 22 of the current s/390 Reference Summary. +e.g. + +tr prog 10 will trace segment translation exceptions. + +tr prog on its own will trace all program interruption codes. + +Trace Sets +---------- +On starting VM you are initially in the INITIAL trace set. +You can do a Q TR to verify this. +If you have a complex tracing situation where you wish to wait for instance +till a driver is open before you start tracing IO, but know in your +heart that you are going to have to make several runs through the code till you +have a clue whats going on. + +What you can do is:: + + TR I PSWA <Driver open address> + +hit b to continue till breakpoint + +reach the breakpoint + +now do your:: + + TR GOTO B + TR IO 7c08-7c09 inst int run + +or whatever the IO channels you wish to trace are & hit b + +To got back to the initial trace set do:: + + TR GOTO INITIAL + +& the TR I PSWA <Driver open address> will be the only active breakpoint again. + + +Tracing linux syscalls under VM +------------------------------- +Syscalls are implemented on Linux for S390 by the Supervisor call instruction +(SVC). There 256 possibilities of these as the instruction is made up of a 0xA +opcode and the second byte being the syscall number. They are traced using the +simple command:: + + TR SVC <Optional value or range> + +the syscalls are defined in linux/arch/s390/include/asm/unistd.h +e.g. to trace all file opens just do:: + + TR SVC 5 ( as this is the syscall number of open ) + + +SMP Specific commands +--------------------- +To find out how many cpus you have +Q CPUS displays all the CPU's available to your virtual machine +To find the cpu that the current cpu VM debugger commands are being directed at +do Q CPU to change the current cpu VM debugger commands are being directed at +do:: + + CPU <desired cpu no> + +On a SMP guest issue a command to all CPUs try prefixing the command with cpu +all. To issue a command to a particular cpu try cpu <cpu number> e.g.:: + + CPU 01 TR I R 2000.3000 + +If you are running on a guest with several cpus & you have a IO related problem +& cannot follow the flow of code but you know it isn't smp related. + +from the bash prompt issue:: + + shutdown -h now or halt. + +do a:: + + Q CPUS + +to find out how many cpus you have detach each one of them from cp except +cpu 0 by issuing a:: + + DETACH CPU 01-(number of cpus in configuration) + +& boot linux again. + +TR SIGP + will trace inter processor signal processor instructions. + +DEFINE CPU 01-(number in configuration) + will get your guests cpus back. + + +Help for displaying ascii textstrings +------------------------------------- +On the very latest VM Nucleus'es VM can now display ascii +( thanks Neale for the hint ) by doing:: + + D TX<lowaddr>.<len> + +e.g.:: + + D TX0.100 + +Alternatively +============= +Under older VM debuggers (I love EBDIC too) you can use following little +program which converts a command line of hex digits to ascii text. It can be +compiled under linux and you can copy the hex digits from your x3270 terminal +to your xterm if you are debugging from a linuxbox. + +This is quite useful when looking at a parameter passed in as a text string +under VM ( unless you are good at decoding ASCII in your head ). + +e.g. consider tracing an open syscall:: + + TR SVC 5 + +We have stopped at a breakpoint:: + + 000151B0' SVC 0A05 -> 0001909A' CC 0 + +D 20.8 to check the SVC old psw in the prefix area and see was it from userspace +(for the layout of the prefix area consult the "Fixed Storage Locations" +chapter of the s/390 Reference Summary if you have it available). + +:: + + V00000020 070C2000 800151B2 + +The problem state bit wasn't set & it's also too early in the boot sequence +for it to be a userspace SVC if it was we would have to temporarily switch the +psw to user space addressing so we could get at the first parameter of the open +in gpr2. + +Next do a:: + + D G2 + GPR 2 = 00014CB4 + +Now display what gpr2 is pointing to:: + + D 00014CB4.20 + V00014CB4 2F646576 2F636F6E 736F6C65 00001BF5 + V00014CC4 FC00014C B4001001 E0001000 B8070707 + +Now copy the text till the first 00 hex ( which is the end of the string +to an xterm & do hex2ascii on it:: + + hex2ascii 2F646576 2F636F6E 736F6C65 00 + +outputs:: + + Decoded Hex:=/ d e v / c o n s o l e 0x00 + +We were opening the console device, + +You can compile the code below yourself for practice :-), + +:: + + /* + * hex2ascii.c + * a useful little tool for converting a hexadecimal command line to ascii + * + * Author(s): Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com) + * (C) 2000 IBM Deutschland Entwicklung GmbH, IBM Corporation. + */ + #include <stdio.h> + + int main(int argc,char *argv[]) + { + int cnt1,cnt2,len,toggle=0; + int startcnt=1; + unsigned char c,hex; + + if(argc>1&&(strcmp(argv[1],"-a")==0)) + startcnt=2; + printf("Decoded Hex:="); + for(cnt1=startcnt;cnt1<argc;cnt1++) + { + len=strlen(argv[cnt1]); + for(cnt2=0;cnt2<len;cnt2++) + { + c=argv[cnt1][cnt2]; + if(c>='0'&&c<='9') + c=c-'0'; + if(c>='A'&&c<='F') + c=c-'A'+10; + if(c>='a'&&c<='f') + c=c-'a'+10; + switch(toggle) + { + case 0: + hex=c<<4; + toggle=1; + break; + case 1: + hex+=c; + if(hex<32||hex>127) + { + if(startcnt==1) + printf("0x%02X ",(int)hex); + else + printf("."); + } + else + { + printf("%c",hex); + if(startcnt==1) + printf(" "); + } + toggle=0; + break; + } + } + } + printf("\n"); + } + + + + +Stack tracing under VM +---------------------- +A basic backtrace +----------------- + +Here are the tricks I use 9 out of 10 times it works pretty well, + +When your backchain reaches a dead end +-------------------------------------- +This can happen when an exception happens in the kernel and the kernel is +entered twice. If you reach the NULL pointer at the end of the back chain you +should be able to sniff further back if you follow the following tricks. +1) A kernel address should be easy to recognise since it is in +primary space & the problem state bit isn't set & also +The Hi bit of the address is set. +2) Another backchain should also be easy to recognise since it is an +address pointing to another address approximately 100 bytes or 0x70 hex +behind the current stackpointer. + + +Here is some practice. + +boot the kernel & hit PA1 at some random time + +d g to display the gprs, this should display something like:: + + GPR 0 = 00000001 00156018 0014359C 00000000 + GPR 4 = 00000001 001B8888 000003E0 00000000 + GPR 8 = 00100080 00100084 00000000 000FE000 + GPR 12 = 00010400 8001B2DC 8001B36A 000FFED8 + +Note that GPR14 is a return address but as we are real men we are going to +trace the stack. +display 0x40 bytes after the stack pointer:: + + V000FFED8 000FFF38 8001B838 80014C8E 000FFF38 + V000FFEE8 00000000 00000000 000003E0 00000000 + V000FFEF8 00100080 00100084 00000000 000FE000 + V000FFF08 00010400 8001B2DC 8001B36A 000FFED8 + + +Ah now look at whats in sp+56 (sp+0x38) this is 8001B36A our saved r14 if +you look above at our stackframe & also agrees with GPR14. + +now backchain:: + + d 000FFF38.40 + +we now are taking the contents of SP to get our first backchain:: + + V000FFF38 000FFFA0 00000000 00014995 00147094 + V000FFF48 00147090 001470A0 000003E0 00000000 + V000FFF58 00100080 00100084 00000000 001BF1D0 + V000FFF68 00010400 800149BA 80014CA6 000FFF38 + +This displays a 2nd return address of 80014CA6 + +now do:: + + d 000FFFA0.40 + +for our 3rd backchain:: + + V000FFFA0 04B52002 0001107F 00000000 00000000 + V000FFFB0 00000000 00000000 FF000000 0001107F + V000FFFC0 00000000 00000000 00000000 00000000 + V000FFFD0 00010400 80010802 8001085A 000FFFA0 + + +our 3rd return address is 8001085A + +as the 04B52002 looks suspiciously like rubbish it is fair to assume that the +kernel entry routines for the sake of optimisation don't set up a backchain. + +now look at System.map to see if the addresses make any sense:: + + grep -i 0001b3 System.map + +outputs among other things:: + + 0001b304 T cpu_idle + +so 8001B36A +is cpu_idle+0x66 ( quiet the cpu is asleep, don't wake it ) + +:: + + grep -i 00014 System.map + +produces among other things:: + + 00014a78 T start_kernel + +so 0014CA6 is start_kernel+some hex number I can't add in my head. + +:: + + grep -i 00108 System.map + +this produces:: + + 00010800 T _stext + +so 8001085A is _stext+0x5a + +Congrats you've done your first backchain. + + + +s/390 & z/Architecture IO Overview +================================== + +I am not going to give a course in 390 IO architecture as this would take me +quite a while and I'm no expert. Instead I'll give a 390 IO architecture +summary for Dummies. If you have the s/390 principles of operation available +read this instead. If nothing else you may find a few useful keywords in here +and be able to use them on a web search engine to find more useful information. + +Unlike other bus architectures modern 390 systems do their IO using mostly +fibre optics and devices such as tapes and disks can be shared between several +mainframes. Also S390 can support up to 65536 devices while a high end PC based +system might be choking with around 64. + +Here is some of the common IO terminology: + +Subchannel: + This is the logical number most IO commands use to talk to an IO device. There + can be up to 0x10000 (65536) of these in a configuration, typically there are a + few hundred. Under VM for simplicity they are allocated contiguously, however + on the native hardware they are not. They typically stay consistent between + boots provided no new hardware is inserted or removed. + + Under Linux for s390 we use these as IRQ's and also when issuing an IO command + (CLEAR SUBCHANNEL, HALT SUBCHANNEL, MODIFY SUBCHANNEL, RESUME SUBCHANNEL, + START SUBCHANNEL, STORE SUBCHANNEL and TEST SUBCHANNEL). We use this as the ID + of the device we wish to talk to. The most important of these instructions are + START SUBCHANNEL (to start IO), TEST SUBCHANNEL (to check whether the IO + completed successfully) and HALT SUBCHANNEL (to kill IO). A subchannel can have + up to 8 channel paths to a device, this offers redundancy if one is not + available. + +Device Number: + This number remains static and is closely tied to the hardware. There are 65536 + of these, made up of a CHPID (Channel Path ID, the most significant 8 bits) and + another lsb 8 bits. These remain static even if more devices are inserted or + removed from the hardware. There is a 1 to 1 mapping between subchannels and + device numbers, provided devices aren't inserted or removed. + +Channel Control Words: + CCWs are linked lists of instructions initially pointed to by an operation + request block (ORB), which is initially given to Start Subchannel (SSCH) + command along with the subchannel number for the IO subsystem to process + while the CPU continues executing normal code. + CCWs come in two flavours, Format 0 (24 bit for backward compatibility) and + Format 1 (31 bit). These are typically used to issue read and write (and many + other) instructions. They consist of a length field and an absolute address + field. + + Each IO typically gets 1 or 2 interrupts, one for channel end (primary status) + when the channel is idle, and the second for device end (secondary status). + Sometimes you get both concurrently. You check how the IO went on by issuing a + TEST SUBCHANNEL at each interrupt, from which you receive an Interruption + response block (IRB). If you get channel and device end status in the IRB + without channel checks etc. your IO probably went okay. If you didn't you + probably need to examine the IRB, extended status word etc. + If an error occurs, more sophisticated control units have a facility known as + concurrent sense. This means that if an error occurs Extended sense information + will be presented in the Extended status word in the IRB. If not you have to + issue a subsequent SENSE CCW command after the test subchannel. + + +TPI (Test pending interrupt) can also be used for polled IO, but in +multitasking multiprocessor systems it isn't recommended except for +checking special cases (i.e. non looping checks for pending IO etc.). + +Store Subchannel and Modify Subchannel can be used to examine and modify +operating characteristics of a subchannel (e.g. channel paths). + +Other IO related Terms: + +Sysplex: + S390's Clustering Technology +QDIO: + S390's new high speed IO architecture to support devices such as gigabit + ethernet, this architecture is also designed to be forward compatible with + upcoming 64 bit machines. + + +General Concepts +---------------- + +Input Output Processors (IOP's) are responsible for communicating between +the mainframe CPU's & the channel & relieve the mainframe CPU's from the +burden of communicating with IO devices directly, this allows the CPU's to +concentrate on data processing. + +IOP's can use one or more links ( known as channel paths ) to talk to each +IO device. It first checks for path availability & chooses an available one, +then starts ( & sometimes terminates IO ). +There are two types of channel path: ESCON & the Parallel IO interface. + +IO devices are attached to control units, control units provide the +logic to interface the channel paths & channel path IO protocols to +the IO devices, they can be integrated with the devices or housed separately +& often talk to several similar devices ( typical examples would be raid +controllers or a control unit which connects to 1000 3270 terminals ):: + + + +---------------------------------------------------------------+ + | +-----+ +-----+ +-----+ +-----+ +----------+ +----------+ | + | | CPU | | CPU | | CPU | | CPU | | Main | | Expanded | | + | | | | | | | | | | Memory | | Storage | | + | +-----+ +-----+ +-----+ +-----+ +----------+ +----------+ | + |---------------------------------------------------------------+ + | IOP | IOP | IOP | + |--------------------------------------------------------------- + | C | C | C | C | C | C | C | C | C | C | C | C | C | C | C | C | + ---------------------------------------------------------------- + || || + || Bus & Tag Channel Path || ESCON + || ====================== || Channel + || || || || Path + +----------+ +----------+ +----------+ + | | | | | | + | CU | | CU | | CU | + | | | | | | + +----------+ +----------+ +----------+ + | | | | | + +----------+ +----------+ +----------+ +----------+ +----------+ + |I/O Device| |I/O Device| |I/O Device| |I/O Device| |I/O Device| + +----------+ +----------+ +----------+ +----------+ +----------+ + CPU = Central Processing Unit + C = Channel + IOP = IP Processor + CU = Control Unit + +The 390 IO systems come in 2 flavours the current 390 machines support both + +The Older 360 & 370 Interface,sometimes called the Parallel I/O interface, +sometimes called Bus-and Tag & sometimes Original Equipment Manufacturers +Interface (OEMI). + +This byte wide Parallel channel path/bus has parity & data on the "Bus" cable +and control lines on the "Tag" cable. These can operate in byte multiplex mode +for sharing between several slow devices or burst mode and monopolize the +channel for the whole burst. Up to 256 devices can be addressed on one of these +cables. These cables are about one inch in diameter. The maximum unextended +length supported by these cables is 125 Meters but this can be extended up to +2km with a fibre optic channel extended such as a 3044. The maximum burst speed +supported is 4.5 megabytes per second. However, some really old processors +support only transfer rates of 3.0, 2.0 & 1.0 MB/sec. +One of these paths can be daisy chained to up to 8 control units. + + +ESCON if fibre optic it is also called FICON +Was introduced by IBM in 1990. Has 2 fibre optic cables and uses either leds or +lasers for communication at a signaling rate of up to 200 megabits/sec. As +10bits are transferred for every 8 bits info this drops to 160 megabits/sec +and to 18.6 Megabytes/sec once control info and CRC are added. ESCON only +operates in burst mode. + +ESCONs typical max cable length is 3km for the led version and 20km for the +laser version known as XDF (extended distance facility). This can be further +extended by using an ESCON director which triples the above mentioned ranges. +Unlike Bus & Tag as ESCON is serial it uses a packet switching architecture, +the standard Bus & Tag control protocol is however present within the packets. +Up to 256 devices can be attached to each control unit that uses one of these +interfaces. + +Common 390 Devices include: +Network adapters typically OSA2,3172's,2116's & OSA-E gigabit ethernet adapters, +Consoles 3270 & 3215 (a teletype emulated under linux for a line mode console). +DASD's direct access storage devices ( otherwise known as hard disks ). +Tape Drives. +CTC ( Channel to Channel Adapters ), +ESCON or Parallel Cables used as a very high speed serial link +between 2 machines. + + +Debugging IO on s/390 & z/Architecture under VM +=============================================== + +Now we are ready to go on with IO tracing commands under VM + +A few self explanatory queries:: + + Q OSA + Q CTC + Q DISK ( This command is CMS specific ) + Q DASD + +Q OSA on my machine returns:: + + OSA 7C08 ON OSA 7C08 SUBCHANNEL = 0000 + OSA 7C09 ON OSA 7C09 SUBCHANNEL = 0001 + OSA 7C14 ON OSA 7C14 SUBCHANNEL = 0002 + OSA 7C15 ON OSA 7C15 SUBCHANNEL = 0003 + +If you have a guest with certain privileges you may be able to see devices +which don't belong to you. To avoid this, add the option V. +e.g.:: + + Q V OSA + +Now using the device numbers returned by this command we will +Trace the io starting up on the first device 7c08 & 7c09 +In our simplest case we can trace the +start subchannels +like TR SSCH 7C08-7C09 +or the halt subchannels +or TR HSCH 7C08-7C09 +MSCH's ,STSCH's I think you can guess the rest + +A good trick is tracing all the IO's and CCWS and spooling them into the reader +of another VM guest so he can ftp the logfile back to his own machine. I'll do +a small bit of this and give you a look at the output. + +1) Spool stdout to VM reader:: + + SP PRT TO (another vm guest ) or * for the local vm guest + +2) Fill the reader with the trace:: + + TR IO 7c08-7c09 INST INT CCW PRT RUN + +3) Start up linux:: + + i 00c +4) Finish the trace:: + + TR END + +5) close the reader:: + + C PRT + +6) list reader contents:: + + RDRLIST + +7) copy it to linux4's minidisk:: + + RECEIVE / LOG TXT A1 ( replace + +8) +filel & press F11 to look at it +You should see something like:: + + 00020942' SSCH B2334000 0048813C CC 0 SCH 0000 DEV 7C08 + CPA 000FFDF0 PARM 00E2C9C4 KEY 0 FPI C0 LPM 80 + CCW 000FFDF0 E4200100 00487FE8 0000 E4240100 ........ + IDAL 43D8AFE8 + IDAL 0FB76000 + 00020B0A' I/O DEV 7C08 -> 000197BC' SCH 0000 PARM 00E2C9C4 + 00021628' TSCH B2354000 >> 00488164 CC 0 SCH 0000 DEV 7C08 + CCWA 000FFDF8 DEV STS 0C SCH STS 00 CNT 00EC + KEY 0 FPI C0 CC 0 CTLS 4007 + 00022238' STSCH B2344000 >> 00488108 CC 0 SCH 0000 DEV 7C08 + +If you don't like messing up your readed ( because you possibly booted from it ) +you can alternatively spool it to another readers guest. + + +Other common VM device related commands +--------------------------------------------- +These commands are listed only because they have +been of use to me in the past & may be of use to +you too. For more complete info on each of the commands +use type HELP <command> from CMS. + +detaching devices:: + + DET <devno range> + ATT <devno range> <guest> + +attach a device to guest * for your own guest + +READY <devno> + cause VM to issue a fake interrupt. + +The VARY command is normally only available to VM administrators:: + + VARY ON PATH <path> TO <devno range> + VARY OFF PATH <PATH> FROM <devno range> + +This is used to switch on or off channel paths to devices. + +Q CHPID <channel path ID> + This displays state of devices using this channel path + +D SCHIB <subchannel> + This displays the subchannel information SCHIB block for the device. + this I believe is also only available to administrators. + +DEFINE CTC <devno> + defines a virtual CTC channel to channel connection + 2 need to be defined on each guest for the CTC driver to use. + +COUPLE devno userid remote devno + Joins a local virtual device to a remote virtual device + ( commonly used for the CTC driver ). + +Building a VM ramdisk under CMS which linux can use:: + + def vfb-<blocksize> <subchannel> <number blocks> + +blocksize is commonly 4096 for linux. + +Formatting it:: + + format <subchannel> <driver letter e.g. x> (blksize <blocksize> + +Sharing a disk between multiple guests:: + + LINK userid devno1 devno2 mode password + + + +GDB on S390 +=========== +N.B. if compiling for debugging gdb works better without optimisation +( see Compiling programs for debugging ) + +invocation +---------- +gdb <victim program> <optional corefile> + +Online help +----------- +help: gives help on commands + +e.g.:: + + help + help display + +Note gdb's online help is very good use it. + + +Assembly +-------- +info registers: + displays registers other than floating point. + +info all-registers: + displays floating points as well. + +disassemble: + disassembles + +e.g.:: + + disassemble without parameters will disassemble the current function + disassemble $pc $pc+10 + +Viewing & modifying variables +----------------------------- +print or p: + displays variable or register + +e.g. p/x $sp will display the stack pointer + +display: + prints variable or register each time program stops + +e.g.:: + + display/x $pc will display the program counter + display argc + +undisplay: + undo's display's + +info breakpoints: + shows all current breakpoints + +info stack: + shows stack back trace (if this doesn't work too well, I'll show + you the stacktrace by hand below). + +info locals: + displays local variables. + +info args: + display current procedure arguments. + +set args: + will set argc & argv each time the victim program is invoked + +e.g.:: + + set <variable>=value + set argc=100 + set $pc=0 + + + +Modifying execution +------------------- +step: + steps n lines of sourcecode + +step + steps 1 line. + +step 100 + steps 100 lines of code. + +next: + like step except this will not step into subroutines + +stepi: + steps a single machine code instruction. + +e.g.:: + + stepi 100 + +nexti: + steps a single machine code instruction but will not step into + subroutines. + +finish: + will run until exit of the current routine + +run: + (re)starts a program + +cont: + continues a program + +quit: + exits gdb. + + +breakpoints +------------ + +break + sets a breakpoint + +e.g.:: + + break main + break *$pc + break *0x400618 + +Here's a really useful one for large programs + +rbr + Set a breakpoint for all functions matching REGEXP + +e.g.:: + + rbr 390 + +will set a breakpoint with all functions with 390 in their name. + +info breakpoints + lists all breakpoints + +delete: + delete breakpoint by number or delete them all + +e.g. + +delete 1 + will delete the first breakpoint + + +delete + will delete them all + +watch: + This will set a watchpoint ( usually hardware assisted ), + +This will watch a variable till it changes + +e.g. + +watch cnt + will watch the variable cnt till it changes. + +As an aside unfortunately gdb's, architecture independent watchpoint code +is inconsistent & not very good, watchpoints usually work but not always. + +info watchpoints: + Display currently active watchpoints + +condition: ( another useful one ) + Specify breakpoint number N to break only if COND is true. + +Usage is `condition N COND`, where N is an integer and COND is an +expression to be evaluated whenever breakpoint N is reached. + + + +User defined functions/macros +----------------------------- +define: ( Note this is very very useful,simple & powerful ) + +usage define <name> <list of commands> end + +examples which you should consider putting into .gdbinit in your home +directory:: + + define d + stepi + disassemble $pc $pc+10 + end + define e + nexti + disassemble $pc $pc+10 + end + + +Other hard to classify stuff +---------------------------- +signal n: + sends the victim program a signal. + +e.g. `signal 3` will send a SIGQUIT. + +info signals: + what gdb does when the victim receives certain signals. + +list: + +e.g.: + +list + lists current function source +list 1,10 + list first 10 lines of current file. + +list test.c:1,10 + + +directory: + Adds directories to be searched for source if gdb cannot find the source. + (note it is a bit sensitive about slashes) + +e.g. To add the root of the filesystem to the searchpath do:: + + directory // + + +call <function> +This calls a function in the victim program, this is pretty powerful +e.g. +(gdb) call printf("hello world") +outputs: +$1 = 11 + +You might now be thinking that the line above didn't work, something extra had +to be done. +(gdb) call fflush(stdout) +hello world$2 = 0 +As an aside the debugger also calls malloc & free under the hood +to make space for the "hello world" string. + + + +hints +----- +1) command completion works just like bash + ( if you are a bad typist like me this really helps ) + +e.g. hit br <TAB> & cursor up & down :-). + +2) if you have a debugging problem that takes a few steps to recreate +put the steps into a file called .gdbinit in your current working directory +if you have defined a few extra useful user defined commands put these in +your home directory & they will be read each time gdb is launched. + +A typical .gdbinit file might be.:: + + break main + run + break runtime_exception + cont + + +stack chaining in gdb by hand +----------------------------- +This is done using a the same trick described for VM:: + + p/x (*($sp+56))&0x7fffffff + +get the first backchain. + +For z/Architecture +Replace 56 with 112 & ignore the &0x7fffffff +in the macros below & do nasty casts to longs like the following +as gdb unfortunately deals with printed arguments as ints which +messes up everything. + +i.e. here is a 3rd backchain dereference:: + + p/x *(long *)(***(long ***)$sp+112) + + +this outputs:: + + $5 = 0x528f18 + +on my machine. + +Now you can use:: + + info symbol (*($sp+56))&0x7fffffff + +you might see something like:: + + rl_getc + 36 in section .text + +telling you what is located at address 0x528f18 +Now do:: + + p/x (*(*$sp+56))&0x7fffffff + +This outputs:: + + $6 = 0x528ed0 + +Now do:: + + info symbol (*(*$sp+56))&0x7fffffff + rl_read_key + 180 in section .text + +now do:: + + p/x (*(**$sp+56))&0x7fffffff + +& so on. + +Disassembling instructions without debug info +--------------------------------------------- +gdb typically complains if there is a lack of debugging +symbols in the disassemble command with +"No function contains specified address." To get around +this do:: + + x/<number lines to disassemble>xi <address> + +e.g.:: + + x/20xi 0x400730 + + + +Note: + Remember gdb has history just like bash you don't need to retype the + whole line just use the up & down arrows. + + + +For more info +------------- +From your linuxbox do:: + + man gdb + +or:: + + info gdb. + +core dumps +---------- + +What a core dump ? +^^^^^^^^^^^^^^^^^^ + +A core dump is a file generated by the kernel (if allowed) which contains the +registers and all active pages of the program which has crashed. + +From this file gdb will allow you to look at the registers, stack trace and +memory of the program as if it just crashed on your system. It is usually +called core and created in the current working directory. + +This is very useful in that a customer can mail a core dump to a technical +support department and the technical support department can reconstruct what +happened. Provided they have an identical copy of this program with debugging +symbols compiled in and the source base of this build is available. + +In short it is far more useful than something like a crash log could ever hope +to be. + +Why have I never seen one ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Probably because you haven't used the command:: + + ulimit -c unlimited in bash + +to allow core dumps, now do:: + + ulimit -a + +to verify that the limit was accepted. + +A sample core dump + To create this I'm going to do:: + + ulimit -c unlimited + gdb + +to launch gdb (my victim app. ) now be bad & do the following from another +telnet/xterm session to the same machine:: + + ps -aux | grep gdb + kill -SIGSEGV <gdb's pid> + +or alternatively use `killall -SIGSEGV gdb` if you have the killall command. + +Now look at the core dump:: + + ./gdb core + +Displays the following:: + + GNU gdb 4.18 + Copyright 1998 Free Software Foundation, Inc. + GDB is free software, covered by the GNU General Public License, and you are + welcome to change it and/or distribute copies of it under certain conditions. + Type "show copying" to see the conditions. + There is absolutely no warranty for GDB. Type "show warranty" for details. + This GDB was configured as "s390-ibm-linux"... + Core was generated by `./gdb'. + Program terminated with signal 11, Segmentation fault. + Reading symbols from /usr/lib/libncurses.so.4...done. + Reading symbols from /lib/libm.so.6...done. + Reading symbols from /lib/libc.so.6...done. + Reading symbols from /lib/ld-linux.so.2...done. + #0 0x40126d1a in read () from /lib/libc.so.6 + Setting up the environment for debugging gdb. + Breakpoint 1 at 0x4dc6f8: file utils.c, line 471. + Breakpoint 2 at 0x4d87a4: file top.c, line 2609. + (top-gdb) info stack + #0 0x40126d1a in read () from /lib/libc.so.6 + #1 0x528f26 in rl_getc (stream=0x7ffffde8) at input.c:402 + #2 0x528ed0 in rl_read_key () at input.c:381 + #3 0x5167e6 in readline_internal_char () at readline.c:454 + #4 0x5168ee in readline_internal_charloop () at readline.c:507 + #5 0x51692c in readline_internal () at readline.c:521 + #6 0x5164fe in readline (prompt=0x7ffff810) + at readline.c:349 + #7 0x4d7a8a in command_line_input (prompt=0x564420 "(gdb) ", repeat=1, + annotation_suffix=0x4d6b44 "prompt") at top.c:2091 + #8 0x4d6cf0 in command_loop () at top.c:1345 + #9 0x4e25bc in main (argc=1, argv=0x7ffffdf4) at main.c:635 + + +LDD +=== +This is a program which lists the shared libraries which a library needs, +Note you also get the relocations of the shared library text segments which +help when using objdump --source. + +e.g.:: + + ldd ./gdb + +outputs:: + + libncurses.so.4 => /usr/lib/libncurses.so.4 (0x40018000) + libm.so.6 => /lib/libm.so.6 (0x4005e000) + libc.so.6 => /lib/libc.so.6 (0x40084000) + /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) + + +Debugging shared libraries +========================== +Most programs use shared libraries, however it can be very painful +when you single step instruction into a function like printf for the +first time & you end up in functions like _dl_runtime_resolve this is +the ld.so doing lazy binding, lazy binding is a concept in ELF where +shared library functions are not loaded into memory unless they are +actually used, great for saving memory but a pain to debug. + +To get around this either relink the program -static or exit gdb type +export LD_BIND_NOW=true this will stop lazy binding & restart the gdb'ing +the program in question. + + + +Debugging modules +================= +As modules are dynamically loaded into the kernel their address can be +anywhere to get around this use the -m option with insmod to emit a load +map which can be piped into a file if required. + +The proc file system +==================== +What is it ?. +It is a filesystem created by the kernel with files which are created on demand +by the kernel if read, or can be used to modify kernel parameters, +it is a powerful concept. + +e.g.:: + + cat /proc/sys/net/ipv4/ip_forward + +On my machine outputs:: + + 0 + +telling me ip_forwarding is not on to switch it on I can do:: + + echo 1 > /proc/sys/net/ipv4/ip_forward + +cat it again:: + + cat /proc/sys/net/ipv4/ip_forward + +On my machine now outputs:: + + 1 + +IP forwarding is on. + +There is a lot of useful info in here best found by going in and having a look +around, so I'll take you through some entries I consider important. + +All the processes running on the machine have their own entry defined by +/proc/<pid> + +So lets have a look at the init process:: + + cd /proc/1 + cat cmdline + +emits:: + + init [2] + +:: + + cd /proc/1/fd + +This contains numerical entries of all the open files, +some of these you can cat e.g. stdout (2):: + + cat /proc/29/maps + +on my machine emits:: + + 00400000-00478000 r-xp 00000000 5f:00 4103 /bin/bash + 00478000-0047e000 rw-p 00077000 5f:00 4103 /bin/bash + 0047e000-00492000 rwxp 00000000 00:00 0 + 40000000-40015000 r-xp 00000000 5f:00 14382 /lib/ld-2.1.2.so + 40015000-40016000 rw-p 00014000 5f:00 14382 /lib/ld-2.1.2.so + 40016000-40017000 rwxp 00000000 00:00 0 + 40017000-40018000 rw-p 00000000 00:00 0 + 40018000-4001b000 r-xp 00000000 5f:00 14435 /lib/libtermcap.so.2.0.8 + 4001b000-4001c000 rw-p 00002000 5f:00 14435 /lib/libtermcap.so.2.0.8 + 4001c000-4010d000 r-xp 00000000 5f:00 14387 /lib/libc-2.1.2.so + 4010d000-40111000 rw-p 000f0000 5f:00 14387 /lib/libc-2.1.2.so + 40111000-40114000 rw-p 00000000 00:00 0 + 40114000-4011e000 r-xp 00000000 5f:00 14408 /lib/libnss_files-2.1.2.so + 4011e000-4011f000 rw-p 00009000 5f:00 14408 /lib/libnss_files-2.1.2.so + 7fffd000-80000000 rwxp ffffe000 00:00 0 + + +Showing us the shared libraries init uses where they are in memory +& memory access permissions for each virtual memory area. + +/proc/1/cwd is a softlink to the current working directory. + +/proc/1/root is the root of the filesystem for this process. + +/proc/1/mem is the current running processes memory which you +can read & write to like a file. + +strace uses this sometimes as it is a bit faster than the +rather inefficient ptrace interface for peeking at DATA. + +:: + + cat status + + Name: init + State: S (sleeping) + Pid: 1 + PPid: 0 + Uid: 0 0 0 0 + Gid: 0 0 0 0 + Groups: + VmSize: 408 kB + VmLck: 0 kB + VmRSS: 208 kB + VmData: 24 kB + VmStk: 8 kB + VmExe: 368 kB + VmLib: 0 kB + SigPnd: 0000000000000000 + SigBlk: 0000000000000000 + SigIgn: 7fffffffd7f0d8fc + SigCgt: 00000000280b2603 + CapInh: 00000000fffffeff + CapPrm: 00000000ffffffff + CapEff: 00000000fffffeff + + User PSW: 070de000 80414146 + task: 004b6000 tss: 004b62d8 ksp: 004b7ca8 pt_regs: 004b7f68 + User GPRS: + 00000400 00000000 0000000b 7ffffa90 + 00000000 00000000 00000000 0045d9f4 + 0045cafc 7ffffa90 7fffff18 0045cb08 + 00010400 804039e8 80403af8 7ffff8b0 + User ACRS: + 00000000 00000000 00000000 00000000 + 00000001 00000000 00000000 00000000 + 00000000 00000000 00000000 00000000 + 00000000 00000000 00000000 00000000 + Kernel BackChain CallChain BackChain CallChain + 004b7ca8 8002bd0c 004b7d18 8002b92c + 004b7db8 8005cd50 004b7e38 8005d12a + 004b7f08 80019114 + +Showing among other things memory usage & status of some signals & +the processes'es registers from the kernel task_structure +as well as a backchain which may be useful if a process crashes +in the kernel for some unknown reason. + +Some driver debugging techniques +================================ +debug feature +------------- +Some of our drivers now support a "debug feature" in +/proc/s390dbf see s390dbf.txt in the linux/Documentation directory +for more info. + +e.g. +to switch on the lcs "debug feature":: + + echo 5 > /proc/s390dbf/lcs/level + +& then after the error occurred:: + + cat /proc/s390dbf/lcs/sprintf >/logfile + +the logfile now contains some information which may help +tech support resolve a problem in the field. + + + +high level debugging network drivers +------------------------------------ +ifconfig is a quite useful command +it gives the current state of network drivers. + +If you suspect your network device driver is dead +one way to check is type:: + + ifconfig <network device> + +e.g. tr0 + +You should see something like:: + + ifconfig tr0 + tr0 Link encap:16/4 Mbps Token Ring (New) HWaddr 00:04:AC:20:8E:48 + inet addr:9.164.185.132 Bcast:9.164.191.255 Mask:255.255.224.0 + UP BROADCAST RUNNING MULTICAST MTU:2000 Metric:1 + RX packets:246134 errors:0 dropped:0 overruns:0 frame:0 + TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:100 + +if the device doesn't say up +try:: + + /etc/rc.d/init.d/network start + +( this starts the network stack & hopefully calls ifconfig tr0 up ). +ifconfig looks at the output of /proc/net/dev and presents it in a more +presentable form. + +Now ping the device from a machine in the same subnet. + +if the RX packets count & TX packets counts don't increment you probably +have problems. + +next:: + + cat /proc/net/arp + +Do you see any hardware addresses in the cache if not you may have problems. +Next try:: + + ping -c 5 <broadcast_addr> + +i.e. the Bcast field above in the output of +ifconfig. Do you see any replies from machines other than the local machine +if not you may have problems. also if the TX packets count in ifconfig +hasn't incremented either you have serious problems in your driver +(e.g. the txbusy field of the network device being stuck on ) +or you may have multiple network devices connected. + + +chandev +------- +There is a new device layer for channel devices, some +drivers e.g. lcs are registered with this layer. + +If the device uses the channel device layer you'll be +able to find what interrupts it uses & the current state +of the device. + +See the manpage chandev.8 &type cat /proc/chandev for more info. + + +SysRq +===== +This is now supported by linux for s/390 & z/Architecture. + +To enable it do compile the kernel with:: + + Kernel Hacking -> Magic SysRq Key Enabled + +Then:: + + echo "1" > /proc/sys/kernel/sysrq + +also type:: + + echo "8" >/proc/sys/kernel/printk + +To make printk output go to console. + +On 390 all commands are prefixed with:: + + ^- + +e.g.:: + + ^-t will show tasks. + ^-? or some unknown command will display help. + +The sysrq key reading is very picky ( I have to type the keys in an +xterm session & paste them into the x3270 console ) +& it may be wise to predefine the keys as described in the VM hints above + +This is particularly useful for syncing disks unmounting & rebooting +if the machine gets partially hung. + +Read Documentation/admin-guide/sysrq.rst for more info + +References: +=========== +- Enterprise Systems Architecture Reference Summary +- Enterprise Systems Architecture Principles of Operation +- Hartmut Penners s390 stack frame sheet. +- IBM Mainframe Channel Attachment a technology brief from a CISCO webpage +- Various bits of man & info pages of Linux. +- Linux & GDB source. +- Various info & man pages. +- CMS Help on tracing commands. +- Linux for s/390 Elf Application Binary Interface +- Linux for z/Series Elf Application Binary Interface ( Both Highly Recommended ) +- z/Architecture Principles of Operation SA22-7832-00 +- Enterprise Systems Architecture/390 Reference Summary SA22-7209-01 & the +- Enterprise Systems Architecture/390 Principles of Operation SA22-7201-05 + +Special Thanks +============== +Special thanks to Neale Ferguson who maintains a much +prettier HTML version of this page at +http://linuxvm.org/penguinvm/ +Bob Grainger Stefan Bader & others for reporting bugs diff --git a/Documentation/s390/driver-model.txt b/Documentation/s390/driver-model.rst index ed265cf54cde..ad4bc2dbea43 100644 --- a/Documentation/s390/driver-model.txt +++ b/Documentation/s390/driver-model.rst @@ -1,5 +1,6 @@ +============================= S/390 driver model interfaces ------------------------------ +============================= 1. CCW devices -------------- @@ -7,13 +8,13 @@ S/390 driver model interfaces All devices which can be addressed by means of ccws are called 'CCW devices' - even if they aren't actually driven by ccws. -All ccw devices are accessed via a subchannel, this is reflected in the -structures under devices/: +All ccw devices are accessed via a subchannel, this is reflected in the +structures under devices/:: -devices/ + devices/ - system/ - css0/ - - 0.0.0000/0.0.0815/ + - 0.0.0000/0.0.0815/ - 0.0.0001/0.0.4711/ - 0.0.0002/ - 0.1.0000/0.1.1234/ @@ -35,14 +36,18 @@ be found under bus/ccw/devices/. All ccw devices export some data via sysfs. -cutype: The control unit type / model. +cutype: + The control unit type / model. -devtype: The device type / model, if applicable. +devtype: + The device type / model, if applicable. -availability: Can be 'good' or 'boxed'; 'no path' or 'no device' for +availability: + Can be 'good' or 'boxed'; 'no path' or 'no device' for disconnected devices. -online: An interface to set the device online and offline. +online: + An interface to set the device online and offline. In the special case of the device being disconnected (see the notify function under 1.2), piping 0 to online will forcibly delete the device. @@ -52,9 +57,11 @@ The device drivers can add entries to export per-device data and interfaces. There is also some data exported on a per-subchannel basis (see under bus/css/devices/): -chpids: Via which chpids the device is connected. +chpids: + Via which chpids the device is connected. -pimpampom: The path installed, path available and path operational masks. +pimpampom: + The path installed, path available and path operational masks. There also might be additional data, for example for block devices. @@ -74,77 +81,93 @@ b. After a. has been performed, if necessary, the device is finally brought up ------------------------------------ The basic struct ccw_device and struct ccw_driver data structures can be found -under include/asm/ccwdev.h. +under include/asm/ccwdev.h:: -struct ccw_device { - spinlock_t *ccwlock; - struct ccw_device_private *private; - struct ccw_device_id id; + struct ccw_device { + spinlock_t *ccwlock; + struct ccw_device_private *private; + struct ccw_device_id id; - struct ccw_driver *drv; - struct device dev; + struct ccw_driver *drv; + struct device dev; int online; void (*handler) (struct ccw_device *dev, unsigned long intparm, - struct irb *irb); -}; + struct irb *irb); + }; -struct ccw_driver { - struct module *owner; - struct ccw_device_id *ids; - int (*probe) (struct ccw_device *); + struct ccw_driver { + struct module *owner; + struct ccw_device_id *ids; + int (*probe) (struct ccw_device *); int (*remove) (struct ccw_device *); int (*set_online) (struct ccw_device *); int (*set_offline) (struct ccw_device *); int (*notify) (struct ccw_device *, int); struct device_driver driver; char *name; -}; + }; The 'private' field contains data needed for internal i/o operation only, and is not available to the device driver. Each driver should declare in a MODULE_DEVICE_TABLE into which CU types/models and/or device types/models it is interested. This information can later be found -in the struct ccw_device_id fields: +in the struct ccw_device_id fields:: -struct ccw_device_id { - __u16 match_flags; + struct ccw_device_id { + __u16 match_flags; - __u16 cu_type; - __u16 dev_type; - __u8 cu_model; - __u8 dev_model; + __u16 cu_type; + __u16 dev_type; + __u8 cu_model; + __u8 dev_model; unsigned long driver_info; -}; + }; The functions in ccw_driver should be used in the following way: -probe: This function is called by the device layer for each device the driver + +probe: + This function is called by the device layer for each device the driver is interested in. The driver should only allocate private structures to put in dev->driver_data and create attributes (if needed). Also, the interrupt handler (see below) should be set here. -int (*probe) (struct ccw_device *cdev); +:: + + int (*probe) (struct ccw_device *cdev); -Parameters: cdev - the device to be probed. +Parameters: + cdev + - the device to be probed. -remove: This function is called by the device layer upon removal of the driver, +remove: + This function is called by the device layer upon removal of the driver, the device or the module. The driver should perform cleanups here. -int (*remove) (struct ccw_device *cdev); +:: -Parameters: cdev - the device to be removed. + int (*remove) (struct ccw_device *cdev); +Parameters: + cdev + - the device to be removed. -set_online: This function is called by the common I/O layer when the device is + +set_online: + This function is called by the common I/O layer when the device is activated via the 'online' attribute. The driver should finally setup and activate the device here. -int (*set_online) (struct ccw_device *); +:: + + int (*set_online) (struct ccw_device *); -Parameters: cdev - the device to be activated. The common layer has +Parameters: + cdev + - the device to be activated. The common layer has verified that the device is not already online. @@ -152,15 +175,22 @@ set_offline: This function is called by the common I/O layer when the device is de-activated via the 'online' attribute. The driver should shut down the device, but not de-allocate its private data. -int (*set_offline) (struct ccw_device *); +:: -Parameters: cdev - the device to be deactivated. The common layer has + int (*set_offline) (struct ccw_device *); + +Parameters: + cdev + - the device to be deactivated. The common layer has verified that the device is online. -notify: This function is called by the common I/O layer for some state changes +notify: + This function is called by the common I/O layer for some state changes of the device. + Signalled to the driver are: + * In online state, device detached (CIO_GONE) or last path gone (CIO_NO_PATH). The driver must return !0 to keep the device; for return code 0, the device will be deleted as usual (also when no @@ -173,32 +203,40 @@ notify: This function is called by the common I/O layer for some state changes return code of the notify function the device driver signals if it wants the device back: !0 for keeping, 0 to make the device being removed and re-registered. - -int (*notify) (struct ccw_device *, int); -Parameters: cdev - the device whose state changed. - event - the event that happened. This can be one of CIO_GONE, - CIO_NO_PATH or CIO_OPER. +:: + + int (*notify) (struct ccw_device *, int); + +Parameters: + cdev + - the device whose state changed. + + event + - the event that happened. This can be one of CIO_GONE, + CIO_NO_PATH or CIO_OPER. The handler field of the struct ccw_device is meant to be set to the interrupt -handler for the device. In order to accommodate drivers which use several +handler for the device. In order to accommodate drivers which use several distinct handlers (e.g. multi subchannel devices), this is a member of ccw_device instead of ccw_driver. The handler is registered with the common layer during set_online() processing before the driver is called, and is deregistered during set_offline() after the -driver has been called. Also, after registering / before deregistering, path +driver has been called. Also, after registering / before deregistering, path grouping resp. disbanding of the path group (if applicable) are performed. -void (*handler) (struct ccw_device *dev, unsigned long intparm, struct irb *irb); +:: -Parameters: dev - the device the handler is called for + void (*handler) (struct ccw_device *dev, unsigned long intparm, struct irb *irb); + +Parameters: dev - the device the handler is called for intparm - the intparm which allows the device driver to identify - the i/o the interrupt is associated with, or to recognize - the interrupt as unsolicited. - irb - interruption response block which contains the accumulated - status. + the i/o the interrupt is associated with, or to recognize + the interrupt as unsolicited. + irb - interruption response block which contains the accumulated + status. -The device driver is called from the common ccw_device layer and can retrieve +The device driver is called from the common ccw_device layer and can retrieve information about the interrupt from the irb parameter. @@ -237,23 +275,27 @@ only the logical state and not the physical state, since we cannot track the latter consistently due to lacking machine support (we don't need to be aware of it anyway). -status - Can be 'online' or 'offline'. +status + - Can be 'online' or 'offline'. Piping 'on' or 'off' sets the chpid logically online/offline. Piping 'on' to an online chpid triggers path reprobing for all devices the chpid connects to. This can be used to force the kernel to re-use a channel path the user knows to be online, but the machine hasn't created a machine check for. -type - The physical type of the channel path. +type + - The physical type of the channel path. -shared - Whether the channel path is shared. +shared + - Whether the channel path is shared. -cmg - The channel measurement group. +cmg + - The channel measurement group. 3. System devices ----------------- -3.1 xpram +3.1 xpram --------- xpram shows up under devices/system/ as 'xpram'. @@ -279,9 +321,8 @@ Netiucv connections show up under devices/iucv/ as "netiucv<ifnum>". The interfa number is assigned sequentially to the connections defined via the 'connection' attribute. -user - shows the connection partner. - -buffer - maximum buffer size. - Pipe to it to change buffer size. - +user + - shows the connection partner. +buffer + - maximum buffer size. Pipe to it to change buffer size. diff --git a/Documentation/s390/index.rst b/Documentation/s390/index.rst new file mode 100644 index 000000000000..1a914da2a07b --- /dev/null +++ b/Documentation/s390/index.rst @@ -0,0 +1,30 @@ +:orphan: + +================= +s390 Architecture +================= + +.. toctree:: + :maxdepth: 1 + + cds + 3270 + debugging390 + driver-model + monreader + qeth + s390dbf + vfio-ap + vfio-ccw + zfcpdump + dasd + common_io + + text_files + +.. only:: subproject and html + + Indices + ======= + + * :ref:`genindex` diff --git a/Documentation/s390/monreader.txt b/Documentation/s390/monreader.rst index d3729585fdb0..1e857575c113 100644 --- a/Documentation/s390/monreader.txt +++ b/Documentation/s390/monreader.rst @@ -1,24 +1,26 @@ +================================================= +Linux API for read access to z/VM Monitor Records +================================================= Date : 2004-Nov-26 + Author: Gerald Schaefer (geraldsc@de.ibm.com) - Linux API for read access to z/VM Monitor Records - ================================================= Description =========== This item delivers a new Linux API in the form of a misc char device that is usable from user space and allows read access to the z/VM Monitor Records -collected by the *MONITOR System Service of z/VM. +collected by the `*MONITOR` System Service of z/VM. User Requirements ================= The z/VM guest on which you want to access this API needs to be configured in -order to allow IUCV connections to the *MONITOR service, i.e. it needs the -IUCV *MONITOR statement in its user entry. If the monitor DCSS to be used is +order to allow IUCV connections to the `*MONITOR` service, i.e. it needs the +IUCV `*MONITOR` statement in its user entry. If the monitor DCSS to be used is restricted (likely), you also need the NAMESAVE <DCSS NAME> statement. This item will use the IUCV device driver to access the z/VM services, so you need a kernel with IUCV support. You also need z/VM version 4.4 or 5.1. @@ -50,7 +52,9 @@ Your guest virtual storage has to end below the starting address of the DCSS and you have to specify the "mem=" kernel parameter in your parmfile with a value greater than the ending address of the DCSS. -Example: DEF STOR 140M +Example:: + + DEF STOR 140M This defines 140MB storage size for your guest, the parameter "mem=160M" is added to the parmfile. @@ -66,24 +70,27 @@ kernel, the kernel parameter "monreader.mondcss=<DCSS NAME>" can be specified in the parmfile. The default name for the DCSS is "MONDCSS" if none is specified. In case that -there are other users already connected to the *MONITOR service (e.g. +there are other users already connected to the `*MONITOR` service (e.g. Performance Toolkit), the monitor DCSS is already defined and you have to use the same DCSS. The CP command Q MONITOR (Class E privileged) shows the name of the monitor DCSS, if already defined, and the users connected to the -*MONITOR service. +`*MONITOR` service. Refer to the "z/VM Performance" book (SC24-6109-00) on how to create a monitor DCSS if your z/VM doesn't have one already, you need Class E privileges to define and save a DCSS. Example: -------- -modprobe monreader mondcss=MYDCSS + +:: + + modprobe monreader mondcss=MYDCSS This loads the module and sets the DCSS name to "MYDCSS". NOTE: ----- -This API provides no interface to control the *MONITOR service, e.g. specify +This API provides no interface to control the `*MONITOR` service, e.g. specify which data should be collected. This can be done by the CP command MONITOR (Class E privileged), see "CP Command and Utility Reference". @@ -98,6 +105,7 @@ If your distribution does not support udev, a device node will not be created automatically and you have to create it manually after loading the module. Therefore you need to know the major and minor numbers of the device. These numbers can be found in /sys/class/misc/monreader/dev. + Typing cat /sys/class/misc/monreader/dev will give an output of the form <major>:<minor>. The device node can be created via the mknod command, enter mknod <name> c <major> <minor>, where <name> is the name of the device node @@ -105,10 +113,13 @@ to be created. Example: -------- -# modprobe monreader -# cat /sys/class/misc/monreader/dev -10:63 -# mknod /dev/monreader c 10 63 + +:: + + # modprobe monreader + # cat /sys/class/misc/monreader/dev + 10:63 + # mknod /dev/monreader c 10 63 This loads the module with the default monitor DCSS (MONDCSS) and creates a device node. @@ -133,20 +144,21 @@ last byte of data. The start address is needed to handle "end-of-frame" records correctly (domain 1, record 13), i.e. it can be used to determine the record start offset relative to a 4K page (frame) boundary. -See "Appendix A: *MONITOR" in the "z/VM Performance" document for a description +See "Appendix A: `*MONITOR`" in the "z/VM Performance" document for a description of the monitor control element layout. The layout of the monitor records can be found here (z/VM 5.1): http://www.vm.ibm.com/pubs/mon510/index.html -The layout of the data stream provided by the monreader device is as follows: -... -<0 byte read> -<first MCE> \ -<first set of records> | -... |- data set -<last MCE> | -<last set of records> / -<0 byte read> -... +The layout of the data stream provided by the monreader device is as follows:: + + ... + <0 byte read> + <first MCE> \ + <first set of records> | + ... |- data set + <last MCE> | + <last set of records> / + <0 byte read> + ... There may be more than one combination of MCE and corresponding record set within one data set and the end of each data set is indicated by a successful @@ -165,15 +177,19 @@ As with most char devices, error conditions are indicated by returning a negative value for the number of bytes read. In this case, the errno variable indicates the error condition: -EIO: reply failed, read data is invalid and the application +EIO: + reply failed, read data is invalid and the application should discard the data read since the last successful read with 0 size. -EFAULT: copy_to_user failed, read data is invalid and the application should - discard the data read since the last successful read with 0 size. -EAGAIN: occurs on a non-blocking read if there is no data available at the - moment. There is no data missing or corrupted, just try again or rather - use polling for non-blocking reads. -EOVERFLOW: message limit reached, the data read since the last successful - read with 0 size is valid but subsequent records may be missing. +EFAULT: + copy_to_user failed, read data is invalid and the application should + discard the data read since the last successful read with 0 size. +EAGAIN: + occurs on a non-blocking read if there is no data available at the + moment. There is no data missing or corrupted, just try again or rather + use polling for non-blocking reads. +EOVERFLOW: + message limit reached, the data read since the last successful + read with 0 size is valid but subsequent records may be missing. In the last case (EOVERFLOW) there may be missing data, in the first two cases (EIO, EFAULT) there will be missing data. It's up to the application if it will @@ -183,7 +199,7 @@ Open: ----- Only one user is allowed to open the char device. If it is already in use, the open function will fail (return a negative value) and set errno to EBUSY. -The open function may also fail if an IUCV connection to the *MONITOR service +The open function may also fail if an IUCV connection to the `*MONITOR` service cannot be established. In this case errno will be set to EIO and an error message with an IPUSER SEVER code will be printed into syslog. The IPUSER SEVER codes are described in the "z/VM Performance" book, Appendix A. @@ -194,4 +210,3 @@ As soon as the device is opened, incoming messages will be accepted and they will account for the message limit, i.e. opening the device without reading from it will provoke the "message limit reached" error (EOVERFLOW error code) eventually. - diff --git a/Documentation/s390/qeth.txt b/Documentation/s390/qeth.rst index aa06fcf5f8c2..f02fdaa68de0 100644 --- a/Documentation/s390/qeth.txt +++ b/Documentation/s390/qeth.rst @@ -1,8 +1,12 @@ +============================= IBM s390 QDIO Ethernet Driver +============================= OSA and HiperSockets Bridge Port Support +======================================== Uevents +------- To generate the events the device must be assigned a role of either a primary or a secondary Bridge Port. For more information, see @@ -13,12 +17,15 @@ of some configured Bridge Port device on the channel changes, a udev event with ACTION=CHANGE is emitted on behalf of the corresponding ccwgroup device. The event has the following attributes: -BRIDGEPORT=statechange - indicates that the Bridge Port device changed +BRIDGEPORT=statechange + indicates that the Bridge Port device changed its state. -ROLE={primary|secondary|none} - the role assigned to the port. +ROLE={primary|secondary|none} + the role assigned to the port. -STATE={active|standby|inactive} - the newly assumed state of the port. +STATE={active|standby|inactive} + the newly assumed state of the port. When run on HiperSockets Bridge Capable Port hardware with host address notifications enabled, a udev event with ACTION=CHANGE is emitted. @@ -26,25 +33,32 @@ It is emitted on behalf of the corresponding ccwgroup device when a host or a VLAN is registered or unregistered on the network served by the device. The event has the following attributes: -BRIDGEDHOST={reset|register|deregister|abort} - host address +BRIDGEDHOST={reset|register|deregister|abort} + host address notifications are started afresh, a new host or VLAN is registered or deregistered on the Bridge Port HiperSockets channel, or address notifications are aborted. -VLAN=numeric-vlan-id - VLAN ID on which the event occurred. Not included +VLAN=numeric-vlan-id + VLAN ID on which the event occurred. Not included if no VLAN is involved in the event. -MAC=xx:xx:xx:xx:xx:xx - MAC address of the host that is being registered +MAC=xx:xx:xx:xx:xx:xx + MAC address of the host that is being registered or deregistered from the HiperSockets channel. Not reported if the event reports the creation or destruction of a VLAN. -NTOK_BUSID=x.y.zzzz - device bus ID (CSSID, SSID and device number). +NTOK_BUSID=x.y.zzzz + device bus ID (CSSID, SSID and device number). -NTOK_IID=xx - device IID. +NTOK_IID=xx + device IID. -NTOK_CHPID=xx - device CHPID. +NTOK_CHPID=xx + device CHPID. -NTOK_CHID=xxxx - device channel ID. +NTOK_CHID=xxxx + device channel ID. -Note that the NTOK_* attributes refer to devices other than the one +Note that the `NTOK_*` attributes refer to devices other than the one connected to the system on which the OS is running. diff --git a/Documentation/s390/s390dbf.rst b/Documentation/s390/s390dbf.rst new file mode 100644 index 000000000000..cdb36842b898 --- /dev/null +++ b/Documentation/s390/s390dbf.rst @@ -0,0 +1,487 @@ +================== +S390 Debug Feature +================== + +files: + - arch/s390/kernel/debug.c + - arch/s390/include/asm/debug.h + +Description: +------------ +The goal of this feature is to provide a kernel debug logging API +where log records can be stored efficiently in memory, where each component +(e.g. device drivers) can have one separate debug log. +One purpose of this is to inspect the debug logs after a production system crash +in order to analyze the reason for the crash. + +If the system still runs but only a subcomponent which uses dbf fails, +it is possible to look at the debug logs on a live system via the Linux +debugfs filesystem. + +The debug feature may also very useful for kernel and driver development. + +Design: +------- +Kernel components (e.g. device drivers) can register themselves at the debug +feature with the function call :c:func:`debug_register()`. +This function initializes a +debug log for the caller. For each debug log exists a number of debug areas +where exactly one is active at one time. Each debug area consists of contiguous +pages in memory. In the debug areas there are stored debug entries (log records) +which are written by event- and exception-calls. + +An event-call writes the specified debug entry to the active debug +area and updates the log pointer for the active area. If the end +of the active debug area is reached, a wrap around is done (ring buffer) +and the next debug entry will be written at the beginning of the active +debug area. + +An exception-call writes the specified debug entry to the log and +switches to the next debug area. This is done in order to be sure +that the records which describe the origin of the exception are not +overwritten when a wrap around for the current area occurs. + +The debug areas themselves are also ordered in form of a ring buffer. +When an exception is thrown in the last debug area, the following debug +entries are then written again in the very first area. + +There are four versions for the event- and exception-calls: One for +logging raw data, one for text, one for numbers (unsigned int and long), +and one for sprintf-like formatted strings. + +Each debug entry contains the following data: + +- Timestamp +- Cpu-Number of calling task +- Level of debug entry (0...6) +- Return Address to caller +- Flag, if entry is an exception or not + +The debug logs can be inspected in a live system through entries in +the debugfs-filesystem. Under the toplevel directory "``s390dbf``" there is +a directory for each registered component, which is named like the +corresponding component. The debugfs normally should be mounted to +``/sys/kernel/debug`` therefore the debug feature can be accessed under +``/sys/kernel/debug/s390dbf``. + +The content of the directories are files which represent different views +to the debug log. Each component can decide which views should be +used through registering them with the function :c:func:`debug_register_view()`. +Predefined views for hex/ascii, sprintf and raw binary data are provided. +It is also possible to define other views. The content of +a view can be inspected simply by reading the corresponding debugfs file. + +All debug logs have an actual debug level (range from 0 to 6). +The default level is 3. Event and Exception functions have a :c:data:`level` +parameter. Only debug entries with a level that is lower or equal +than the actual level are written to the log. This means, when +writing events, high priority log entries should have a low level +value whereas low priority entries should have a high one. +The actual debug level can be changed with the help of the debugfs-filesystem +through writing a number string "x" to the ``level`` debugfs file which is +provided for every debug log. Debugging can be switched off completely +by using "-" on the ``level`` debugfs file. + +Example:: + + > echo "-" > /sys/kernel/debug/s390dbf/dasd/level + +It is also possible to deactivate the debug feature globally for every +debug log. You can change the behavior using 2 sysctl parameters in +``/proc/sys/s390dbf``: + +There are currently 2 possible triggers, which stop the debug feature +globally. The first possibility is to use the ``debug_active`` sysctl. If +set to 1 the debug feature is running. If ``debug_active`` is set to 0 the +debug feature is turned off. + +The second trigger which stops the debug feature is a kernel oops. +That prevents the debug feature from overwriting debug information that +happened before the oops. After an oops you can reactivate the debug feature +by piping 1 to ``/proc/sys/s390dbf/debug_active``. Nevertheless, it's not +suggested to use an oopsed kernel in a production environment. + +If you want to disallow the deactivation of the debug feature, you can use +the ``debug_stoppable`` sysctl. If you set ``debug_stoppable`` to 0 the debug +feature cannot be stopped. If the debug feature is already stopped, it +will stay deactivated. + +Kernel Interfaces: +------------------ + +.. kernel-doc:: arch/s390/kernel/debug.c +.. kernel-doc:: arch/s390/include/asm/debug.h + +Predefined views: +----------------- + +.. code-block:: c + + extern struct debug_view debug_hex_ascii_view; + + extern struct debug_view debug_raw_view; + + extern struct debug_view debug_sprintf_view; + +Examples +-------- + +.. code-block:: c + + /* + * hex_ascii- + raw-view Example + */ + + #include <linux/init.h> + #include <asm/debug.h> + + static debug_info_t *debug_info; + + static int init(void) + { + /* register 4 debug areas with one page each and 4 byte data field */ + + debug_info = debug_register("test", 1, 4, 4 ); + debug_register_view(debug_info, &debug_hex_ascii_view); + debug_register_view(debug_info, &debug_raw_view); + + debug_text_event(debug_info, 4 , "one "); + debug_int_exception(debug_info, 4, 4711); + debug_event(debug_info, 3, &debug_info, 4); + + return 0; + } + + static void cleanup(void) + { + debug_unregister(debug_info); + } + + module_init(init); + module_exit(cleanup); + +.. code-block:: c + + /* + * sprintf-view Example + */ + + #include <linux/init.h> + #include <asm/debug.h> + + static debug_info_t *debug_info; + + static int init(void) + { + /* register 4 debug areas with one page each and data field for */ + /* format string pointer + 2 varargs (= 3 * sizeof(long)) */ + + debug_info = debug_register("test", 1, 4, sizeof(long) * 3); + debug_register_view(debug_info, &debug_sprintf_view); + + debug_sprintf_event(debug_info, 2 , "first event in %s:%i\n",__FILE__,__LINE__); + debug_sprintf_exception(debug_info, 1, "pointer to debug info: %p\n",&debug_info); + + return 0; + } + + static void cleanup(void) + { + debug_unregister(debug_info); + } + + module_init(init); + module_exit(cleanup); + +Debugfs Interface +----------------- +Views to the debug logs can be investigated through reading the corresponding +debugfs-files: + +Example:: + + > ls /sys/kernel/debug/s390dbf/dasd + flush hex_ascii level pages raw + > cat /sys/kernel/debug/s390dbf/dasd/hex_ascii | sort -k2,2 -s + 00 00974733272:680099 2 - 02 0006ad7e 07 ea 4a 90 | .... + 00 00974733272:682210 2 - 02 0006ade6 46 52 45 45 | FREE + 00 00974733272:682213 2 - 02 0006adf6 07 ea 4a 90 | .... + 00 00974733272:682281 1 * 02 0006ab08 41 4c 4c 43 | EXCP + 01 00974733272:682284 2 - 02 0006ab16 45 43 4b 44 | ECKD + 01 00974733272:682287 2 - 02 0006ab28 00 00 00 04 | .... + 01 00974733272:682289 2 - 02 0006ab3e 00 00 00 20 | ... + 01 00974733272:682297 2 - 02 0006ad7e 07 ea 4a 90 | .... + 01 00974733272:684384 2 - 00 0006ade6 46 52 45 45 | FREE + 01 00974733272:684388 2 - 00 0006adf6 07 ea 4a 90 | .... + +See section about predefined views for explanation of the above output! + +Changing the debug level +------------------------ + +Example:: + + + > cat /sys/kernel/debug/s390dbf/dasd/level + 3 + > echo "5" > /sys/kernel/debug/s390dbf/dasd/level + > cat /sys/kernel/debug/s390dbf/dasd/level + 5 + +Flushing debug areas +-------------------- +Debug areas can be flushed with piping the number of the desired +area (0...n) to the debugfs file "flush". When using "-" all debug areas +are flushed. + +Examples: + +1. Flush debug area 0:: + + > echo "0" > /sys/kernel/debug/s390dbf/dasd/flush + +2. Flush all debug areas:: + + > echo "-" > /sys/kernel/debug/s390dbf/dasd/flush + +Changing the size of debug areas +------------------------------------ +It is possible the change the size of debug areas through piping +the number of pages to the debugfs file "pages". The resize request will +also flush the debug areas. + +Example: + +Define 4 pages for the debug areas of debug feature "dasd":: + + > echo "4" > /sys/kernel/debug/s390dbf/dasd/pages + +Stopping the debug feature +-------------------------- +Example: + +1. Check if stopping is allowed:: + + > cat /proc/sys/s390dbf/debug_stoppable + +2. Stop debug feature:: + + > echo 0 > /proc/sys/s390dbf/debug_active + +crash Interface +---------------- +The ``crash`` tool since v5.1.0 has a built-in command +``s390dbf`` to display all the debug logs or export them to the file system. +With this tool it is possible +to investigate the debug logs on a live system and with a memory dump after +a system crash. + +Investigating raw memory +------------------------ +One last possibility to investigate the debug logs at a live +system and after a system crash is to look at the raw memory +under VM or at the Service Element. +It is possible to find the anchor of the debug-logs through +the ``debug_area_first`` symbol in the System map. Then one has +to follow the correct pointers of the data-structures defined +in debug.h and find the debug-areas in memory. +Normally modules which use the debug feature will also have +a global variable with the pointer to the debug-logs. Following +this pointer it will also be possible to find the debug logs in +memory. + +For this method it is recommended to use '16 * x + 4' byte (x = 0..n) +for the length of the data field in :c:func:`debug_register()` in +order to see the debug entries well formatted. + + +Predefined Views +---------------- + +There are three predefined views: hex_ascii, raw and sprintf. +The hex_ascii view shows the data field in hex and ascii representation +(e.g. ``45 43 4b 44 | ECKD``). +The raw view returns a bytestream as the debug areas are stored in memory. + +The sprintf view formats the debug entries in the same way as the sprintf +function would do. The sprintf event/exception functions write to the +debug entry a pointer to the format string (size = sizeof(long)) +and for each vararg a long value. So e.g. for a debug entry with a format +string plus two varargs one would need to allocate a (3 * sizeof(long)) +byte data area in the debug_register() function. + +IMPORTANT: + Using "%s" in sprintf event functions is dangerous. You can only + use "%s" in the sprintf event functions, if the memory for the passed string + is available as long as the debug feature exists. The reason behind this is + that due to performance considerations only a pointer to the string is stored + in the debug feature. If you log a string that is freed afterwards, you will + get an OOPS when inspecting the debug feature, because then the debug feature + will access the already freed memory. + +NOTE: + If using the sprintf view do NOT use other event/exception functions + than the sprintf-event and -exception functions. + +The format of the hex_ascii and sprintf view is as follows: + +- Number of area +- Timestamp (formatted as seconds and microseconds since 00:00:00 Coordinated + Universal Time (UTC), January 1, 1970) +- level of debug entry +- Exception flag (* = Exception) +- Cpu-Number of calling task +- Return Address to caller +- data field + +The format of the raw view is: + +- Header as described in debug.h +- datafield + +A typical line of the hex_ascii view will look like the following (first line +is only for explanation and will not be displayed when 'cating' the view):: + + area time level exception cpu caller data (hex + ascii) + -------------------------------------------------------------------------- + 00 00964419409:440690 1 - 00 88023fe + + +Defining views +-------------- + +Views are specified with the 'debug_view' structure. There are defined +callback functions which are used for reading and writing the debugfs files: + +.. code-block:: c + + struct debug_view { + char name[DEBUG_MAX_PROCF_LEN]; + debug_prolog_proc_t* prolog_proc; + debug_header_proc_t* header_proc; + debug_format_proc_t* format_proc; + debug_input_proc_t* input_proc; + void* private_data; + }; + +where: + +.. code-block:: c + + typedef int (debug_header_proc_t) (debug_info_t* id, + struct debug_view* view, + int area, + debug_entry_t* entry, + char* out_buf); + + typedef int (debug_format_proc_t) (debug_info_t* id, + struct debug_view* view, char* out_buf, + const char* in_buf); + typedef int (debug_prolog_proc_t) (debug_info_t* id, + struct debug_view* view, + char* out_buf); + typedef int (debug_input_proc_t) (debug_info_t* id, + struct debug_view* view, + struct file* file, const char* user_buf, + size_t in_buf_size, loff_t* offset); + + +The "private_data" member can be used as pointer to view specific data. +It is not used by the debug feature itself. + +The output when reading a debugfs file is structured like this:: + + "prolog_proc output" + + "header_proc output 1" "format_proc output 1" + "header_proc output 2" "format_proc output 2" + "header_proc output 3" "format_proc output 3" + ... + +When a view is read from the debugfs, the Debug Feature calls the +'prolog_proc' once for writing the prolog. +Then 'header_proc' and 'format_proc' are called for each +existing debug entry. + +The input_proc can be used to implement functionality when it is written to +the view (e.g. like with ``echo "0" > /sys/kernel/debug/s390dbf/dasd/level``). + +For header_proc there can be used the default function +:c:func:`debug_dflt_header_fn()` which is defined in debug.h. +and which produces the same header output as the predefined views. +E.g:: + + 00 00964419409:440761 2 - 00 88023ec + +In order to see how to use the callback functions check the implementation +of the default views! + +Example: + +.. code-block:: c + + #include <asm/debug.h> + + #define UNKNOWNSTR "data: %08x" + + const char* messages[] = + {"This error...........\n", + "That error...........\n", + "Problem..............\n", + "Something went wrong.\n", + "Everything ok........\n", + NULL + }; + + static int debug_test_format_fn( + debug_info_t *id, struct debug_view *view, + char *out_buf, const char *in_buf + ) + { + int i, rc = 0; + + if (id->buf_size >= 4) { + int msg_nr = *((int*)in_buf); + if (msg_nr < sizeof(messages) / sizeof(char*) - 1) + rc += sprintf(out_buf, "%s", messages[msg_nr]); + else + rc += sprintf(out_buf, UNKNOWNSTR, msg_nr); + } + return rc; + } + + struct debug_view debug_test_view = { + "myview", /* name of view */ + NULL, /* no prolog */ + &debug_dflt_header_fn, /* default header for each entry */ + &debug_test_format_fn, /* our own format function */ + NULL, /* no input function */ + NULL /* no private data */ + }; + +test: +===== + +.. code-block:: c + + debug_info_t *debug_info; + int i; + ... + debug_info = debug_register("test", 0, 4, 4); + debug_register_view(debug_info, &debug_test_view); + for (i = 0; i < 10; i ++) + debug_int_event(debug_info, 1, i); + +:: + + > cat /sys/kernel/debug/s390dbf/test/myview + 00 00964419734:611402 1 - 00 88042ca This error........... + 00 00964419734:611405 1 - 00 88042ca That error........... + 00 00964419734:611408 1 - 00 88042ca Problem.............. + 00 00964419734:611411 1 - 00 88042ca Something went wrong. + 00 00964419734:611414 1 - 00 88042ca Everything ok........ + 00 00964419734:611417 1 - 00 88042ca data: 00000005 + 00 00964419734:611419 1 - 00 88042ca data: 00000006 + 00 00964419734:611422 1 - 00 88042ca data: 00000007 + 00 00964419734:611425 1 - 00 88042ca data: 00000008 + 00 00964419734:611428 1 - 00 88042ca data: 00000009 diff --git a/Documentation/s390/s390dbf.txt b/Documentation/s390/s390dbf.txt deleted file mode 100644 index 61329fd62e89..000000000000 --- a/Documentation/s390/s390dbf.txt +++ /dev/null @@ -1,667 +0,0 @@ -S390 Debug Feature -================== - -files: arch/s390/kernel/debug.c - arch/s390/include/asm/debug.h - -Description: ------------- -The goal of this feature is to provide a kernel debug logging API -where log records can be stored efficiently in memory, where each component -(e.g. device drivers) can have one separate debug log. -One purpose of this is to inspect the debug logs after a production system crash -in order to analyze the reason for the crash. -If the system still runs but only a subcomponent which uses dbf fails, -it is possible to look at the debug logs on a live system via the Linux -debugfs filesystem. -The debug feature may also very useful for kernel and driver development. - -Design: -------- -Kernel components (e.g. device drivers) can register themselves at the debug -feature with the function call debug_register(). This function initializes a -debug log for the caller. For each debug log exists a number of debug areas -where exactly one is active at one time. Each debug area consists of contiguous -pages in memory. In the debug areas there are stored debug entries (log records) -which are written by event- and exception-calls. - -An event-call writes the specified debug entry to the active debug -area and updates the log pointer for the active area. If the end -of the active debug area is reached, a wrap around is done (ring buffer) -and the next debug entry will be written at the beginning of the active -debug area. - -An exception-call writes the specified debug entry to the log and -switches to the next debug area. This is done in order to be sure -that the records which describe the origin of the exception are not -overwritten when a wrap around for the current area occurs. - -The debug areas themselves are also ordered in form of a ring buffer. -When an exception is thrown in the last debug area, the following debug -entries are then written again in the very first area. - -There are three versions for the event- and exception-calls: One for -logging raw data, one for text and one for numbers. - -Each debug entry contains the following data: - -- Timestamp -- Cpu-Number of calling task -- Level of debug entry (0...6) -- Return Address to caller -- Flag, if entry is an exception or not - -The debug logs can be inspected in a live system through entries in -the debugfs-filesystem. Under the toplevel directory "s390dbf" there is -a directory for each registered component, which is named like the -corresponding component. The debugfs normally should be mounted to -/sys/kernel/debug therefore the debug feature can be accessed under -/sys/kernel/debug/s390dbf. - -The content of the directories are files which represent different views -to the debug log. Each component can decide which views should be -used through registering them with the function debug_register_view(). -Predefined views for hex/ascii, sprintf and raw binary data are provided. -It is also possible to define other views. The content of -a view can be inspected simply by reading the corresponding debugfs file. - -All debug logs have an actual debug level (range from 0 to 6). -The default level is 3. Event and Exception functions have a 'level' -parameter. Only debug entries with a level that is lower or equal -than the actual level are written to the log. This means, when -writing events, high priority log entries should have a low level -value whereas low priority entries should have a high one. -The actual debug level can be changed with the help of the debugfs-filesystem -through writing a number string "x" to the 'level' debugfs file which is -provided for every debug log. Debugging can be switched off completely -by using "-" on the 'level' debugfs file. - -Example: - -> echo "-" > /sys/kernel/debug/s390dbf/dasd/level - -It is also possible to deactivate the debug feature globally for every -debug log. You can change the behavior using 2 sysctl parameters in -/proc/sys/s390dbf: -There are currently 2 possible triggers, which stop the debug feature -globally. The first possibility is to use the "debug_active" sysctl. If -set to 1 the debug feature is running. If "debug_active" is set to 0 the -debug feature is turned off. -The second trigger which stops the debug feature is a kernel oops. -That prevents the debug feature from overwriting debug information that -happened before the oops. After an oops you can reactivate the debug feature -by piping 1 to /proc/sys/s390dbf/debug_active. Nevertheless, its not -suggested to use an oopsed kernel in a production environment. -If you want to disallow the deactivation of the debug feature, you can use -the "debug_stoppable" sysctl. If you set "debug_stoppable" to 0 the debug -feature cannot be stopped. If the debug feature is already stopped, it -will stay deactivated. - -Kernel Interfaces: ------------------- - ----------------------------------------------------------------------------- -debug_info_t *debug_register(char *name, int pages, int nr_areas, - int buf_size); - -Parameter: name: Name of debug log (e.g. used for debugfs entry) - pages: number of pages, which will be allocated per area - nr_areas: number of debug areas - buf_size: size of data area in each debug entry - -Return Value: Handle for generated debug area - NULL if register failed - -Description: Allocates memory for a debug log - Must not be called within an interrupt handler - ----------------------------------------------------------------------------- -debug_info_t *debug_register_mode(char *name, int pages, int nr_areas, - int buf_size, mode_t mode, uid_t uid, - gid_t gid); - -Parameter: name: Name of debug log (e.g. used for debugfs entry) - pages: Number of pages, which will be allocated per area - nr_areas: Number of debug areas - buf_size: Size of data area in each debug entry - mode: File mode for debugfs files. E.g. S_IRWXUGO - uid: User ID for debugfs files. Currently only 0 is - supported. - gid: Group ID for debugfs files. Currently only 0 is - supported. - -Return Value: Handle for generated debug area - NULL if register failed - -Description: Allocates memory for a debug log - Must not be called within an interrupt handler - ---------------------------------------------------------------------------- -void debug_unregister (debug_info_t * id); - -Parameter: id: handle for debug log - -Return Value: none - -Description: frees memory for a debug log and removes all registered debug - views. - Must not be called within an interrupt handler - ---------------------------------------------------------------------------- -void debug_set_level (debug_info_t * id, int new_level); - -Parameter: id: handle for debug log - new_level: new debug level - -Return Value: none - -Description: Sets new actual debug level if new_level is valid. - ---------------------------------------------------------------------------- -bool debug_level_enabled (debug_info_t * id, int level); - -Parameter: id: handle for debug log - level: debug level - -Return Value: True if level is less or equal to the current debug level. - -Description: Returns true if debug events for the specified level would be - logged. Otherwise returns false. ---------------------------------------------------------------------------- -void debug_stop_all(void); - -Parameter: none - -Return Value: none - -Description: stops the debug feature if stopping is allowed. Currently - used in case of a kernel oops. - ---------------------------------------------------------------------------- -debug_entry_t* debug_event (debug_info_t* id, int level, void* data, - int length); - -Parameter: id: handle for debug log - level: debug level - data: pointer to data for debug entry - length: length of data in bytes - -Return Value: Address of written debug entry - -Description: writes debug entry to active debug area (if level <= actual - debug level) - ---------------------------------------------------------------------------- -debug_entry_t* debug_int_event (debug_info_t * id, int level, - unsigned int data); -debug_entry_t* debug_long_event(debug_info_t * id, int level, - unsigned long data); - -Parameter: id: handle for debug log - level: debug level - data: integer value for debug entry - -Return Value: Address of written debug entry - -Description: writes debug entry to active debug area (if level <= actual - debug level) - ---------------------------------------------------------------------------- -debug_entry_t* debug_text_event (debug_info_t * id, int level, - const char* data); - -Parameter: id: handle for debug log - level: debug level - data: string for debug entry - -Return Value: Address of written debug entry - -Description: writes debug entry in ascii format to active debug area - (if level <= actual debug level) - ---------------------------------------------------------------------------- -debug_entry_t* debug_sprintf_event (debug_info_t * id, int level, - char* string,...); - -Parameter: id: handle for debug log - level: debug level - string: format string for debug entry - ...: varargs used as in sprintf() - -Return Value: Address of written debug entry - -Description: writes debug entry with format string and varargs (longs) to - active debug area (if level $<=$ actual debug level). - floats and long long datatypes cannot be used as varargs. - ---------------------------------------------------------------------------- - -debug_entry_t* debug_exception (debug_info_t* id, int level, void* data, - int length); - -Parameter: id: handle for debug log - level: debug level - data: pointer to data for debug entry - length: length of data in bytes - -Return Value: Address of written debug entry - -Description: writes debug entry to active debug area (if level <= actual - debug level) and switches to next debug area - ---------------------------------------------------------------------------- -debug_entry_t* debug_int_exception (debug_info_t * id, int level, - unsigned int data); -debug_entry_t* debug_long_exception(debug_info_t * id, int level, - unsigned long data); - -Parameter: id: handle for debug log - level: debug level - data: integer value for debug entry - -Return Value: Address of written debug entry - -Description: writes debug entry to active debug area (if level <= actual - debug level) and switches to next debug area - ---------------------------------------------------------------------------- -debug_entry_t* debug_text_exception (debug_info_t * id, int level, - const char* data); - -Parameter: id: handle for debug log - level: debug level - data: string for debug entry - -Return Value: Address of written debug entry - -Description: writes debug entry in ascii format to active debug area - (if level <= actual debug level) and switches to next debug - area - ---------------------------------------------------------------------------- -debug_entry_t* debug_sprintf_exception (debug_info_t * id, int level, - char* string,...); - -Parameter: id: handle for debug log - level: debug level - string: format string for debug entry - ...: varargs used as in sprintf() - -Return Value: Address of written debug entry - -Description: writes debug entry with format string and varargs (longs) to - active debug area (if level $<=$ actual debug level) and - switches to next debug area. - floats and long long datatypes cannot be used as varargs. - ---------------------------------------------------------------------------- - -int debug_register_view (debug_info_t * id, struct debug_view *view); - -Parameter: id: handle for debug log - view: pointer to debug view struct - -Return Value: 0 : ok - < 0: Error - -Description: registers new debug view and creates debugfs dir entry - ---------------------------------------------------------------------------- -int debug_unregister_view (debug_info_t * id, struct debug_view *view); - -Parameter: id: handle for debug log - view: pointer to debug view struct - -Return Value: 0 : ok - < 0: Error - -Description: unregisters debug view and removes debugfs dir entry - - - -Predefined views: ------------------ - -extern struct debug_view debug_hex_ascii_view; -extern struct debug_view debug_raw_view; -extern struct debug_view debug_sprintf_view; - -Examples --------- - -/* - * hex_ascii- + raw-view Example - */ - -#include <linux/init.h> -#include <asm/debug.h> - -static debug_info_t* debug_info; - -static int init(void) -{ - /* register 4 debug areas with one page each and 4 byte data field */ - - debug_info = debug_register ("test", 1, 4, 4 ); - debug_register_view(debug_info,&debug_hex_ascii_view); - debug_register_view(debug_info,&debug_raw_view); - - debug_text_event(debug_info, 4 , "one "); - debug_int_exception(debug_info, 4, 4711); - debug_event(debug_info, 3, &debug_info, 4); - - return 0; -} - -static void cleanup(void) -{ - debug_unregister (debug_info); -} - -module_init(init); -module_exit(cleanup); - ---------------------------------------------------------------------------- - -/* - * sprintf-view Example - */ - -#include <linux/init.h> -#include <asm/debug.h> - -static debug_info_t* debug_info; - -static int init(void) -{ - /* register 4 debug areas with one page each and data field for */ - /* format string pointer + 2 varargs (= 3 * sizeof(long)) */ - - debug_info = debug_register ("test", 1, 4, sizeof(long) * 3); - debug_register_view(debug_info,&debug_sprintf_view); - - debug_sprintf_event(debug_info, 2 , "first event in %s:%i\n",__FILE__,__LINE__); - debug_sprintf_exception(debug_info, 1, "pointer to debug info: %p\n",&debug_info); - - return 0; -} - -static void cleanup(void) -{ - debug_unregister (debug_info); -} - -module_init(init); -module_exit(cleanup); - - - -Debugfs Interface ----------------- -Views to the debug logs can be investigated through reading the corresponding -debugfs-files: - -Example: - -> ls /sys/kernel/debug/s390dbf/dasd -flush hex_ascii level pages raw -> cat /sys/kernel/debug/s390dbf/dasd/hex_ascii | sort -k2,2 -s -00 00974733272:680099 2 - 02 0006ad7e 07 ea 4a 90 | .... -00 00974733272:682210 2 - 02 0006ade6 46 52 45 45 | FREE -00 00974733272:682213 2 - 02 0006adf6 07 ea 4a 90 | .... -00 00974733272:682281 1 * 02 0006ab08 41 4c 4c 43 | EXCP -01 00974733272:682284 2 - 02 0006ab16 45 43 4b 44 | ECKD -01 00974733272:682287 2 - 02 0006ab28 00 00 00 04 | .... -01 00974733272:682289 2 - 02 0006ab3e 00 00 00 20 | ... -01 00974733272:682297 2 - 02 0006ad7e 07 ea 4a 90 | .... -01 00974733272:684384 2 - 00 0006ade6 46 52 45 45 | FREE -01 00974733272:684388 2 - 00 0006adf6 07 ea 4a 90 | .... - -See section about predefined views for explanation of the above output! - -Changing the debug level ------------------------- - -Example: - - -> cat /sys/kernel/debug/s390dbf/dasd/level -3 -> echo "5" > /sys/kernel/debug/s390dbf/dasd/level -> cat /sys/kernel/debug/s390dbf/dasd/level -5 - -Flushing debug areas --------------------- -Debug areas can be flushed with piping the number of the desired -area (0...n) to the debugfs file "flush". When using "-" all debug areas -are flushed. - -Examples: - -1. Flush debug area 0: -> echo "0" > /sys/kernel/debug/s390dbf/dasd/flush - -2. Flush all debug areas: -> echo "-" > /sys/kernel/debug/s390dbf/dasd/flush - -Changing the size of debug areas ------------------------------------- -It is possible the change the size of debug areas through piping -the number of pages to the debugfs file "pages". The resize request will -also flush the debug areas. - -Example: - -Define 4 pages for the debug areas of debug feature "dasd": -> echo "4" > /sys/kernel/debug/s390dbf/dasd/pages - -Stooping the debug feature --------------------------- -Example: - -1. Check if stopping is allowed -> cat /proc/sys/s390dbf/debug_stoppable -2. Stop debug feature -> echo 0 > /proc/sys/s390dbf/debug_active - -lcrash Interface ----------------- -It is planned that the dump analysis tool lcrash gets an additional command -'s390dbf' to display all the debug logs. With this tool it will be possible -to investigate the debug logs on a live system and with a memory dump after -a system crash. - -Investigating raw memory ------------------------- -One last possibility to investigate the debug logs at a live -system and after a system crash is to look at the raw memory -under VM or at the Service Element. -It is possible to find the anker of the debug-logs through -the 'debug_area_first' symbol in the System map. Then one has -to follow the correct pointers of the data-structures defined -in debug.h and find the debug-areas in memory. -Normally modules which use the debug feature will also have -a global variable with the pointer to the debug-logs. Following -this pointer it will also be possible to find the debug logs in -memory. - -For this method it is recommended to use '16 * x + 4' byte (x = 0..n) -for the length of the data field in debug_register() in -order to see the debug entries well formatted. - - -Predefined Views ----------------- - -There are three predefined views: hex_ascii, raw and sprintf. -The hex_ascii view shows the data field in hex and ascii representation -(e.g. '45 43 4b 44 | ECKD'). -The raw view returns a bytestream as the debug areas are stored in memory. - -The sprintf view formats the debug entries in the same way as the sprintf -function would do. The sprintf event/exception functions write to the -debug entry a pointer to the format string (size = sizeof(long)) -and for each vararg a long value. So e.g. for a debug entry with a format -string plus two varargs one would need to allocate a (3 * sizeof(long)) -byte data area in the debug_register() function. - -IMPORTANT: Using "%s" in sprintf event functions is dangerous. You can only -use "%s" in the sprintf event functions, if the memory for the passed string is -available as long as the debug feature exists. The reason behind this is that -due to performance considerations only a pointer to the string is stored in -the debug feature. If you log a string that is freed afterwards, you will get -an OOPS when inspecting the debug feature, because then the debug feature will -access the already freed memory. - -NOTE: If using the sprintf view do NOT use other event/exception functions -than the sprintf-event and -exception functions. - -The format of the hex_ascii and sprintf view is as follows: -- Number of area -- Timestamp (formatted as seconds and microseconds since 00:00:00 Coordinated - Universal Time (UTC), January 1, 1970) -- level of debug entry -- Exception flag (* = Exception) -- Cpu-Number of calling task -- Return Address to caller -- data field - -The format of the raw view is: -- Header as described in debug.h -- datafield - -A typical line of the hex_ascii view will look like the following (first line -is only for explanation and will not be displayed when 'cating' the view): - -area time level exception cpu caller data (hex + ascii) --------------------------------------------------------------------------- -00 00964419409:440690 1 - 00 88023fe - - -Defining views --------------- - -Views are specified with the 'debug_view' structure. There are defined -callback functions which are used for reading and writing the debugfs files: - -struct debug_view { - char name[DEBUG_MAX_PROCF_LEN]; - debug_prolog_proc_t* prolog_proc; - debug_header_proc_t* header_proc; - debug_format_proc_t* format_proc; - debug_input_proc_t* input_proc; - void* private_data; -}; - -where - -typedef int (debug_header_proc_t) (debug_info_t* id, - struct debug_view* view, - int area, - debug_entry_t* entry, - char* out_buf); - -typedef int (debug_format_proc_t) (debug_info_t* id, - struct debug_view* view, char* out_buf, - const char* in_buf); -typedef int (debug_prolog_proc_t) (debug_info_t* id, - struct debug_view* view, - char* out_buf); -typedef int (debug_input_proc_t) (debug_info_t* id, - struct debug_view* view, - struct file* file, const char* user_buf, - size_t in_buf_size, loff_t* offset); - - -The "private_data" member can be used as pointer to view specific data. -It is not used by the debug feature itself. - -The output when reading a debugfs file is structured like this: - -"prolog_proc output" - -"header_proc output 1" "format_proc output 1" -"header_proc output 2" "format_proc output 2" -"header_proc output 3" "format_proc output 3" -... - -When a view is read from the debugfs, the Debug Feature calls the -'prolog_proc' once for writing the prolog. -Then 'header_proc' and 'format_proc' are called for each -existing debug entry. - -The input_proc can be used to implement functionality when it is written to -the view (e.g. like with 'echo "0" > /sys/kernel/debug/s390dbf/dasd/level). - -For header_proc there can be used the default function -debug_dflt_header_fn() which is defined in debug.h. -and which produces the same header output as the predefined views. -E.g: -00 00964419409:440761 2 - 00 88023ec - -In order to see how to use the callback functions check the implementation -of the default views! - -Example - -#include <asm/debug.h> - -#define UNKNOWNSTR "data: %08x" - -const char* messages[] = -{"This error...........\n", - "That error...........\n", - "Problem..............\n", - "Something went wrong.\n", - "Everything ok........\n", - NULL -}; - -static int debug_test_format_fn( - debug_info_t * id, struct debug_view *view, - char *out_buf, const char *in_buf -) -{ - int i, rc = 0; - - if(id->buf_size >= 4) { - int msg_nr = *((int*)in_buf); - if(msg_nr < sizeof(messages)/sizeof(char*) - 1) - rc += sprintf(out_buf, "%s", messages[msg_nr]); - else - rc += sprintf(out_buf, UNKNOWNSTR, msg_nr); - } - out: - return rc; -} - -struct debug_view debug_test_view = { - "myview", /* name of view */ - NULL, /* no prolog */ - &debug_dflt_header_fn, /* default header for each entry */ - &debug_test_format_fn, /* our own format function */ - NULL, /* no input function */ - NULL /* no private data */ -}; - -===== -test: -===== -debug_info_t *debug_info; -... -debug_info = debug_register ("test", 0, 4, 4 )); -debug_register_view(debug_info, &debug_test_view); -for(i = 0; i < 10; i ++) debug_int_event(debug_info, 1, i); - -> cat /sys/kernel/debug/s390dbf/test/myview -00 00964419734:611402 1 - 00 88042ca This error........... -00 00964419734:611405 1 - 00 88042ca That error........... -00 00964419734:611408 1 - 00 88042ca Problem.............. -00 00964419734:611411 1 - 00 88042ca Something went wrong. -00 00964419734:611414 1 - 00 88042ca Everything ok........ -00 00964419734:611417 1 - 00 88042ca data: 00000005 -00 00964419734:611419 1 - 00 88042ca data: 00000006 -00 00964419734:611422 1 - 00 88042ca data: 00000007 -00 00964419734:611425 1 - 00 88042ca data: 00000008 -00 00964419734:611428 1 - 00 88042ca data: 00000009 diff --git a/Documentation/s390/text_files.rst b/Documentation/s390/text_files.rst new file mode 100644 index 000000000000..c94d05d4fa17 --- /dev/null +++ b/Documentation/s390/text_files.rst @@ -0,0 +1,11 @@ +ibm 3270 changelog +------------------ + +.. include:: 3270.ChangeLog + :literal: + +ibm 3270 config3270.sh +---------------------- + +.. literalinclude:: config3270.sh + :language: shell diff --git a/Documentation/s390/vfio-ap.txt b/Documentation/s390/vfio-ap.rst index 65167cfe4485..b5c51f7c748d 100644 --- a/Documentation/s390/vfio-ap.txt +++ b/Documentation/s390/vfio-ap.rst @@ -1,4 +1,9 @@ -Introduction: +=============================== +Adjunct Processor (AP) facility +=============================== + + +Introduction ============ The Adjunct Processor (AP) facility is an IBM Z cryptographic facility comprised of three AP instructions and from 1 up to 256 PCIe cryptographic adapter cards. @@ -11,7 +16,7 @@ framework. This implementation relies considerably on the s390 virtualization facilities which do most of the hard work of providing direct access to AP devices. -AP Architectural Overview: +AP Architectural Overview ========================= To facilitate the comprehension of the design, let's start with some definitions: @@ -31,13 +36,13 @@ definitions: in the LPAR, the AP bus detects the AP adapter cards assigned to the LPAR and creates a sysfs device for each assigned adapter. For example, if AP adapters 4 and 10 (0x0a) are assigned to the LPAR, the AP bus will create the following - sysfs device entries: + sysfs device entries:: /sys/devices/ap/card04 /sys/devices/ap/card0a Symbolic links to these devices will also be created in the AP bus devices - sub-directory: + sub-directory:: /sys/bus/ap/devices/[card04] /sys/bus/ap/devices/[card04] @@ -84,7 +89,7 @@ definitions: the cross product of the AP adapter and usage domain numbers detected when the AP bus module is loaded. For example, if adapters 4 and 10 (0x0a) and usage domains 6 and 71 (0x47) are assigned to the LPAR, the AP bus will create the - following sysfs entries: + following sysfs entries:: /sys/devices/ap/card04/04.0006 /sys/devices/ap/card04/04.0047 @@ -92,7 +97,7 @@ definitions: /sys/devices/ap/card0a/0a.0047 The following symbolic links to these devices will be created in the AP bus - devices subdirectory: + devices subdirectory:: /sys/bus/ap/devices/[04.0006] /sys/bus/ap/devices/[04.0047] @@ -112,7 +117,7 @@ definitions: domain that is not one of the usage domains, but the modified domain must be one of the control domains. -AP and SIE: +AP and SIE ========== Let's now take a look at how AP instructions executed on a guest are interpreted by the hardware. @@ -153,7 +158,7 @@ and 2 and usage domains 5 and 6 are assigned to a guest, the APQNs (1,5), (1,6), The APQNs can provide secure key functionality - i.e., a private key is stored on the adapter card for each of its domains - so each APQN must be assigned to -at most one guest or to the linux host. +at most one guest or to the linux host:: Example 1: Valid configuration: ------------------------------ @@ -181,8 +186,8 @@ at most one guest or to the linux host. This is an invalid configuration because both guests have access to APQN (1,6). -The Design: -=========== +The Design +========== The design introduces three new objects: 1. AP matrix device @@ -205,43 +210,43 @@ The VFIO AP (vfio_ap) device driver serves the following purposes: Reserve APQNs for exclusive use of KVM guests --------------------------------------------- The following block diagram illustrates the mechanism by which APQNs are -reserved: - - +------------------+ - 7 remove | | - +--------------------> cex4queue driver | - | | | - | +------------------+ - | - | - | +------------------+ +-----------------+ - | 5 register driver | | 3 create | | - | +----------------> Device core +----------> matrix device | - | | | | | | - | | +--------^---------+ +-----------------+ - | | | - | | +-------------------+ - | | +-----------------------------------+ | - | | | 4 register AP driver | | 2 register device - | | | | | -+--------+---+-v---+ +--------+-------+-+ -| | | | -| ap_bus +--------------------- > vfio_ap driver | -| | 8 probe | | -+--------^---------+ +--^--^------------+ -6 edit | | | - apmask | +-----------------------------+ | 9 mdev create - aqmask | | 1 modprobe | -+--------+-----+---+ +----------------+-+ +------------------+ -| | | |8 create | mediated | -| admin | | VFIO device core |---------> matrix | -| + | | | device | -+------+-+---------+ +--------^---------+ +--------^---------+ - | | | | - | | 9 create vfio_ap-passthrough | | - | +------------------------------+ | - +-------------------------------------------------------------+ - 10 assign adapter/domain/control domain +reserved:: + + +------------------+ + 7 remove | | + +--------------------> cex4queue driver | + | | | + | +------------------+ + | + | + | +------------------+ +----------------+ + | 5 register driver | | 3 create | | + | +----------------> Device core +----------> matrix device | + | | | | | | + | | +--------^---------+ +----------------+ + | | | + | | +-------------------+ + | | +-----------------------------------+ | + | | | 4 register AP driver | | 2 register device + | | | | | + +--------+---+-v---+ +--------+-------+-+ + | | | | + | ap_bus +--------------------- > vfio_ap driver | + | | 8 probe | | + +--------^---------+ +--^--^------------+ + 6 edit | | | + apmask | +-----------------------------+ | 9 mdev create + aqmask | | 1 modprobe | + +--------+-----+---+ +----------------+-+ +----------------+ + | | | |8 create | mediated | + | admin | | VFIO device core |---------> matrix | + | + | | | device | + +------+-+---------+ +--------^---------+ +--------^-------+ + | | | | + | | 9 create vfio_ap-passthrough | | + | +------------------------------+ | + +-------------------------------------------------------------+ + 10 assign adapter/domain/control domain The process for reserving an AP queue for use by a KVM guest is: @@ -250,7 +255,7 @@ The process for reserving an AP queue for use by a KVM guest is: device with the device core. This will serve as the parent device for all mediated matrix devices used to configure an AP matrix for a guest. 3. The /sys/devices/vfio_ap/matrix device is created by the device core -4 The vfio_ap device driver will register with the AP bus for AP queue devices +4. The vfio_ap device driver will register with the AP bus for AP queue devices of type 10 and higher (CEX4 and newer). The driver will provide the vfio_ap driver's probe and remove callback interfaces. Devices older than CEX4 queues are not supported to simplify the implementation by not needlessly @@ -266,13 +271,14 @@ The process for reserving an AP queue for use by a KVM guest is: it. 9. The administrator creates a passthrough type mediated matrix device to be used by a guest -10 The administrator assigns the adapters, usage domains and control domains - to be exclusively used by a guest. +10. The administrator assigns the adapters, usage domains and control domains + to be exclusively used by a guest. Set up the VFIO mediated device interfaces ------------------------------------------ The VFIO AP device driver utilizes the common interface of the VFIO mediated device core driver to: + * Register an AP mediated bus driver to add a mediated matrix device to and remove it from a VFIO group. * Create and destroy a mediated matrix device @@ -280,25 +286,25 @@ device core driver to: * Add a mediated matrix device to and remove it from an IOMMU group The following high-level block diagram shows the main components and interfaces -of the VFIO AP mediated matrix device driver: - - +-------------+ - | | - | +---------+ | mdev_register_driver() +--------------+ - | | Mdev | +<-----------------------+ | - | | bus | | | vfio_mdev.ko | - | | driver | +----------------------->+ |<-> VFIO user - | +---------+ | probe()/remove() +--------------+ APIs - | | - | MDEV CORE | - | MODULE | - | mdev.ko | - | +---------+ | mdev_register_device() +--------------+ - | |Physical | +<-----------------------+ | - | | device | | | vfio_ap.ko |<-> matrix - | |interface| +----------------------->+ | device - | +---------+ | callback +--------------+ - +-------------+ +of the VFIO AP mediated matrix device driver:: + + +-------------+ + | | + | +---------+ | mdev_register_driver() +--------------+ + | | Mdev | +<-----------------------+ | + | | bus | | | vfio_mdev.ko | + | | driver | +----------------------->+ |<-> VFIO user + | +---------+ | probe()/remove() +--------------+ APIs + | | + | MDEV CORE | + | MODULE | + | mdev.ko | + | +---------+ | mdev_register_device() +--------------+ + | |Physical | +<-----------------------+ | + | | device | | | vfio_ap.ko |<-> matrix + | |interface| +----------------------->+ | device + | +---------+ | callback +--------------+ + +-------------+ During initialization of the vfio_ap module, the matrix device is registered with an 'mdev_parent_ops' structure that provides the sysfs attribute @@ -306,7 +312,8 @@ structures, mdev functions and callback interfaces for managing the mediated matrix device. * sysfs attribute structures: - * supported_type_groups + + supported_type_groups The VFIO mediated device framework supports creation of user-defined mediated device types. These mediated device types are specified via the 'supported_type_groups' structure when a device is registered @@ -318,61 +325,72 @@ matrix device. The VFIO AP device driver will register one mediated device type for passthrough devices: + /sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough + Only the read-only attributes required by the VFIO mdev framework will - be provided: - ... name - ... device_api - ... available_instances - ... device_api - Where: - * name: specifies the name of the mediated device type - * device_api: the mediated device type's API - * available_instances: the number of mediated matrix passthrough devices - that can be created - * device_api: specifies the VFIO API - * mdev_attr_groups + be provided:: + + ... name + ... device_api + ... available_instances + ... device_api + + Where: + + * name: + specifies the name of the mediated device type + * device_api: + the mediated device type's API + * available_instances: + the number of mediated matrix passthrough devices + that can be created + * device_api: + specifies the VFIO API + mdev_attr_groups This attribute group identifies the user-defined sysfs attributes of the mediated device. When a device is registered with the VFIO mediated device framework, the sysfs attribute files identified in the 'mdev_attr_groups' structure will be created in the mediated matrix device's directory. The sysfs attributes for a mediated matrix device are: - * assign_adapter: - * unassign_adapter: + + assign_adapter / unassign_adapter: Write-only attributes for assigning/unassigning an AP adapter to/from the mediated matrix device. To assign/unassign an adapter, the APID of the adapter is echoed to the respective attribute file. - * assign_domain: - * unassign_domain: + assign_domain / unassign_domain: Write-only attributes for assigning/unassigning an AP usage domain to/from the mediated matrix device. To assign/unassign a domain, the domain number of the the usage domain is echoed to the respective attribute file. - * matrix: + matrix: A read-only file for displaying the APQNs derived from the cross product of the adapter and domain numbers assigned to the mediated matrix device. - * assign_control_domain: - * unassign_control_domain: + assign_control_domain / unassign_control_domain: Write-only attributes for assigning/unassigning an AP control domain to/from the mediated matrix device. To assign/unassign a control domain, the ID of the domain to be assigned/unassigned is echoed to the respective attribute file. - * control_domains: + control_domains: A read-only file for displaying the control domain numbers assigned to the mediated matrix device. * functions: - * create: + + create: allocates the ap_matrix_mdev structure used by the vfio_ap driver to: + * Store the reference to the KVM structure for the guest using the mdev * Store the AP matrix configuration for the adapters, domains, and control domains assigned via the corresponding sysfs attributes files - * remove: + + remove: deallocates the mediated matrix device's ap_matrix_mdev structure. This will be allowed only if a running guest is not using the mdev. * callback interfaces - * open: + + open: The vfio_ap driver uses this callback to register a VFIO_GROUP_NOTIFY_SET_KVM notifier callback function for the mdev matrix device. The open is invoked when QEMU connects the VFIO iommu group @@ -380,16 +398,17 @@ matrix device. to configure the KVM guest is provided via this callback. The KVM structure, is used to configure the guest's access to the AP matrix defined via the mediated matrix device's sysfs attribute files. - * release: + release: unregisters the VFIO_GROUP_NOTIFY_SET_KVM notifier callback function for the mdev matrix device and deconfigures the guest's AP matrix. -Configure the APM, AQM and ADM in the CRYCB: +Configure the APM, AQM and ADM in the CRYCB ------------------------------------------- Configuring the AP matrix for a KVM guest will be performed when the VFIO_GROUP_NOTIFY_SET_KVM notifier callback is invoked. The notifier function is called when QEMU connects to KVM. The guest's AP matrix is configured via it's CRYCB by: + * Setting the bits in the APM corresponding to the APIDs assigned to the mediated matrix device via its 'assign_adapter' interface. * Setting the bits in the AQM corresponding to the domains assigned to the @@ -418,12 +437,12 @@ available to a KVM guest via the following CPU model features: Note: If the user chooses to specify a CPU model different than the 'host' model to QEMU, the CPU model features and facilities need to be turned on -explicitly; for example: +explicitly; for example:: /usr/bin/qemu-system-s390x ... -cpu z13,ap=on,apqci=on,apft=on A guest can be precluded from using AP features/facilities by turning them off -explicitly; for example: +explicitly; for example:: /usr/bin/qemu-system-s390x ... -cpu host,ap=off,apqci=off,apft=off @@ -435,7 +454,7 @@ the APFT facility is not installed on the guest, then the probe of device drivers will fail since only type 10 and newer devices can be configured for guest use. -Example: +Example ======= Let's now provide an example to illustrate how KVM guests may be given access to AP facilities. For this example, we will show how to configure @@ -444,30 +463,36 @@ look like this: Guest1 ------ +=========== ===== ============ CARD.DOMAIN TYPE MODE ------------------------------- +=========== ===== ============ 05 CEX5C CCA-Coproc 05.0004 CEX5C CCA-Coproc 05.00ab CEX5C CCA-Coproc 06 CEX5A Accelerator 06.0004 CEX5A Accelerator 06.00ab CEX5C CCA-Coproc +=========== ===== ============ Guest2 ------ +=========== ===== ============ CARD.DOMAIN TYPE MODE ------------------------------- +=========== ===== ============ 05 CEX5A Accelerator 05.0047 CEX5A Accelerator 05.00ff CEX5A Accelerator +=========== ===== ============ Guest2 ------ +=========== ===== ============ CARD.DOMAIN TYPE MODE ------------------------------- +=========== ===== ============ 06 CEX5A Accelerator 06.0047 CEX5A Accelerator 06.00ff CEX5A Accelerator +=========== ===== ============ These are the steps: @@ -492,25 +517,26 @@ These are the steps: * VFIO_MDEV_DEVICE * KVM - If using make menuconfig select the following to build the vfio_ap module: - -> Device Drivers - -> IOMMU Hardware Support - select S390 AP IOMMU Support - -> VFIO Non-Privileged userspace driver framework - -> Mediated device driver frramework - -> VFIO driver for Mediated devices - -> I/O subsystem - -> VFIO support for AP devices + If using make menuconfig select the following to build the vfio_ap module:: + + -> Device Drivers + -> IOMMU Hardware Support + select S390 AP IOMMU Support + -> VFIO Non-Privileged userspace driver framework + -> Mediated device driver frramework + -> VFIO driver for Mediated devices + -> I/O subsystem + -> VFIO support for AP devices 2. Secure the AP queues to be used by the three guests so that the host can not access them. To secure them, there are two sysfs files that specify bitmasks marking a subset of the APQN range as 'usable by the default AP queue device drivers' or 'not usable by the default device drivers' and thus available for use by the vfio_ap device driver'. The location of the sysfs - files containing the masks are: + files containing the masks are:: - /sys/bus/ap/apmask - /sys/bus/ap/aqmask + /sys/bus/ap/apmask + /sys/bus/ap/aqmask The 'apmask' is a 256-bit mask that identifies a set of AP adapter IDs (APID). Each bit in the mask, from left to right (i.e., from most significant @@ -526,7 +552,7 @@ These are the steps: queue device drivers; otherwise, the APQI is usable by the vfio_ap device driver. - Take, for example, the following mask: + Take, for example, the following mask:: 0x7dffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff @@ -548,68 +574,70 @@ These are the steps: respective sysfs mask file in one of two formats: * An absolute hex string starting with 0x - like "0x12345678" - sets - the mask. If the given string is shorter than the mask, it is padded - with 0s on the right; for example, specifying a mask value of 0x41 is - the same as specifying: + the mask. If the given string is shorter than the mask, it is padded + with 0s on the right; for example, specifying a mask value of 0x41 is + the same as specifying:: - 0x4100000000000000000000000000000000000000000000000000000000000000 + 0x4100000000000000000000000000000000000000000000000000000000000000 - Keep in mind that the mask reads from left to right (i.e., most - significant to least significant bit in big endian order), so the mask - above identifies device numbers 1 and 7 (01000001). + Keep in mind that the mask reads from left to right (i.e., most + significant to least significant bit in big endian order), so the mask + above identifies device numbers 1 and 7 (01000001). - If the string is longer than the mask, the operation is terminated with - an error (EINVAL). + If the string is longer than the mask, the operation is terminated with + an error (EINVAL). * Individual bits in the mask can be switched on and off by specifying - each bit number to be switched in a comma separated list. Each bit - number string must be prepended with a ('+') or minus ('-') to indicate - the corresponding bit is to be switched on ('+') or off ('-'). Some - valid values are: + each bit number to be switched in a comma separated list. Each bit + number string must be prepended with a ('+') or minus ('-') to indicate + the corresponding bit is to be switched on ('+') or off ('-'). Some + valid values are: - "+0" switches bit 0 on - "-13" switches bit 13 off - "+0x41" switches bit 65 on - "-0xff" switches bit 255 off + - "+0" switches bit 0 on + - "-13" switches bit 13 off + - "+0x41" switches bit 65 on + - "-0xff" switches bit 255 off - The following example: - +0,-6,+0x47,-0xf0 + The following example: - Switches bits 0 and 71 (0x47) on - Switches bits 6 and 240 (0xf0) off + +0,-6,+0x47,-0xf0 - Note that the bits not specified in the list remain as they were before - the operation. + Switches bits 0 and 71 (0x47) on + + Switches bits 6 and 240 (0xf0) off + + Note that the bits not specified in the list remain as they were before + the operation. 2. The masks can also be changed at boot time via parameters on the kernel command line like this: - ap.apmask=0xffff ap.aqmask=0x40 + ap.apmask=0xffff ap.aqmask=0x40 - This would create the following masks: + This would create the following masks:: - apmask: - 0xffff000000000000000000000000000000000000000000000000000000000000 + apmask: + 0xffff000000000000000000000000000000000000000000000000000000000000 - aqmask: - 0x4000000000000000000000000000000000000000000000000000000000000000 + aqmask: + 0x4000000000000000000000000000000000000000000000000000000000000000 - Resulting in these two pools: + Resulting in these two pools:: - default drivers pool: adapter 0-15, domain 1 - alternate drivers pool: adapter 16-255, domains 0, 2-255 + default drivers pool: adapter 0-15, domain 1 + alternate drivers pool: adapter 16-255, domains 0, 2-255 - Securing the APQNs for our example: - ---------------------------------- +Securing the APQNs for our example +---------------------------------- To secure the AP queues 05.0004, 05.0047, 05.00ab, 05.00ff, 06.0004, 06.0047, 06.00ab, and 06.00ff for use by the vfio_ap device driver, the corresponding - APQNs can either be removed from the default masks: + APQNs can either be removed from the default masks:: echo -5,-6 > /sys/bus/ap/apmask echo -4,-0x47,-0xab,-0xff > /sys/bus/ap/aqmask - Or the masks can be set as follows: + Or the masks can be set as follows:: echo 0xf9ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff \ > apmask @@ -620,19 +648,19 @@ These are the steps: This will result in AP queues 05.0004, 05.0047, 05.00ab, 05.00ff, 06.0004, 06.0047, 06.00ab, and 06.00ff getting bound to the vfio_ap device driver. The sysfs directory for the vfio_ap device driver will now contain symbolic links - to the AP queue devices bound to it: - - /sys/bus/ap - ... [drivers] - ...... [vfio_ap] - ......... [05.0004] - ......... [05.0047] - ......... [05.00ab] - ......... [05.00ff] - ......... [06.0004] - ......... [06.0047] - ......... [06.00ab] - ......... [06.00ff] + to the AP queue devices bound to it:: + + /sys/bus/ap + ... [drivers] + ...... [vfio_ap] + ......... [05.0004] + ......... [05.0047] + ......... [05.00ab] + ......... [05.00ff] + ......... [06.0004] + ......... [06.0047] + ......... [06.00ab] + ......... [06.00ff] Keep in mind that only type 10 and newer adapters (i.e., CEX4 and later) can be bound to the vfio_ap device driver. The reason for this is to @@ -645,96 +673,96 @@ These are the steps: queue device can be read from the parent card's sysfs directory. For example, to see the hardware type of the queue 05.0004: - cat /sys/bus/ap/devices/card05/hwtype + cat /sys/bus/ap/devices/card05/hwtype The hwtype must be 10 or higher (CEX4 or newer) in order to be bound to the vfio_ap device driver. 3. Create the mediated devices needed to configure the AP matrixes for the three guests and to provide an interface to the vfio_ap driver for - use by the guests: + use by the guests:: - /sys/devices/vfio_ap/matrix/ - --- [mdev_supported_types] - ------ [vfio_ap-passthrough] (passthrough mediated matrix device type) - --------- create - --------- [devices] + /sys/devices/vfio_ap/matrix/ + --- [mdev_supported_types] + ------ [vfio_ap-passthrough] (passthrough mediated matrix device type) + --------- create + --------- [devices] - To create the mediated devices for the three guests: + To create the mediated devices for the three guests:: uuidgen > create uuidgen > create uuidgen > create - or + or - echo $uuid1 > create - echo $uuid2 > create - echo $uuid3 > create + echo $uuid1 > create + echo $uuid2 > create + echo $uuid3 > create This will create three mediated devices in the [devices] subdirectory named after the UUID written to the create attribute file. We call them $uuid1, - $uuid2 and $uuid3 and this is the sysfs directory structure after creation: - - /sys/devices/vfio_ap/matrix/ - --- [mdev_supported_types] - ------ [vfio_ap-passthrough] - --------- [devices] - ------------ [$uuid1] - --------------- assign_adapter - --------------- assign_control_domain - --------------- assign_domain - --------------- matrix - --------------- unassign_adapter - --------------- unassign_control_domain - --------------- unassign_domain - - ------------ [$uuid2] - --------------- assign_adapter - --------------- assign_control_domain - --------------- assign_domain - --------------- matrix - --------------- unassign_adapter - ----------------unassign_control_domain - ----------------unassign_domain - - ------------ [$uuid3] - --------------- assign_adapter - --------------- assign_control_domain - --------------- assign_domain - --------------- matrix - --------------- unassign_adapter - ----------------unassign_control_domain - ----------------unassign_domain + $uuid2 and $uuid3 and this is the sysfs directory structure after creation:: + + /sys/devices/vfio_ap/matrix/ + --- [mdev_supported_types] + ------ [vfio_ap-passthrough] + --------- [devices] + ------------ [$uuid1] + --------------- assign_adapter + --------------- assign_control_domain + --------------- assign_domain + --------------- matrix + --------------- unassign_adapter + --------------- unassign_control_domain + --------------- unassign_domain + + ------------ [$uuid2] + --------------- assign_adapter + --------------- assign_control_domain + --------------- assign_domain + --------------- matrix + --------------- unassign_adapter + ----------------unassign_control_domain + ----------------unassign_domain + + ------------ [$uuid3] + --------------- assign_adapter + --------------- assign_control_domain + --------------- assign_domain + --------------- matrix + --------------- unassign_adapter + ----------------unassign_control_domain + ----------------unassign_domain 4. The administrator now needs to configure the matrixes for the mediated devices $uuid1 (for Guest1), $uuid2 (for Guest2) and $uuid3 (for Guest3). - This is how the matrix is configured for Guest1: + This is how the matrix is configured for Guest1:: echo 5 > assign_adapter echo 6 > assign_adapter echo 4 > assign_domain echo 0xab > assign_domain - Control domains can similarly be assigned using the assign_control_domain - sysfs file. + Control domains can similarly be assigned using the assign_control_domain + sysfs file. - If a mistake is made configuring an adapter, domain or control domain, - you can use the unassign_xxx files to unassign the adapter, domain or - control domain. + If a mistake is made configuring an adapter, domain or control domain, + you can use the unassign_xxx files to unassign the adapter, domain or + control domain. - To display the matrix configuration for Guest1: + To display the matrix configuration for Guest1:: - cat matrix + cat matrix - This is how the matrix is configured for Guest2: + This is how the matrix is configured for Guest2:: echo 5 > assign_adapter echo 0x47 > assign_domain echo 0xff > assign_domain - This is how the matrix is configured for Guest3: + This is how the matrix is configured for Guest3:: echo 6 > assign_adapter echo 0x47 > assign_domain @@ -783,24 +811,24 @@ These are the steps: configured for the system. If a control domain number higher than the maximum is specified, the operation will terminate with an error (ENODEV). -5. Start Guest1: +5. Start Guest1:: - /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ - -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid1 ... + /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ + -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid1 ... -7. Start Guest2: +7. Start Guest2:: - /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ - -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid2 ... + /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ + -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid2 ... -7. Start Guest3: +7. Start Guest3:: - /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ - -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid3 ... + /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ + -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid3 ... When the guest is shut down, the mediated matrix devices may be removed. -Using our example again, to remove the mediated matrix device $uuid1: +Using our example again, to remove the mediated matrix device $uuid1:: /sys/devices/vfio_ap/matrix/ --- [mdev_supported_types] @@ -809,18 +837,19 @@ Using our example again, to remove the mediated matrix device $uuid1: ------------ [$uuid1] --------------- remove +:: echo 1 > remove - This will remove all of the mdev matrix device's sysfs structures including - the mdev device itself. To recreate and reconfigure the mdev matrix device, - all of the steps starting with step 3 will have to be performed again. Note - that the remove will fail if a guest using the mdev is still running. +This will remove all of the mdev matrix device's sysfs structures including +the mdev device itself. To recreate and reconfigure the mdev matrix device, +all of the steps starting with step 3 will have to be performed again. Note +that the remove will fail if a guest using the mdev is still running. - It is not necessary to remove an mdev matrix device, but one may want to - remove it if no guest will use it during the remaining lifetime of the linux - host. If the mdev matrix device is removed, one may want to also reconfigure - the pool of adapters and queues reserved for use by the default drivers. +It is not necessary to remove an mdev matrix device, but one may want to +remove it if no guest will use it during the remaining lifetime of the linux +host. If the mdev matrix device is removed, one may want to also reconfigure +the pool of adapters and queues reserved for use by the default drivers. Limitations =========== diff --git a/Documentation/s390/vfio-ccw.txt b/Documentation/s390/vfio-ccw.rst index 2be11ad864ff..1f6d0b56d53e 100644 --- a/Documentation/s390/vfio-ccw.txt +++ b/Documentation/s390/vfio-ccw.rst @@ -1,3 +1,4 @@ +================================== vfio-ccw: the basic infrastructure ================================== @@ -11,9 +12,11 @@ virtual machine, while vfio is the means. Different than other hardware architectures, s390 has defined a unified I/O access method, which is so called Channel I/O. It has its own access patterns: + - Channel programs run asynchronously on a separate (co)processor. - The channel subsystem will access any memory designated by the caller in the channel program directly, i.e. there is no iommu involved. + Thus when we introduce vfio support for these devices, we realize it with a mediated device (mdev) implementation. The vfio mdev will be added to an iommu group, so as to make itself able to be managed by the @@ -24,6 +27,7 @@ to perform I/O instructions. This document does not intend to explain the s390 I/O architecture in every detail. More information/reference could be found here: + - A good start to know Channel I/O in general: https://en.wikipedia.org/wiki/Channel_I/O - s390 architecture: @@ -80,6 +84,7 @@ until interrupted. The I/O completion result is received by the interrupt handler in the form of interrupt response block (IRB). Back to vfio-ccw, in short: + - ORBs and channel programs are built in guest kernel (with guest physical addresses). - ORBs and channel programs are passed to the host kernel. @@ -106,6 +111,7 @@ it gets sent to hardware. Within this implementation, we have two drivers for two types of devices: + - The vfio_ccw driver for the physical subchannel device. This is an I/O subchannel driver for the real subchannel device. It realizes a group of callbacks and registers to the mdev framework as a @@ -137,7 +143,7 @@ devices: vfio_pin_pages and a vfio_unpin_pages interfaces from the vfio iommu backend for the physical devices to pin and unpin pages by demand. -Below is a high Level block diagram. +Below is a high Level block diagram:: +-------------+ | | @@ -158,6 +164,7 @@ Below is a high Level block diagram. +-------------+ The process of how these work together. + 1. vfio_ccw.ko drives the physical I/O subchannel, and registers the physical device (with callbacks) to mdev framework. When vfio_ccw probing the subchannel device, it registers device @@ -178,17 +185,17 @@ vfio-ccw I/O region An I/O region is used to accept channel program request from user space and store I/O interrupt result for user space to retrieve. The -definition of the region is: - -struct ccw_io_region { -#define ORB_AREA_SIZE 12 - __u8 orb_area[ORB_AREA_SIZE]; -#define SCSW_AREA_SIZE 12 - __u8 scsw_area[SCSW_AREA_SIZE]; -#define IRB_AREA_SIZE 96 - __u8 irb_area[IRB_AREA_SIZE]; - __u32 ret_code; -} __packed; +definition of the region is:: + + struct ccw_io_region { + #define ORB_AREA_SIZE 12 + __u8 orb_area[ORB_AREA_SIZE]; + #define SCSW_AREA_SIZE 12 + __u8 scsw_area[SCSW_AREA_SIZE]; + #define IRB_AREA_SIZE 96 + __u8 irb_area[IRB_AREA_SIZE]; + __u32 ret_code; + } __packed; While starting an I/O request, orb_area should be filled with the guest ORB, and scsw_area should be filled with the SCSW of the Virtual @@ -205,7 +212,7 @@ vfio-ccw follows what vfio-pci did on the s390 platform and uses vfio-iommu-type1 as the vfio iommu backend. * CCW translation APIs - A group of APIs (start with 'cp_') to do CCW translation. The CCWs + A group of APIs (start with `cp_`) to do CCW translation. The CCWs passed in by a user space program are organized with their guest physical memory addresses. These APIs will copy the CCWs into kernel space, and assemble a runnable kernel channel program by updating the @@ -217,12 +224,14 @@ vfio-iommu-type1 as the vfio iommu backend. This driver utilizes the CCW translation APIs and introduces vfio_ccw, which is the driver for the I/O subchannel devices you want to pass through. - vfio_ccw implements the following vfio ioctls: + vfio_ccw implements the following vfio ioctls:: + VFIO_DEVICE_GET_INFO VFIO_DEVICE_GET_IRQ_INFO VFIO_DEVICE_GET_REGION_INFO VFIO_DEVICE_RESET VFIO_DEVICE_SET_IRQS + This provides an I/O region, so that the user space program can pass a channel program to the kernel, to do further CCW translation before issuing them to a real device. @@ -236,32 +245,49 @@ bit more detail how an I/O request triggered by the QEMU guest will be handled (without error handling). Explanation: -Q1-Q7: QEMU side process. -K1-K5: Kernel side process. -Q1. Get I/O region info during initialization. -Q2. Setup event notifier and handler to handle I/O completion. +- Q1-Q7: QEMU side process. +- K1-K5: Kernel side process. + +Q1. + Get I/O region info during initialization. + +Q2. + Setup event notifier and handler to handle I/O completion. ... ... -Q3. Intercept a ssch instruction. -Q4. Write the guest channel program and ORB to the I/O region. - K1. Copy from guest to kernel. - K2. Translate the guest channel program to a host kernel space - channel program, which becomes runnable for a real device. - K3. With the necessary information contained in the orb passed in - by QEMU, issue the ccwchain to the device. - K4. Return the ssch CC code. -Q5. Return the CC code to the guest. +Q3. + Intercept a ssch instruction. +Q4. + Write the guest channel program and ORB to the I/O region. + + K1. + Copy from guest to kernel. + K2. + Translate the guest channel program to a host kernel space + channel program, which becomes runnable for a real device. + K3. + With the necessary information contained in the orb passed in + by QEMU, issue the ccwchain to the device. + K4. + Return the ssch CC code. +Q5. + Return the CC code to the guest. ... ... - K5. Interrupt handler gets the I/O result and write the result to - the I/O region. - K6. Signal QEMU to retrieve the result. -Q6. Get the signal and event handler reads out the result from the I/O + K5. + Interrupt handler gets the I/O result and write the result to + the I/O region. + K6. + Signal QEMU to retrieve the result. + +Q6. + Get the signal and event handler reads out the result from the I/O region. -Q7. Update the irb for the guest. +Q7. + Update the irb for the guest. Limitations ----------- @@ -295,6 +321,6 @@ Reference 1. ESA/s390 Principles of Operation manual (IBM Form. No. SA22-7832) 2. ESA/390 Common I/O Device Commands manual (IBM Form. No. SA22-7204) 3. https://en.wikipedia.org/wiki/Channel_I/O -4. Documentation/s390/cds.txt +4. Documentation/s390/cds.rst 5. Documentation/vfio.txt 6. Documentation/vfio-mediated-device.txt diff --git a/Documentation/s390/zfcpdump.txt b/Documentation/s390/zfcpdump.rst index b064aa59714d..54e8e7caf7e7 100644 --- a/Documentation/s390/zfcpdump.txt +++ b/Documentation/s390/zfcpdump.rst @@ -1,4 +1,6 @@ +================================== The s390 SCSI dump tool (zfcpdump) +================================== System z machines (z900 or higher) provide hardware support for creating system dumps on SCSI disks. The dump process is initiated by booting a dump tool, which diff --git a/Documentation/scheduler/sched-pelt.c b/Documentation/scheduler/sched-pelt.c index e4219139386a..7238b355919c 100644 --- a/Documentation/scheduler/sched-pelt.c +++ b/Documentation/scheduler/sched-pelt.c @@ -20,7 +20,8 @@ void calc_runnable_avg_yN_inv(void) int i; unsigned int x; - printf("static const u32 runnable_avg_yN_inv[] = {"); + /* To silence -Wunused-but-set-variable warnings. */ + printf("static const u32 runnable_avg_yN_inv[] __maybe_unused = {"); for (i = 0; i < HALFLIFE; i++) { x = ((1UL<<32)-1)*pow(y, i); diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index 9d0a7f08f93b..1159405cb920 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -37,7 +37,19 @@ import glob from docutils import nodes, statemachine from docutils.statemachine import ViewList from docutils.parsers.rst import directives, Directive -from sphinx.ext.autodoc import AutodocReporter + +# +# AutodocReporter is only good up to Sphinx 1.7 +# +import sphinx + +Use_SSI = sphinx.__version__[:3] >= '1.7' +if Use_SSI: + from sphinx.util.docutils import switch_source_input +else: + from sphinx.ext.autodoc import AutodocReporter + +import kernellog __version__ = '1.0' @@ -90,7 +102,8 @@ class KernelDocDirective(Directive): cmd += [filename] try: - env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd))) + kernellog.verbose(env.app, + 'calling kernel-doc \'%s\'' % (" ".join(cmd))) p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate() @@ -100,7 +113,8 @@ class KernelDocDirective(Directive): if p.returncode != 0: sys.stderr.write(err) - env.app.warn('kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode)) + kernellog.warn(env.app, + 'kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode)) return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] elif env.config.kerneldoc_verbosity > 0: sys.stderr.write(err) @@ -121,20 +135,28 @@ class KernelDocDirective(Directive): lineoffset += 1 node = nodes.section() - buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter + self.do_parse(result, node) + + return node.children + + except Exception as e: # pylint: disable=W0703 + kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with: %s' % + (" ".join(cmd), str(e))) + return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] + + def do_parse(self, result, node): + if Use_SSI: + with switch_source_input(self.state, result): + self.state.nested_parse(result, 0, node, match_titles=1) + else: + save = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter) self.state.memo.title_styles, self.state.memo.section_level = [], 0 try: self.state.nested_parse(result, 0, node, match_titles=1) finally: - self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf + self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = save - return node.children - - except Exception as e: # pylint: disable=W0703 - env.app.warn('kernel-doc \'%s\' processing failed with: %s' % - (" ".join(cmd), str(e))) - return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] def setup(app): app.add_config_value('kerneldoc_bin', None, 'env') diff --git a/Documentation/sphinx/kernellog.py b/Documentation/sphinx/kernellog.py new file mode 100644 index 000000000000..af924f51a7dc --- /dev/null +++ b/Documentation/sphinx/kernellog.py @@ -0,0 +1,28 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Sphinx has deprecated its older logging interface, but the replacement +# only goes back to 1.6. So here's a wrapper layer to keep around for +# as long as we support 1.4. +# +import sphinx + +if sphinx.__version__[:3] >= '1.6': + UseLogging = True + from sphinx.util import logging + logger = logging.getLogger('kerneldoc') +else: + UseLogging = False + +def warn(app, message): + if UseLogging: + logger.warning(message) + else: + app.warn(message) + +def verbose(app, message): + if UseLogging: + logger.verbose(message) + else: + app.verbose(message) + + diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py index b97228d2cc0e..fbfe6693bb60 100644 --- a/Documentation/sphinx/kfigure.py +++ b/Documentation/sphinx/kfigure.py @@ -60,6 +60,8 @@ import sphinx from sphinx.util.nodes import clean_astext from six import iteritems +import kernellog + PY3 = sys.version_info[0] == 3 if PY3: @@ -171,20 +173,20 @@ def setupTools(app): This function is called once, when the builder is initiated. """ global dot_cmd, convert_cmd # pylint: disable=W0603 - app.verbose("kfigure: check installed tools ...") + kernellog.verbose(app, "kfigure: check installed tools ...") dot_cmd = which('dot') convert_cmd = which('convert') if dot_cmd: - app.verbose("use dot(1) from: " + dot_cmd) + kernellog.verbose(app, "use dot(1) from: " + dot_cmd) else: - app.warn("dot(1) not found, for better output quality install " - "graphviz from http://www.graphviz.org") + kernellog.warn(app, "dot(1) not found, for better output quality install " + "graphviz from http://www.graphviz.org") if convert_cmd: - app.verbose("use convert(1) from: " + convert_cmd) + kernellog.verbose(app, "use convert(1) from: " + convert_cmd) else: - app.warn( + kernellog.warn(app, "convert(1) not found, for SVG to PDF conversion install " "ImageMagick (https://www.imagemagick.org)") @@ -220,12 +222,13 @@ def convert_image(img_node, translator, src_fname=None): # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages - app.verbose('assert best format for: ' + img_node['uri']) + kernellog.verbose(app, 'assert best format for: ' + img_node['uri']) if in_ext == '.dot': if not dot_cmd: - app.verbose("dot from graphviz not available / include DOT raw.") + kernellog.verbose(app, + "dot from graphviz not available / include DOT raw.") img_node.replace_self(file2literal(src_fname)) elif translator.builder.format == 'latex': @@ -252,7 +255,8 @@ def convert_image(img_node, translator, src_fname=None): if translator.builder.format == 'latex': if convert_cmd is None: - app.verbose("no SVG to PDF conversion available / include SVG raw.") + kernellog.verbose(app, + "no SVG to PDF conversion available / include SVG raw.") img_node.replace_self(file2literal(src_fname)) else: dst_fname = path.join(translator.builder.outdir, fname + '.pdf') @@ -265,18 +269,19 @@ def convert_image(img_node, translator, src_fname=None): _name = dst_fname[len(translator.builder.outdir) + 1:] if isNewer(dst_fname, src_fname): - app.verbose("convert: {out}/%s already exists and is newer" % _name) + kernellog.verbose(app, + "convert: {out}/%s already exists and is newer" % _name) else: ok = False mkdir(path.dirname(dst_fname)) if in_ext == '.dot': - app.verbose('convert DOT to: {out}/' + _name) + kernellog.verbose(app, 'convert DOT to: {out}/' + _name) ok = dot2format(app, src_fname, dst_fname) elif in_ext == '.svg': - app.verbose('convert SVG to: {out}/' + _name) + kernellog.verbose(app, 'convert SVG to: {out}/' + _name) ok = svg2pdf(app, src_fname, dst_fname) if not ok: @@ -305,7 +310,8 @@ def dot2format(app, dot_fname, out_fname): with open(out_fname, "w") as out: exit_code = subprocess.call(cmd, stdout = out) if exit_code != 0: - app.warn("Error #%d when calling: %s" % (exit_code, " ".join(cmd))) + kernellog.warn(app, + "Error #%d when calling: %s" % (exit_code, " ".join(cmd))) return bool(exit_code == 0) def svg2pdf(app, svg_fname, pdf_fname): @@ -322,7 +328,7 @@ def svg2pdf(app, svg_fname, pdf_fname): # use stdout and stderr from parent exit_code = subprocess.call(cmd) if exit_code != 0: - app.warn("Error #%d when calling: %s" % (exit_code, " ".join(cmd))) + kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd))) return bool(exit_code == 0) @@ -415,15 +421,15 @@ def visit_kernel_render(self, node): app = self.builder.app srclang = node.get('srclang') - app.verbose('visit kernel-render node lang: "%s"' % (srclang)) + kernellog.verbose(app, 'visit kernel-render node lang: "%s"' % (srclang)) tmp_ext = RENDER_MARKUP_EXT.get(srclang, None) if tmp_ext is None: - app.warn('kernel-render: "%s" unknown / include raw.' % (srclang)) + kernellog.warn(app, 'kernel-render: "%s" unknown / include raw.' % (srclang)) return if not dot_cmd and tmp_ext == '.dot': - app.verbose("dot from graphviz not available / include raw.") + kernellog.verbose(app, "dot from graphviz not available / include raw.") return literal_block = node[0] diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt index f0c86fbb3b48..5af8b131ccbc 100644 --- a/Documentation/sysctl/kernel.txt +++ b/Documentation/sysctl/kernel.txt @@ -23,7 +23,6 @@ show up in /proc/sys/kernel: - auto_msgmni - bootloader_type [ X86 only ] - bootloader_version [ X86 only ] -- callhome [ S390 only ] - cap_last_cap - core_pattern - core_pipe_limit @@ -171,21 +170,6 @@ Documentation/x86/boot.txt for additional information. ============================================================== -callhome: - -Controls the kernel's callhome behavior in case of a kernel panic. - -The s390 hardware allows an operating system to send a notification -to a service organization (callhome) in case of an operating system panic. - -When the value in this file is 0 (which is the default behavior) -nothing happens in case of a kernel panic. If this value is set to "1" -the complete kernel oops message is send to the IBM customer service -organization in case the mainframe the Linux operating system is running -on has a service contract with IBM. - -============================================================== - cap_last_cap Highest valid capability of the running kernel. Exports diff --git a/Documentation/translations/ko_KR/memory-barriers.txt b/Documentation/translations/ko_KR/memory-barriers.txt index db0b9d8619f1..5f3c74dcad43 100644 --- a/Documentation/translations/ko_KR/memory-barriers.txt +++ b/Documentation/translations/ko_KR/memory-barriers.txt @@ -24,7 +24,7 @@ Documentation/memory-barriers.txt ========================= 저자: David Howells <dhowells@redhat.com> - Paul E. McKenney <paulmck@linux.vnet.ibm.com> + Paul E. McKenney <paulmck@linux.ibm.com> Will Deacon <will.deacon@arm.com> Peter Zijlstra <peterz@infradead.org> diff --git a/Documentation/usb/rio.txt b/Documentation/usb/rio.txt index ca9adcf56355..ea73475471db 100644 --- a/Documentation/usb/rio.txt +++ b/Documentation/usb/rio.txt @@ -76,70 +76,30 @@ Additional Information and userspace tools Requirements ============ -A host with a USB port. Ideally, either a UHCI (Intel) or OHCI -(Compaq and others) hardware port should work. +A host with a USB port running a Linux kernel with RIO 500 support enabled. -A Linux development kernel (2.3.x) with USB support enabled or a -backported version to linux-2.2.x. See http://www.linux-usb.org for -more information on accomplishing this. +The driver is a module called rio500, which should be automatically loaded +as you plug in your device. If that fails you can manually load it with -A Linux kernel with RIO 500 support enabled. + modprobe rio500 -'lspci' which is only needed to determine the type of USB hardware -available in your machine. - -Configuration - -Using `lspci -v`, determine the type of USB hardware available. - - If you see something like:: - - USB Controller: ...... - Flags: ..... - I/O ports at .... - - Then you have a UHCI based controller. - - If you see something like:: - - USB Controller: ..... - Flags: .... - Memory at ..... - - Then you have a OHCI based controller. - -Using `make menuconfig` or your preferred method for configuring the -kernel, select 'Support for USB', 'OHCI/UHCI' depending on your -hardware (determined from the steps above), 'USB Diamond Rio500 support', and -'Preliminary USB device filesystem'. Compile and install the modules -(you may need to execute `depmod -a` to update the module -dependencies). - -Add a device for the USB rio500:: +Udev should automatically create a device node as soon as plug in your device. +If that fails, you can manually add a device for the USB rio500:: mknod /dev/usb/rio500 c 180 64 -Set appropriate permissions for /dev/usb/rio500 (don't forget about -group and world permissions). Both read and write permissions are +In that case, set appropriate permissions for /dev/usb/rio500 (don't forget +about group and world permissions). Both read and write permissions are required for proper operation. -Load the appropriate modules (if compiled as modules): - - OHCI:: - - modprobe usbcore - modprobe usb-ohci - modprobe rio500 - - UHCI:: - - modprobe usbcore - modprobe usb-uhci (or uhci) - modprobe rio500 - That's it. The Rio500 Utils at: http://rio500.sourceforge.net should be able to access the rio500. +Limits +====== + +You can use only a single rio500 device at a time with your computer. + Bugs ==== diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index ba6c42c576dd..2a4531bb06bd 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -1079,7 +1079,7 @@ yet and must be cleared on entry. 4.35 KVM_SET_USER_MEMORY_REGION -Capability: KVM_CAP_USER_MEM +Capability: KVM_CAP_USER_MEMORY Architectures: all Type: vm ioctl Parameters: struct kvm_userspace_memory_region (in) @@ -3857,43 +3857,59 @@ Type: vcpu ioctl Parameters: struct kvm_nested_state (in/out) Returns: 0 on success, -1 on error Errors: - E2BIG: the total state size (including the fixed-size part of struct - kvm_nested_state) exceeds the value of 'size' specified by + E2BIG: the total state size exceeds the value of 'size' specified by the user; the size required will be written into size. struct kvm_nested_state { __u16 flags; __u16 format; __u32 size; + union { - struct kvm_vmx_nested_state vmx; - struct kvm_svm_nested_state svm; + struct kvm_vmx_nested_state_hdr vmx; + struct kvm_svm_nested_state_hdr svm; + + /* Pad the header to 128 bytes. */ __u8 pad[120]; - }; - __u8 data[0]; + } hdr; + + union { + struct kvm_vmx_nested_state_data vmx[0]; + struct kvm_svm_nested_state_data svm[0]; + } data; }; #define KVM_STATE_NESTED_GUEST_MODE 0x00000001 #define KVM_STATE_NESTED_RUN_PENDING 0x00000002 +#define KVM_STATE_NESTED_EVMCS 0x00000004 -#define KVM_STATE_NESTED_SMM_GUEST_MODE 0x00000001 -#define KVM_STATE_NESTED_SMM_VMXON 0x00000002 +#define KVM_STATE_NESTED_FORMAT_VMX 0 +#define KVM_STATE_NESTED_FORMAT_SVM 1 -struct kvm_vmx_nested_state { +#define KVM_STATE_NESTED_VMX_VMCS_SIZE 0x1000 + +#define KVM_STATE_NESTED_VMX_SMM_GUEST_MODE 0x00000001 +#define KVM_STATE_NESTED_VMX_SMM_VMXON 0x00000002 + +struct kvm_vmx_nested_state_hdr { __u64 vmxon_pa; - __u64 vmcs_pa; + __u64 vmcs12_pa; struct { __u16 flags; } smm; }; +struct kvm_vmx_nested_state_data { + __u8 vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE]; + __u8 shadow_vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE]; +}; + This ioctl copies the vcpu's nested virtualization state from the kernel to userspace. -The maximum size of the state, including the fixed-size part of struct -kvm_nested_state, can be retrieved by passing KVM_CAP_NESTED_STATE to -the KVM_CHECK_EXTENSION ioctl(). +The maximum size of the state can be retrieved by passing KVM_CAP_NESTED_STATE +to the KVM_CHECK_EXTENSION ioctl(). 4.115 KVM_SET_NESTED_STATE @@ -3903,8 +3919,8 @@ Type: vcpu ioctl Parameters: struct kvm_nested_state (in) Returns: 0 on success, -1 on error -This copies the vcpu's kvm_nested_state struct from userspace to the kernel. For -the definition of struct kvm_nested_state, see KVM_GET_NESTED_STATE. +This copies the vcpu's kvm_nested_state struct from userspace to the kernel. +For the definition of struct kvm_nested_state, see KVM_GET_NESTED_STATE. 4.116 KVM_(UN)REGISTER_COALESCED_MMIO diff --git a/Documentation/vm/hmm.rst b/Documentation/vm/hmm.rst index ec1efa32af3c..7cdf7282e022 100644 --- a/Documentation/vm/hmm.rst +++ b/Documentation/vm/hmm.rst @@ -288,15 +288,17 @@ For instance if the device flags for device entries are: WRITE (1 << 62) Now let say that device driver wants to fault with at least read a range then -it does set: - range->default_flags = (1 << 63) +it does set:: + + range->default_flags = (1 << 63); range->pfn_flags_mask = 0; and calls hmm_range_fault() as described above. This will fill fault all page in the range with at least read permission. Now let say driver wants to do the same except for one page in the range for -which its want to have write. Now driver set: +which its want to have write. Now driver set:: + range->default_flags = (1 << 63); range->pfn_flags_mask = (1 << 62); range->pfns[index_of_write] = (1 << 62); diff --git a/Documentation/x86/exception-tables.rst b/Documentation/x86/exception-tables.rst index 24596c8210b5..ed6d4b0cf62c 100644 --- a/Documentation/x86/exception-tables.rst +++ b/Documentation/x86/exception-tables.rst @@ -35,7 +35,7 @@ page fault handler:: void do_page_fault(struct pt_regs *regs, unsigned long error_code) in arch/x86/mm/fault.c. The parameters on the stack are set up by -the low level assembly glue in arch/x86/kernel/entry_32.S. The parameter +the low level assembly glue in arch/x86/entry/entry_32.S. The parameter regs is a pointer to the saved registers on the stack, error_code contains a reason code for the exception. diff --git a/Documentation/x86/topology.rst b/Documentation/x86/topology.rst index 6e28dbe818ab..8e9704f61017 100644 --- a/Documentation/x86/topology.rst +++ b/Documentation/x86/topology.rst @@ -49,6 +49,10 @@ Package-related topology information in the kernel: The number of cores in a package. This information is retrieved via CPUID. + - cpuinfo_x86.x86_max_dies: + + The number of dies in a package. This information is retrieved via CPUID. + - cpuinfo_x86.phys_proc_id: The physical ID of the package. This information is retrieved via CPUID |
