aboutsummaryrefslogtreecommitdiffstats
path: root/tools/testing/selftests/rcutorture (follow)
AgeCommit message (Collapse)AuthorFilesLines
2018-06-25rcutorture: Remove obsolete TREE08-T.boot filePaul E. McKenney1-1/+0
The TREE08-T.boot file was used to provide alternative options for debugging, but things have changed, it has not kept up, and it has not been used or missed. This commit therefore removes it. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-06-25torture: Use a single build directory for torture scenariosPaul E. McKenney4-22/+22
The torture scripting currently builds each kernel from a set of parallel runs in its own build directory. This can waste quite a bit of space when running large numbers of concurrent scenarios, and pointlessly given that the builds are run sequentially (albeit with a largish -j number passed to "make"). This commit therefore places all build-command output in the results directory, then does all builds in a single "b1" build directory. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-05-15torture: Make kvm-find-errors.sh find build warningsPaul E. McKenney1-2/+3
Currently, kvm-find-errors.sh looks only for build errors ("error:"), so this commit makes it also locate build warnings ("warning:"). Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Tested-by: Nicholas Piggin <npiggin@gmail.com>
2018-05-15rcutorture: Abbreviate kvm.sh summary linesPaul E. McKenney1-2/+2
With the addition of the end-of-test state, it is not uncommon for the kvm.sh summary lines to overflow 80 characters. This commit therefore applies abbreviations in order to make the line fit into 80 characters with high probability. And yes, I did make heavy use of punched cards back in the day, so 80 columns it is for my xterms! ;-) Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Tested-by: Nicholas Piggin <npiggin@gmail.com>
2018-05-15rcutorture: Print end-of-test state in kvm.sh summaryPaul E. McKenney1-2/+6
This commit adds the end-of-test test, if present in the console output, to the kvm.sh test summary that is printed by kvm-recheck.sh. Note that this only applies to rcutorture console output. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Tested-by: Nicholas Piggin <npiggin@gmail.com>
2018-05-15torture: Fold parse-torture.sh into parse-console.shPaul E. McKenney4-123/+102
The rcutorture scripting scans the console output twice, once to look for various sorts of hangs and again to find warnings and panics. Unfortunately, only the output of the second scan gets written to the console.log.diags file, which can cause hangs to be overlooked. This commit therefore folds the parse-torture.sh script (which looks for hangs) into the parse-console.sh script (which looks for warnings and panics). This allows both types of failure information to be added to console.log.diags, while still reliably removing this file when it proves to be empty. This also fixes a long-standing bug where rcuperf log files would unconditionally complain about a hang. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Tested-by: Nicholas Piggin <npiggin@gmail.com>
2018-05-15torture: Add a script to edit output from failed runsPaul E. McKenney1-0/+55
This commit adds a script that allows viewing the build and/or console output from failed rcutorture, locktorture, or rcuperf runs. This replaces a time-honored but inefficient manual procedure that uses cut and paste. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Tested-by: Nicholas Piggin <npiggin@gmail.com>
2018-02-23Merge branches 'fixes.2018.02.23a', 'srcu.2018.02.20a' and 'torture.2018.02.20a' into HEADPaul E. McKenney5-37/+41
fixes.2018.02.23a: Miscellaneous fixes srcu.2018.02.20a: SRCU updates torture.2018.02.20a: Torture-test updates
2018-02-20torture: Provide more sensible nreader/nwriter defaults for rcuperfPaul E. McKenney1-23/+1
The default values for nreader and nwriter are apparently not all that user-friendly, resulting in people doing scalability tests that ran all runs at large scale. This commit therefore makes both the nreaders and nwriters module default to the number of CPUs, and adds a comment to rcuperf.c stating that the number of CPUs should be specified using the nr_cpus kernel boot parameter. This commit also eliminates the redundant rcuperf scripting specification of default values for these parameters. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20torture: Grace periods do not piggyback off of themselvesPaul E. McKenney1-1/+2
The rcuperf trace-event processing counted every "done" trace event as a piggyback, which is incorrect because the task that started the grace period didn't piggyback at all. This commit fixes this problem by recording the task that started a given grace period and ignoring that task's "done" record for that grace period. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20torture: Adjust rcuperf trace processing to allow for workqueuesPaul E. McKenney1-4/+4
The rcuperf event-trace processing assumes that expedited grace periods start and end on the same task, an assumption that was violated by moving expedited grace-period processing to workqueues. This commit removes this now-fallacious assumption from rcuperf's event-trace processing. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20torture: Default jitter off when running rcuperfPaul E. McKenney1-0/+6
The purpose of jitter is to expose concurrency bugs due to invalid assumptions about forward progress. There is usually little point in jitter when measuring performance. This commit therefore defaults jitter off when running rcuperf. You can override this by specifying the kvm.sh "--jitter" argument -after- the "--torture rcuperf" argument. No idea why you would want this, but if you do, that is how you do it. One example of a conccurrency bug that this jitter might expose is one in which the developer assumed that a given short region of code would be guaranteed to execute within some short time limit. Such assumptions are invalid in virtualized environments because the hupervisor can preempt the guest OS at any point, even when the guest OS thinks that it has disabled interrupts. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20torture: Specify qemu memory size with --memory argumentPaul E. McKenney2-2/+10
The 512 megabyte memory size has served quite well, but more memory is required when using large trace buffers on large systems. This commit therefore adds a --memory argument to the kvm.sh script, which allows the memory size to be specified on the command line, for example, "--memory 768", --memory 800M", or "--memory 2G". Reported-by: Sebastian Siewior <bigeasy@linutronix.de> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20rcutorture: Add basic ARM64 support to run scriptsLihao Liang1-2/+15
This commit adds support of the qemu command qemu-system-aarch64 to rcutorture. Signed-off-by: Lihao Liang <lianglihao@huawei.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20rcutorture: Update kvm.sh header commentPaul E. McKenney1-5/+3
The kvm.sh header comment is a bit of a relic, so this commit brings it up to date. Reported-by: Lihao Liang <lianglihao@huawei.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-20doc: Fix typo in rcutorture documentationLihao Liang1-1/+1
Signed-off-by: Lihao Liang <lianglihao@huawei.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2018-02-15sched/isolation: Eliminate NO_HZ_FULL_ALLPaul E. McKenney5-5/+2
Commit 6f1982fedd59 ("sched/isolation: Handle the nohz_full= parameter") broke CONFIG_NO_HZ_FULL_ALL=y kernels. This breakage is due to the code under CONFIG_NO_HZ_FULL_ALL failing to invoke the shiny new housekeeping functions. This means that rcutorture scenario TREE04 now emits RCU CPU stall warnings due to the RCU grace-period kthreads not being awakened at a time of their choosing, or perhaps even not at all: [ 27.731422] rcu_bh kthread starved for 21001 jiffies! g18446744073709551369 c18446744073709551368 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=3 [ 27.731423] rcu_bh I14936 9 2 0x80080000 [ 27.731435] Call Trace: [ 27.731440] __schedule+0x31a/0x6d0 [ 27.731442] schedule+0x31/0x80 [ 27.731446] schedule_timeout+0x15a/0x320 [ 27.731453] ? call_timer_fn+0x130/0x130 [ 27.731457] rcu_gp_kthread+0x66c/0xea0 [ 27.731458] ? rcu_gp_kthread+0x66c/0xea0 Because no one has complained about CONFIG_NO_HZ_FULL_ALL=y being broken, I hypothesize that no one is in fact using it, other than rcutorture. This commit therefore eliminates CONFIG_NO_HZ_FULL_ALL and updates rcutorture's config files to instead use the nohz_full= kernel parameter to put the desired CPUs into nohz_full mode. Fixes: 6f1982fedd59 ("sched/isolation: Handle the nohz_full= parameter") Reported-by: kernel test robot <xiaolong.ye@intel.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Frederic Weisbecker <frederic@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Chris Metcalf <cmetcalf@mellanox.com> Cc: Christoph Lameter <cl@linux.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luiz Capitulino <lcapitulino@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Wanpeng Li <kernellwp@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: John Stultz <john.stultz@linaro.org> Cc: Jonathan Corbet <corbet@lwn.net>
2017-12-11torture: Eliminate torture_runnable and perf_runnablePaul E. McKenney3-3/+0
The purpose of torture_runnable is to allow rcutorture and locktorture to be started and stopped via sysfs when they are built into the kernel (as in not compiled as loadable modules). However, the 0444 permissions for both instances of torture_runnable prevent this use case from ever being put into practice. Given that there have been no complaints about this deficiency, it is reasonable to conclude that no one actually makes use of this sysfs capability. The perf_runnable module parameter for rcuperf is in the same situation. This commit therefore removes both torture_runnable instances as well as perf_runnable. Reported-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture/kvm-build.sh: Skip build directory checkSeongJae Park1-5/+0
Check for build-directory existence and write permissions are provided in both 'kvm-test-1-run.sh' an 'kvm-build.sh'. Because the 'kvm-build.sh' is dependent on 'kvm-test-1-run.sh' ('kvm-build.sh' uses variables that defined from its caller.), these checks are unnecessarily duplicated. This commit therefore removes the check in from the 'kvm-build.sh' script. Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture: Simplify functions.sh include pathSeongJae Park5-5/+5
Inclusions of 'functions.sh' from 'kvm-test-1-run.sh' and 'kvm-recheck*.sh' use its absolute path. Because the directory containing 'functions.sh' is already in PATH, the full path is unnecessary. This commit therefore simplifies the inclusions to use the short relative path. Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture: Simplify loggingSeongJae Park2-24/+12
Both the 'kvm.sh' and 'kvm-test-1-run.sh' scripts log messages by printing the message to 'stdout' and then also printing it into the log file. Generation of the message thus occurs twice, once for 'stdout' and once for the log file. Moreover, many of the messages contain 'date' output, which results in date being invoked twice (once for stdout print, once for log file write). As a result, the date information in stdout and log file can differ, which could cause confusion. This commit therefore simplifies the logging procedure by using 'tee'. Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture/kvm-recheck-*: Improve result directory readability checkSeongJae Park3-3/+3
The kvm-recheck-(lock|rcu|rcuperf).sh scripts check whether the user-specified results directory exists. If not, it prints out error message that says the specified directory is unreadable. To make the message more precise, this commit adds a readability check. Fixes: 2193e1604eac ("rcutorture: Abstract kvm-recheck.sh") Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture/kvm.sh: Support execution from any directorySeongJae Park1-2/+3
The 'kvm.sh' rcutorture script requires that it be invoked from the top of Linux-kernel source tree. It is just a subtle restriction, but users using it for the first time could forget the restriction and be confused. Moreover, it makes commands a little longer, which can be frustrating. This commit therefore lets users invoke the script from any location. Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture/kvm.sh: Use consistent help text for --qemu-argsSeongJae Park1-2/+2
The '--qemu-args' option's help text is wrongly copied from '--qemu-cmd' option and its argument type description message format is inconsistent with other arguments. This commit fixes the usage and type messages to be consistent with others. Fixes: e9ce640001c6 ("rcutorture: Add --qemu-args argument to kvm.sh") Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture/kvm.sh: Remove unused variable, `alldone`SeongJae Park1-1/+0
The variable `alldone` is defined but not used within an awk script. This commit therefore removes it. Fixes:53954671033d ("rcutorture: Do better bin packing") Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture: Remove unused script, config2frag.shSeongJae Park1-25/+0
The 'config2frag.sh' script is not used, so this commit removes it. Fixes: c87b9c601ac8 ("rcutorture: Add KVM-based test framework") Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-11rcutorture/configinit: Fix build directory error messageSeongJae Park1-1/+1
The 'configinit.sh' script checks the format of optional argument for the build directory, printing an error message if the format is not valid. However, the error message uses the wrong variable, indicating an empty string even though the user entered a non-empty (but erroneous) string. This commit fixes the script to use the correct variable. Fixes: c87b9c601ac8 ("rcutorture: Add KVM-based test framework") Signed-off-by: SeongJae Park <sj38.park@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-11-28torture: Prepare scripting for shift from %p to %pKPaul E. McKenney1-1/+1
Because %p prints "(null)" and %pK prints "0000000000000000" or (on 32-bit systems) "00000000", this commit adjusts torture-test scripting accordingly. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-11-13Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds1-3/+2
Pull core locking updates from Ingo Molnar: "The main changes in this cycle are: - Another attempt at enabling cross-release lockdep dependency tracking (automatically part of CONFIG_PROVE_LOCKING=y), this time with better performance and fewer false positives. (Byungchul Park) - Introduce lockdep_assert_irqs_enabled()/disabled() and convert open-coded equivalents to lockdep variants. (Frederic Weisbecker) - Add down_read_killable() and use it in the VFS's iterate_dir() method. (Kirill Tkhai) - Convert remaining uses of ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE(). Most of the conversion was Coccinelle driven. (Mark Rutland, Paul E. McKenney) - Get rid of lockless_dereference(), by strengthening Alpha atomics, strengthening READ_ONCE() with smp_read_barrier_depends() and thus being able to convert users of lockless_dereference() to READ_ONCE(). (Will Deacon) - Various micro-optimizations: - better PV qspinlocks (Waiman Long), - better x86 barriers (Michael S. Tsirkin) - better x86 refcounts (Kees Cook) - ... plus other fixes and enhancements. (Borislav Petkov, Juergen Gross, Miguel Bernal Marin)" * 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (70 commits) locking/x86: Use LOCK ADD for smp_mb() instead of MFENCE rcu: Use lockdep to assert IRQs are disabled/enabled netpoll: Use lockdep to assert IRQs are disabled/enabled timers/posix-cpu-timers: Use lockdep to assert IRQs are disabled/enabled sched/clock, sched/cputime: Use lockdep to assert IRQs are disabled/enabled irq_work: Use lockdep to assert IRQs are disabled/enabled irq/timings: Use lockdep to assert IRQs are disabled/enabled perf/core: Use lockdep to assert IRQs are disabled/enabled x86: Use lockdep to assert IRQs are disabled/enabled smp/core: Use lockdep to assert IRQs are disabled/enabled timers/hrtimer: Use lockdep to assert IRQs are disabled/enabled timers/nohz: Use lockdep to assert IRQs are disabled/enabled workqueue: Use lockdep to assert IRQs are disabled/enabled irq/softirqs: Use lockdep to assert IRQs are disabled/enabled locking/lockdep: Add IRQs disabled/enabled assertion APIs: lockdep_assert_irqs_enabled()/disabled() locking/pvqspinlock: Implement hybrid PV queued/unfair locks locking/rwlocks: Fix comments x86/paravirt: Set up the virt_spin_lock_key after static keys get initialized block, locking/lockdep: Assign a lock_class per gendisk used for wait_for_completion() workqueue: Remove now redundant lock acquisitions wrt. workqueue flushes ...
2017-11-13Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds8-9/+9
Pull RCU updates from Ingo Molnar: "The main changes in this cycle are: - Documentation updates - RCU CPU stall-warning updates - Torture-test updates - Miscellaneous fixes Size wise the biggest updates are to documentation. Excluding documentation most of the code increase comes from a single commit which expands debugging" * 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (24 commits) srcu: Add parameters to SRCU docbook comments doc: Rewrite confusing statement about memory barriers memory-barriers.txt: Fix typo in pairing example rcu/segcblist: Include rcupdate.h rcu: Add extended-quiescent-state testing advice rcu: Suppress lockdep false-positive ->boost_mtx complaints rcu: Do not include rtmutex_common.h unconditionally torture: Provide TMPDIR environment variable to specify tmpdir rcutorture: Dump writer stack if stalled rcutorture: Add interrupt-disable capability to stall-warning tests rcu: Suppress RCU CPU stall warnings while dumping trace rcu: Turn off tracing before dumping trace rcu: Make RCU CPU stall warnings check for irq-disabled CPUs sched,rcu: Make cond_resched() provide RCU quiescent state sched: Make resched_cpu() unconditional irq_work: Map irq_work_on_queue() to irq_work_on() in !SMP rcu: Create call_rcu_tasks() kthread at boot time rcu: Fix up pending cbs check in rcu_prepare_for_idle memory-barriers: Rework multicopy-atomicity section memory-barriers: Replace uses of "transitive" ...
2017-11-07Merge branch 'linus' into locking/core, to resolve conflictsIngo Molnar20-0/+20
Conflicts: include/linux/compiler-clang.h include/linux/compiler-gcc.h include/linux/compiler-intel.h include/uapi/linux/stddef.h Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-11-02License cleanup: add SPDX GPL-2.0 license identifier to files with no licenseGreg Kroah-Hartman20-0/+20
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-10-25locking/atomics, rcutorture/formal: Prepare for ACCESS_ONCE() removalMark Rutland1-3/+2
For several reasons, it is desirable to use {READ,WRITE}_ONCE() in preference to ACCESS_ONCE(), and new code is expected to use one of the former. So far, there's been no reason to change most existing uses of ACCESS_ONCE(), as these aren't currently harmful. However, for some features it is necessary to instrument reads and writes separately, which is not possible with ACCESS_ONCE(). This distinction is critical to correct operation. The bulk of the kernel code can be transformed via Coccinelle to use {READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(), and not the implementation itself. As such, it has the potential to break homebrew ACCESS_ONCE() macros seen in some user code in the kernel tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7). To avoid fragility if/when that transformation occurs, this patch reworks the definitions of {READ,WRITE}_ONCE() in the rcutorture formal tests, and removes the unused ACCESS_ONCE() helper. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: davem@davemloft.net Cc: linux-arch@vger.kernel.org Cc: mpe@ellerman.id.au Cc: shuah@kernel.org Cc: snitzer@redhat.com Cc: thor.thayer@linux.intel.com Cc: tj@kernel.org Cc: viro@zeniv.linux.org.uk Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1508792849-3115-13-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-10-09torture: Provide TMPDIR environment variable to specify tmpdirPaul E. McKenney8-9/+9
Both rcutorture and locktorture currently place temporary files in /tmp, in keeping with decades-long tradition. However, sometimes it is useful to specify an alternative temporary directory, for example, for space or performance reasons. This commit therefore causes the torture-test scripting to use the path specified in the TMPDIR environment variable, or to fall back to traditional /tmp if this variable is not set. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-08-17Merge branches 'doc.2017.08.17a', 'fixes.2017.08.17a', 'hotplug.2017.07.25b', 'misc.2017.08.17a', 'spin_unlock_wait_no.2017.08.17a', 'srcu.2017.07.27c' and 'torture.2017.07.24c' into HEADPaul E. McKenney10-51/+150
doc.2017.08.17a: Documentation updates. fixes.2017.08.17a: RCU fixes. hotplug.2017.07.25b: CPU-hotplug updates. misc.2017.08.17a: Miscellaneous fixes outside of RCU (give or take conflicts). spin_unlock_wait_no.2017.08.17a: Remove spin_unlock_wait(). srcu.2017.07.27c: SRCU updates. torture.2017.07.24c: Torture-test updates.
2017-08-17rcu: Drive TASKS_RCU directly off of PREEMPTPaul E. McKenney1-1/+1
The actual use of TASKS_RCU is only when PREEMPT, otherwise RCU-sched is used instead. This commit therefore makes synchronize_rcu_tasks() and call_rcu_tasks() available always, but mapped to synchronize_sched() and call_rcu_sched(), respectively, when !PREEMPT. This approach also allows some #ifdefs to be removed from rcutorture. Reported-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Acked-by: Ingo Molnar <mingo@kernel.org>
2017-07-24rcutorture: Use nr_cpus rather than maxcpus to limit test sizePaul E. McKenney3-2/+28
The maxcpus= kernel boot parameter limits the number of CPUs brought online at boot time, but it does nothing to prevent additional CPUs from being brought up later. Placing a hard cap on the total number of CPUs is instead the job of the nr_cpus= boot parameter. This commit therefore switches the configfrag_boot_cpus() shell function from maxcpus= to nr_cpus=. This commit also adds a nr_cpus=43 kernel parameter to RCU's TREE01 test scenario, but retains the maxcpus=8 kernel parameter in order to test the ability of RCU expedited grace periods to handle new CPUs coming online for the first time during grace-period initialization. Finally, this commit makes the torture scheduling allow maxcpus= to override other means of specifying the number of CPUs to allow for. This last works because the torture kernel modules size their workloads based on the number of CPUs present at the start of the test, not the ultimate number of CPUs. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-07-24rcutorture: Place event-traced strings into trace bufferPaul E. McKenney1-1/+1
Strings used in event tracing need to be specially handled, for example, being copied to the trace buffer instead of being pointed to by the trace buffer. Although the TPS() macro can be used to "launder" pointed-to strings, this might not be all that effective within a loadable module. This commit therefore copies rcutorture's strings to the trace buffer. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Steven Rostedt <rostedt@goodmis.org>
2017-07-24rcutorture: Don't wait for kernel when all builds failPaul E. McKenney2-7/+22
Currently, rcutorture groups runs in batches, building each scenario in a given batch, then invoking qemu to run all the kernels in the batch. Of course, if a given scenario's kernel fails to build, there is no qemu run for that scenario. And if all of the kernels in a given batch fail to build, there are no runs, and rcutorture immediately starts on the next batch. But not if --jitter has been specified, which it is by default. In this case, the jitter scripts are started unconditionally, and rcutorture waits for them to complete, even though there are no kernels to run. This commit therefore checks for this situation, and refuses to start jitter unless at least one of the kernels in the batch built successfully. This saves substantial time when all scenarios' kernels fail to build, particularly if a long --duration was specified. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-07-24torture: Add --kconfig argument to kvm.shPaul E. McKenney4-38/+96
Currently, testing a variant of an existing scenario requires editing that scenario's file or creating a new scenario file. This is messy and error prone with respect to changes to scenarios. This commit therefore adds a --kconfig argument to kvm.sh, so that '--kconfig "CONFIG_RCU_TRACE=y CONFIG_RCU_EQS_DEBUG=n" will override those two Kconfig options. In addition, there is now clear precedence: the config fragment overrides CFcommon, and the --kconfig argument overrides both. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-07-24rcutorture: Select CONFIG_PROVE_LOCKING for Tiny SRCU scenarioPaul E. McKenney1-1/+2
This commit selects CONFIG_PROVE_LOCKING for the SRCU-u scenario to get better test coverage. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-07-24rcutorture: Remove obsolete SRCU-C.bootPaul E. McKenney1-1/+0
Classic SRCU is no more, so this commit removes the corresponding rcutorture boot-parameters file. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08rcu: Remove RCU CPU stall warnings from Tiny RCUPaul E. McKenney1-1/+0
Tiny RCU's job is to be tiny, so this commit removes its RCU CPU stall warning code. After this, there is no longer any need for rcu_sched_ctrlblk and rcu_bh_ctrlblk to be in tiny_plugin.h, so this commit also moves them to tiny.c. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08rcu: Eliminate NOCBs CPU-state Kconfig optionsPaul E. McKenney6-8/+5
The CONFIG_RCU_NOCB_CPU_ALL, CONFIG_RCU_NOCB_CPU_NONE, and CONFIG_RCU_NOCB_CPU_ZERO Kconfig options are used only in testing and are redundant with the rcu_nocbs= boot parameter. This commit therefore removes these three Kconfig options and adjusts the rcutorture scripts to use the boot parameter instead. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08rcu: Remove debugfs tracingPaul E. McKenney3-46/+0
RCU's debugfs tracing used to be the only reasonable low-level debug information available, but ftrace and event tracing has since surpassed the RCU debugfs level of usefulness. This commit therefore removes RCU's debugfs tracing. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08srcu: Remove Classic SRCUPaul E. McKenney3-28/+0
Classic SRCU was only ever intended to be a fallback in case of issues with Tree/Tiny SRCU, and the latter two are doing quite well in testing. This commit therefore removes Classic SRCU. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08rcu: Remove SPARSE_RCU_POINTER Kconfig optionPaul E. McKenney1-4/+0
The sparse-based checking for non-RCU accesses to RCU-protected pointers has been around for a very long time, and it is now the only type of sparse-based checking that is optional. This commit therefore makes it unconditional. Reported-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Fengguang Wu <fengguang.wu@intel.com>
2017-06-08rcu: Remove the now-obsolete PROVE_RCU_REPEATEDLY Kconfig optionPaul E. McKenney2-2/+0
The PROVE_RCU_REPEATEDLY Kconfig option was initially added due to the volume of messages from PROVE_RCU: Doing just one per boot would have required excessive numbers of boots to locate them all. However, PROVE_RCU messages are now relatively rare, so there is no longer any reason to need more than one such message per boot. This commit therefore removes the PROVE_RCU_REPEATEDLY Kconfig option. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Ingo Molnar <mingo@kernel.org>
2017-06-08rcu: Remove nohz_full full-system-idle state machinePaul E. McKenney3-8/+1
The NO_HZ_FULL_SYSIDLE full-system-idle capability was added in 2013 by commit 0edd1b1784cb ("nohz_full: Add full-system-idle state machine"), but has not been used. This commit therefore removes it. If it turns out to be needed later, this commit can always be reverted. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Rik van Riel <riel@redhat.com> Cc: Ingo Molnar <mingo@kernel.org> Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-06-08rcu: Remove the RCU_KTHREAD_PRIO Kconfig optionPaul E. McKenney3-2/+1
Anything that can be done with the RCU_KTHREAD_PRIO Kconfig option can also be done with the rcutree.kthread_prio kernel boot parameter. This commit therefore removes this Kconfig option. Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Rik van Riel <riel@redhat.com>