linux-dev/tools/testing/kunit/configs, branch master

kunit: tool: rename all_test_uml.config, use it for --alltests

2022-09-30T19:22:55Z

Context: 1. all_tests_uml.config used to be UML specific back when users to manually specify CONFIG_VIRTIO_UML=y to enable CONFIG_PCI=y. 2. --alltests used allyesconfig along with a curated list of options to disable. It's only ever worked for brief periods of time and has perennially been broken due to compile issues. Now all_tests_uml.config should work across ~all architectures. Let's instead use this to implement --alltests. Note: if anyone was using all_tests_uml.config, this change breaks them. I think that's unlikely since it was added in 5.19 and was a lot to type: --kunitconfig=tools/testing/kunit/configs/all_tests_uml.config. We could make it a symlink to the new name, but I don't think the caution is warranted here. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Signed-off-by: Shuah Khan

kunit: tool: remove UML specific options from all_tests_uml.config

2022-09-30T19:22:37Z

Commit 6fc3a8636a7b ("kunit: tool: Enable virtio/PCI by default on UML") made it so we enable these options by default for UML. Specifying them here is now redundant. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Signed-off-by: Shuah Khan

kunit: tool: Enable virtio/PCI by default on UML

2022-07-08T17:22:29Z

There are several tests which depend on PCI, and hence need a bunch of extra options to run under UML. This makes it awkward to give configuration instructions (whether in documentation, or as part of a .kunitconfig file), as two separate, incompatible sets of config options are required for UML and "most other architectures". For non-UML architectures, it's possible to add default kconfig options via the qemu_config python files, but there's no equivalent for UML. Add a new tools/testing/kunit/configs/arch_uml.config file containing extra kconfig options to use on UML. Tested-by: José Expósito Reviewed-by: Daniel Latypov Signed-off-by: David Gow Reviewed-by: Brendan Higgins Signed-off-by: Daniel Latypov Signed-off-by: Shuah Khan

kunit: add coverage_uml.config to enable GCOV on UML

2022-07-08T00:06:36Z

Now that kunit.py's --kunitconfig is repeatable, let's create a file to hold the various options needed to enable coverage under UML. This can be used like so: $ ./tools/testing/kunit/kunit.py run \ --kunitconfig=tools/testing/kunit/configs/all_tests_uml.config \ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config \ --make_options=CC=/usr/bin/gcc-6 which on my system is enough to get coverage working [1]. This is still a clunky command, but far better than before. [1] at the time of this commit, I get: Overall coverage rate: lines......: 11.6% (34112 of 295033 lines) functions..: 15.3% (3721 of 24368 functions) Signed-off-by: Daniel Latypov Reviewed-by: David Gow Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: Add list of all valid test configs on UML

2022-05-16T19:23:33Z

It's often desirable (particularly in test automation) to run as many tests as possible. This config enables all the tests which work as builtins under UML at present, increasing the total tests run from 156 to 342 (not counting 36 'skipped' tests). They can be run with: ./tools/testing/kunit/kunit.py run --kunitconfig=./tools/testing/kunit/configs/all_tests_uml.config This acts as an in-between point between the KUNIT_ALL_TESTS config (which enables only tests whose dependencies are already enabled), and the kunit_tool --alltests option, which tries to use allyesconfig, taking a very long time to build and breaking very often. Signed-off-by: David Gow Tested-by: Daniel Latypov Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: Remove the unused all_tests.config

2021-06-23T23:49:41Z

This isn't used anywhere. While it's possible that people were manually referencing it, the new default config (in default.config in the same path) provides equivalent functionality. Signed-off-by: David Gow Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: Move default config from arch/um -> tools/testing/kunit

2021-06-23T23:49:17Z

The default .kunitconfig file is currently kept in arch/um/configs/kunit_defconfig, but -- with the impending QEMU patch -- will no-longer be exclusively used for UML-based kernels. Move it alongside the other KUnit configs in tools/testing/kunit/configs, and give it a name which matches the existing all_tests.config and broken_on_uml.config files. Also update the Getting Started documentation to point to the new file. Signed-off-by: David Gow Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: Disable PAGE_POISONING under --alltests

2021-03-11T21:37:37Z

kunit_tool maintains a list of config options which are broken under UML, which we exclude from an otherwise 'make ARCH=um allyesconfig' build used to run all tests with the --alltests option. Something in UML allyesconfig is causing segfaults when page poisining is enabled (and is poisoning with a non-zero value). Previously, this didn't occur, as allyesconfig enabled the CONFIG_PAGE_POISONING_ZERO option, which worked around the problem by zeroing memory. This option has since been removed, and memory is now poisoned with 0xAA, which triggers segfaults in many different codepaths, preventing UML from booting. Note that we have to disable both CONFIG_PAGE_POISONING and CONFIG_DEBUG_PAGEALLOC, as the latter will 'select' the former on architectures (such as UML) which don't implement __kernel_map_pages(). Ideally, we'd fix this properly by tracking down the real root cause, but since this is breaking KUnit's --alltests feature, it's worth disabling there in the meantime so the kernel can boot to the point where tests can actually run. Fixes: f289041ed4cf ("mm, page_poison: remove CONFIG_PAGE_POISONING_ZERO") Signed-off-by: David Gow Acked-by: Vlastimil Babka Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: fix --alltests flag

2020-09-23T21:52:11Z

Alltests flag evidently stopped working when run from outside of the root of the source tree, so fix that. Also add an additional broken config to the broken_on_uml config. Signed-off-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: Run all KUnit tests through allyesconfig

2020-03-20T22:17:39Z

Implemented the functionality to run all KUnit tests through kunit_tool by specifying an --alltests flag, which builds UML with allyesconfig enabled, and consequently runs every KUnit test. A new function was added to kunit_kernel: make_allyesconfig. Firstly, if --alltests is specified, kunit.py triggers build_um_kernel which call make_allyesconfig. This function calls the make command, disables the broken configs that would otherwise prevent UML from building, then starts the kernel with all possible configurations enabled. All stdout and stderr is sent to test.log and read from there then fed through kunit_parser to parse the tests to the user. Also added a signal_handler in case kunit is interrupted while running. Tested: Run under different conditions such as testing with --raw_output, testing program interrupt then immediately running kunit again without --alltests and making sure to clean the console. Signed-off-by: Heidi Fahim Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan