Without the quoting, the example will mess up invocations like
$ run_kunit "Something with spaces"
Note: this example isn't valid, but if ever a usecase arises where a
flag argument might have spaces in it, it'll break.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Now that kunit.py's --kunitconfig is repeatable, let's create a file to
hold the various options needed to enable coverage under UML.
This can be used like so:
$ ./tools/testing/kunit/kunit.py run \
--kunitconfig=tools/testing/kunit/configs/all_tests_uml.config \
--kunitconfig=tools/testing/kunit/configs/coverage_uml.config \
--make_options=CC=/usr/bin/gcc-6
which on my system is enough to get coverage working [1].
This is still a clunky command, but far better than before.
[1] at the time of this commit, I get:
Overall coverage rate:
lines......: 11.6% (34112 of 295033 lines)
functions..: 15.3% (3721 of 24368 functions)
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Recent changes have made it so the current set is not sufficient.
Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
hopefully less fragile (esp. given we're tied to GCC 6 and lower).
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Maxime Ripard <maxime@cerno.tech>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Commit 1d71307a6f ("kunit: add unit test for filtering suites by
names") introduced the ability to filter which suites we run via glob.
This change extends it so we can also filter individual test cases
inside of suites as well.
This is quite useful when, e.g.
* trying to run just the tests cases you've just added or are working on
* trying to debug issues with test hermeticity
Examples:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*exec*.parse*'
...
============================================================
======== [PASSED] kunit_executor_test ========
[PASSED] parse_filter_test
============================================================
Testing complete. 1 tests run. 0 failed. 0 crashed.
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*.no_matching_tests'
...
[ERROR] no tests run!
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit.py currently does not make it possible for users to specify module
parameters (/kernel arguments more generally) unless one directly tweaks
the kunit.py code itself.
This hasn't mattered much so far, but this would make it easier to port
existing tests that expose module parameters over to KUnit and/or let
current KUnit tests take advantage of them.
Tested using an kunit internal parameter:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit \
--kernel_args=kunit.filter_glob=kunit_status
...
Testing complete. 2 tests run. 0 failed. 0 crashed. 0 skipped.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Commit b6d5799b0b ("kunit: Add 'kunit_shutdown' option") changes KUnit
to call kernel_halt() by default when done testing.
This fixes the issue with not having .gcda files due to not calling
atexit() handlers, and therefore we can stop recommending people
manually tweak UML code.
The need to use older versions of GCC (<=6) remains however, due to
linktime issues, same as before. Note: There also might still be issues
with .gcda files as well in newer versions.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
This is long overdue.
There are several things that aren't nailed down (in-tree
.kunitconfig's), or partially broken (GCOV on UML), but having them
documented, warts and all, is better than having nothing.
This covers a bunch of the more recent features
* kunit_filter_glob
* kunit.py run --kunitconfig
* slightly more detail on building tests as modules
* CONFIG_KUNIT_DEBUGFS
By my count, the only headline features now not mentioned are the KASAN
integration and KernelCI json output support (kunit.py run --json).
And then it also discusses how to get code coverage reports under UML
and non-UML since this is a question people have repeatedly asked.
Non-UML coverage collection is no different from normal, but we should
probably explicitly call this out.
As for UML, I was able to get it working again with two small hacks.*
E.g. with CONFIG_KUNIT=y && CONFIG_KUNIT_ALL_TESTS=y
Overall coverage rate:
lines......: 15.1% (18294 of 120776 lines)
functions..: 16.8% (1860 of 11050 functions)
Note: this doesn't document --alltests since this is not stable yet.
Hopefully being run more frequently as part of KernelCI will help...
*Using gcc/gcov-6 and not using uml_abort() in os_dump_core().
I've documented these hacks in "Notes" but left TODOs for
brendanhiggins@google.com who tracked down the runtime issue in GCC.
To be clear: these are not issues specific to KUnit, but rather to UML.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
