| Age | Commit message (Collapse) | Author |
|---|
|
Change every shebang which does not need an argument to use /usr/bin/env.
This is needed as not every distro has everything under /usr/bin,
sometimes not even bash.
Signed-off-by: Finn Behrens <me@kloenk.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
For simplcity, strip all trailing whitespace from parsed output.
I imagine no one is printing out meaningful trailing whitespace via
KUNIT_FAIL() or similar, and that if they are, they really shouldn't.
`isolate_kunit_output()` yielded liens with trailing \n, which results
in artifacty output like this:
$ ./tools/testing/kunit/kunit.py run
[16:16:46] [FAILED] example_simple_test
[16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[16:16:46] Expected 1 + 1 == 3, but
[16:16:46] 1 + 1 == 2
[16:16:46] 3 == 3
[16:16:46] not ok 1 - example_simple_test
[16:16:46]
After this change:
[16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[16:16:46] Expected 1 + 1 == 3, but
[16:16:46] 1 + 1 == 2
[16:16:46] 3 == 3
[16:16:46] not ok 1 - example_simple_test
[16:16:46]
We should *not* be expecting lines to end with \n in kunit_tool_test.py
for this reason.
Do the same for `raw_output()` as well which suffers from the same
issue.
This is a followup to [1], but rebased onto kunit-fixes to pick up the
other raw_output() fix and fixes for kunit_tool_test.py.
[1] https://lore.kernel.org/linux-kselftest/20201020233219.4146059-1-dlatypov@google.com/
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Some tests logs for kunit_tool tests are missing their test plans
causing their tests to fail; fix this by adding the test plans.
Fixes: 45dcbb6f5ef7 ("kunit: test: add test plan to KUnit TAP format")
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Add a --json flag, which when specified generates JSON formatted test
results conforming to the KernelCI API test_group spec[1]. The user can
use the new flag to specify a filename to print the json formatted
results to.
Link[1]: https://api.kernelci.org/schema-test-group.html#post
Signed-off-by: Heidi Fahim <heidifahim@google.com>
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull kunit updates from Shuah Khan:
- Add a generic kunit_resource API extending it to support resources
that are passed in to kunit in addition kunit allocated resources. In
addition, KUnit resources are now refcounted to avoid passed in
resources being released while in use by kunit.
- Add support for named resources.
- Important bug fixes from Brendan Higgins and Will Chen
* tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
kunit: tool: fix improper treatment of file location
kunit: tool: fix broken default args in unit tests
kunit: capture stderr on all make subprocess calls
Documentation: kunit: Remove references to --defconfig
kunit: add support for named resources
kunit: generalize kunit_resource API beyond allocated resources
|
|
Commit 01397e822af4 ("kunit: Fix TabError, remove defconfig code and
handle when there is no kunitconfig") and commit 45ba7a893ad8 ("kunit:
kunit_tool: Separate out config/build/exec/parse") introduced two
closely related issues which built off of each other: they excessively
created the build directory when not present and modified a constant
(constants in Python only exist by convention).
Together these issues broken a number of unit tests for KUnit tool, so
fix them.
Fixed up commit log to fic checkpatch commit description style error.
Shuah Khan <skhan@linuxfoundation.org>
Fixes: 01397e822af4 ("kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig")
Fixes: 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse")
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Commit ddbd60c779b4 ("kunit: use --build_dir=.kunit as default") changed
the default build directory for KUnit tests, but failed to update
associated unit tests for kunit_tool, so update them.
Fixes: ddbd60c779b4 ("kunit: use --build_dir=.kunit as default")
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Currently, if the kernel is configured incorrectly or if it crashes before any
kunit tests are run, kunit finishes without error, reporting
that 0 test cases were run.
To fix this, an error is shown when the tap header is not found, which
indicates that kunit was not able to run at all.
Signed-off-by: Uriel Guajardo <urielguajardo@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Add new subcommands to kunit.py to allow stages of the existing 'run'
subcommand to be run independently:
- 'config': Verifies that .config is a subset of .kunitconfig
- 'build': Compiles a UML kernel for KUnit
- 'exec': Runs the kernel, and outputs the test results.
- 'parse': Parses test results from a file or stdin
The 'run' command continues to behave as before.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Rework kunit_tool in order to allow .kunitconfig files to better enforce
that disabled items in .kunitconfig are disabled in the generated
.config.
Previously, kunit_tool simply enforced that any line present in
.kunitconfig was also present in .config, but this could cause problems
if a config option was disabled in .kunitconfig, but not listed in .config
due to (for example) having disabled dependencies.
To fix this, re-work the parser to track config names and values, and
require values to match unless they are explicitly disabled with the
"CONFIG_x is not set" comment (or by setting its value to 'n'). Those
"disabled" values will pass validation if omitted from the .config, but
not if they have a different value.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Implemented the functionality to run all KUnit tests through kunit_tool
by specifying an --alltests flag, which builds UML with allyesconfig
enabled, and consequently runs every KUnit test. A new function was
added to kunit_kernel: make_allyesconfig.
Firstly, if --alltests is specified, kunit.py triggers build_um_kernel
which call make_allyesconfig. This function calls the make command,
disables the broken configs that would otherwise prevent UML from
building, then starts the kernel with all possible configurations
enabled. All stdout and stderr is sent to test.log and read from there
then fed through kunit_parser to parse the tests to the user. Also added
a signal_handler in case kunit is interrupted while running.
Tested: Run under different conditions such as testing with
--raw_output, testing program interrupt then immediately running kunit
again without --alltests and making sure to clean the console.
Signed-off-by: Heidi Fahim <heidifahim@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Previously, kunit_parser did not properly handle kunit TAP output that
- had any prefixes (generated from different configs e.g.
CONFIG_PRINTK_TIME)
- had unrelated kernel output mixed in the middle of
it, which has shown up when testing with allyesconfig
To remove prefixes, the parser looks for the first line that includes
TAP output, "TAP version 14". It then determines the length of the
string before this sequence, and strips that number of characters off
the beginning of the following lines until the last KUnit output line is
reached.
These fixes have been tested with additional tests in the
KUnitParseTest and their associated logs have also been added.
Signed-off-by: Heidi Fahim <heidifahim@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
This commit adds kunit tool test for the '--build_dir' option.
Signed-off-by: SeongJae Park <sjpark@amazon.de>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
'kunit' writes the 'test.log' under the kernel source directory even
though a 'build_dir' option is given. As users who use the option might
expect the outputs to be placed under the specified directory, this
commit modifies the logic to write the log file under the 'build_dir'.
Signed-off-by: SeongJae Park <sjpark@amazon.de>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
Assert in test_run_timeout was not updated with the build_dir argument
and caused the following error:
AssertionError: Expected call: run_kernel(timeout=3453)
Actual call: run_kernel(build_dir=None, timeout=3453)
Needed to update kunit_tool_test to reflect this fix
https://lkml.org/lkml/2019/9/6/351
Signed-off-by: Heidi Fahim <heidifahim@google.com>
Reviewed-by: SeongJae Park <sjpark@amazon.de>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
The ultimate goal is to create minimal isolated test binaries; in the
meantime we are using UML to provide the infrastructure to run tests, so
define an abstract way to configure and run tests that allow us to
change the context in which tests are built without affecting the user.
This also makes pretty and dynamic error reporting, and a lot of other
nice features easier.
kunit_config.py:
- parse .config and Kconfig files.
kunit_kernel.py: provides helper functions to:
- configure the kernel using kunitconfig.
- build the kernel with the appropriate configuration.
- provide function to invoke the kernel and stream the output back.
kunit_parser.py: parses raw logs returned out by kunit_kernel and
displays them in a user friendly way.
test_data/*: samples of test data for testing kunit.py, kunit_config.py,
etc.
Signed-off-by: Felix Guo <felixguoxiuping@gmail.com>
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
