| Message ID | 20230301012116.1488132-2-frowand.list@gmail.com |
|---|---|
| State | Changes Requested, archived |
| Headers | show |
| Series | of: unittest: option to allow tests that trigger kernel stack dump | expand |
| Context | Check | Description |
|---|---|---|
| robh/checkpatch | warning | total: 0 errors, 1 warnings, 85 lines checked |
| robh/patch-applied | fail | build log |
On 2/28/23 17:21, Frank Rowand wrote: > Commit 74df14cd301a ("of: unittest: add node lifecycle tests") added > some tests that trigger a kernel stack dump. Filtering the boot > messages with scripts/dtc/of_unittest_expect detects that the stack > dump is expected instead of being a test error. > > Test beds might interpret the stack dumps as errors, resulting in > needless debugging and error reports. These test beds are likely > to remove unittests due to these stack dumps. To avoid these problems, > have unittest default to skip the tests that trigger a stack dump. > > Add a kernel cmdline option to not skip those tests. This option can > be used by testers who are able to interpret the stack dumps as not > an error. > > Signed-off-by: Frank Rowand <frowand.list@gmail.com> > --- > drivers/of/unittest.c | 54 ++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 51 insertions(+), 3 deletions(-) > > diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c > index b5a7a31d8bd2..3a9bc2bc4ba1 100644 > --- a/drivers/of/unittest.c > +++ b/drivers/of/unittest.c > @@ -70,6 +70,36 @@ static struct unittest_results { > #define EXPECT_NOT_END(level, fmt, ...) \ > printk(level pr_fmt("EXPECT_NOT / : ") fmt, ##__VA_ARGS__) > > +/* > + * Some tests will cause the kernel to emit a stack dump, aka back trace, > + * when the test is successful. The tests should make it possible for > + * test beds to detect that the trace is not an error via EXPECT_BEGIN(). > + * > + * Most test beds do not process the EXPECT_BEGIN() information and may > + * flag the stack dump as an error, thus reporting a false failure. It > + * is hoped that the KTAP version 4 specification will add the EXPECT_BEGIN() > + * processing to test beds. > + * > + * By default, skip tests that cause a stack dump. Test beds that process > + * EXPECT_BEGIN() information should enable these tests via a kernel boot > + * command line option. > + */ > +static int stackdump_tests_enabled; > + > +static int __init enable_unittest_stackdump(char *str) > +{ > + stackdump_tests_enabled = 1; > + return 0; > +} > + > +static int __init disable_unittest_stackdump(char *str) > +{ > + stackdump_tests_enabled = 0; > + return 0; > +} > +early_param("of_unittest_stackdump", enable_unittest_stackdump); > +early_param("no_of_unittest_stackdump", disable_unittest_stackdump); Does no_of_unittest_stackdump have any benefit or value ? Thanks, Guenter
On 2/28/23 22:07, Guenter Roeck wrote: > On 2/28/23 17:21, Frank Rowand wrote: >> Commit 74df14cd301a ("of: unittest: add node lifecycle tests") added >> some tests that trigger a kernel stack dump. Filtering the boot >> messages with scripts/dtc/of_unittest_expect detects that the stack >> dump is expected instead of being a test error. >> >> Test beds might interpret the stack dumps as errors, resulting in >> needless debugging and error reports. These test beds are likely >> to remove unittests due to these stack dumps. To avoid these problems, >> have unittest default to skip the tests that trigger a stack dump. >> >> Add a kernel cmdline option to not skip those tests. This option can >> be used by testers who are able to interpret the stack dumps as not >> an error. >> >> Signed-off-by: Frank Rowand <frowand.list@gmail.com> >> --- >> drivers/of/unittest.c | 54 ++++++++++++++++++++++++++++++++++++++++--- >> 1 file changed, 51 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c >> index b5a7a31d8bd2..3a9bc2bc4ba1 100644 >> --- a/drivers/of/unittest.c >> +++ b/drivers/of/unittest.c >> @@ -70,6 +70,36 @@ static struct unittest_results { >> #define EXPECT_NOT_END(level, fmt, ...) \ >> printk(level pr_fmt("EXPECT_NOT / : ") fmt, ##__VA_ARGS__) >> +/* >> + * Some tests will cause the kernel to emit a stack dump, aka back trace, >> + * when the test is successful. The tests should make it possible for >> + * test beds to detect that the trace is not an error via EXPECT_BEGIN(). >> + * >> + * Most test beds do not process the EXPECT_BEGIN() information and may >> + * flag the stack dump as an error, thus reporting a false failure. It >> + * is hoped that the KTAP version 4 specification will add the EXPECT_BEGIN() >> + * processing to test beds. >> + * >> + * By default, skip tests that cause a stack dump. Test beds that process >> + * EXPECT_BEGIN() information should enable these tests via a kernel boot >> + * command line option. >> + */ >> +static int stackdump_tests_enabled; >> + >> +static int __init enable_unittest_stackdump(char *str) >> +{ >> + stackdump_tests_enabled = 1; >> + return 0; >> +} >> + >> +static int __init disable_unittest_stackdump(char *str) >> +{ >> + stackdump_tests_enabled = 0; >> + return 0; >> +} >> +early_param("of_unittest_stackdump", enable_unittest_stackdump); >> +early_param("no_of_unittest_stackdump", disable_unittest_stackdump); > > Does no_of_unittest_stackdump have any benefit or value ? I would say no, but it is a common pattern to provide both foo and no_foo. -Frank > > Thanks, > Guenter >
On Wed, Mar 01, 2023 at 10:01:36AM -0600, Frank Rowand wrote: > On 2/28/23 22:07, Guenter Roeck wrote: > > On 2/28/23 17:21, Frank Rowand wrote: > >> Commit 74df14cd301a ("of: unittest: add node lifecycle tests") added > >> some tests that trigger a kernel stack dump. Filtering the boot > >> messages with scripts/dtc/of_unittest_expect detects that the stack > >> dump is expected instead of being a test error. > >> > >> Test beds might interpret the stack dumps as errors, resulting in > >> needless debugging and error reports. These test beds are likely > >> to remove unittests due to these stack dumps. To avoid these problems, > >> have unittest default to skip the tests that trigger a stack dump. > >> > >> Add a kernel cmdline option to not skip those tests. This option can > >> be used by testers who are able to interpret the stack dumps as not > >> an error. > >> > >> Signed-off-by: Frank Rowand <frowand.list@gmail.com> > >> --- > >> drivers/of/unittest.c | 54 ++++++++++++++++++++++++++++++++++++++++--- > >> 1 file changed, 51 insertions(+), 3 deletions(-) > >> > >> diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c > >> index b5a7a31d8bd2..3a9bc2bc4ba1 100644 > >> --- a/drivers/of/unittest.c > >> +++ b/drivers/of/unittest.c > >> @@ -70,6 +70,36 @@ static struct unittest_results { > >> #define EXPECT_NOT_END(level, fmt, ...) \ > >> printk(level pr_fmt("EXPECT_NOT / : ") fmt, ##__VA_ARGS__) > >> +/* > >> + * Some tests will cause the kernel to emit a stack dump, aka back trace, > >> + * when the test is successful. The tests should make it possible for > >> + * test beds to detect that the trace is not an error via EXPECT_BEGIN(). > >> + * > >> + * Most test beds do not process the EXPECT_BEGIN() information and may > >> + * flag the stack dump as an error, thus reporting a false failure. It > >> + * is hoped that the KTAP version 4 specification will add the EXPECT_BEGIN() > >> + * processing to test beds. > >> + * > >> + * By default, skip tests that cause a stack dump. Test beds that process > >> + * EXPECT_BEGIN() information should enable these tests via a kernel boot > >> + * command line option. > >> + */ > >> +static int stackdump_tests_enabled; > >> + > >> +static int __init enable_unittest_stackdump(char *str) > >> +{ > >> + stackdump_tests_enabled = 1; > >> + return 0; > >> +} > >> + > >> +static int __init disable_unittest_stackdump(char *str) > >> +{ > >> + stackdump_tests_enabled = 0; > >> + return 0; > >> +} > >> +early_param("of_unittest_stackdump", enable_unittest_stackdump); > >> +early_param("no_of_unittest_stackdump", disable_unittest_stackdump); > > > > Does no_of_unittest_stackdump have any benefit or value ? > > I would say no, but it is a common pattern to provide both > foo and no_foo. It is? I see one documented example. I see numerous ones that are 'no_foo'. This doesn't scale well if lots of tests need to disable it. Perhaps it should be more generic (at least documentation/naming wise even if the implmentation lives in DT unittest for now). Rob
On 3/24/23 16:43, Rob Herring wrote: > On Wed, Mar 01, 2023 at 10:01:36AM -0600, Frank Rowand wrote: >> On 2/28/23 22:07, Guenter Roeck wrote: >>> On 2/28/23 17:21, Frank Rowand wrote: >>>> Commit 74df14cd301a ("of: unittest: add node lifecycle tests") added >>>> some tests that trigger a kernel stack dump. Filtering the boot >>>> messages with scripts/dtc/of_unittest_expect detects that the stack >>>> dump is expected instead of being a test error. >>>> >>>> Test beds might interpret the stack dumps as errors, resulting in >>>> needless debugging and error reports. These test beds are likely >>>> to remove unittests due to these stack dumps. To avoid these problems, >>>> have unittest default to skip the tests that trigger a stack dump. >>>> >>>> Add a kernel cmdline option to not skip those tests. This option can >>>> be used by testers who are able to interpret the stack dumps as not >>>> an error. >>>> >>>> Signed-off-by: Frank Rowand <frowand.list@gmail.com> >>>> --- >>>> drivers/of/unittest.c | 54 ++++++++++++++++++++++++++++++++++++++++--- >>>> 1 file changed, 51 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c >>>> index b5a7a31d8bd2..3a9bc2bc4ba1 100644 >>>> --- a/drivers/of/unittest.c >>>> +++ b/drivers/of/unittest.c >>>> @@ -70,6 +70,36 @@ static struct unittest_results { >>>> #define EXPECT_NOT_END(level, fmt, ...) \ >>>> printk(level pr_fmt("EXPECT_NOT / : ") fmt, ##__VA_ARGS__) >>>> +/* >>>> + * Some tests will cause the kernel to emit a stack dump, aka back trace, >>>> + * when the test is successful. The tests should make it possible for >>>> + * test beds to detect that the trace is not an error via EXPECT_BEGIN(). >>>> + * >>>> + * Most test beds do not process the EXPECT_BEGIN() information and may >>>> + * flag the stack dump as an error, thus reporting a false failure. It >>>> + * is hoped that the KTAP version 4 specification will add the EXPECT_BEGIN() >>>> + * processing to test beds. >>>> + * >>>> + * By default, skip tests that cause a stack dump. Test beds that process >>>> + * EXPECT_BEGIN() information should enable these tests via a kernel boot >>>> + * command line option. >>>> + */ >>>> +static int stackdump_tests_enabled; >>>> + >>>> +static int __init enable_unittest_stackdump(char *str) >>>> +{ >>>> + stackdump_tests_enabled = 1; >>>> + return 0; >>>> +} >>>> + >>>> +static int __init disable_unittest_stackdump(char *str) >>>> +{ >>>> + stackdump_tests_enabled = 0; >>>> + return 0; >>>> +} >>>> +early_param("of_unittest_stackdump", enable_unittest_stackdump); >>>> +early_param("no_of_unittest_stackdump", disable_unittest_stackdump); >>> >>> Does no_of_unittest_stackdump have any benefit or value ? >> >> I would say no, but it is a common pattern to provide both >> foo and no_foo. > > It is? I see one documented example. I see numerous ones that are > 'no_foo'. I reconsidered. I plan to remove the no_of_unittest_stackdump in v2, updated to the current kernel version. -Frank > > This doesn't scale well if lots of tests need to disable it. Perhaps it > should be more generic (at least documentation/naming wise even if the > implmentation lives in DT unittest for now). > > Rob
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c index b5a7a31d8bd2..3a9bc2bc4ba1 100644 --- a/drivers/of/unittest.c +++ b/drivers/of/unittest.c @@ -70,6 +70,36 @@ static struct unittest_results { #define EXPECT_NOT_END(level, fmt, ...) \ printk(level pr_fmt("EXPECT_NOT / : ") fmt, ##__VA_ARGS__) +/* + * Some tests will cause the kernel to emit a stack dump, aka back trace, + * when the test is successful. The tests should make it possible for + * test beds to detect that the trace is not an error via EXPECT_BEGIN(). + * + * Most test beds do not process the EXPECT_BEGIN() information and may + * flag the stack dump as an error, thus reporting a false failure. It + * is hoped that the KTAP version 4 specification will add the EXPECT_BEGIN() + * processing to test beds. + * + * By default, skip tests that cause a stack dump. Test beds that process + * EXPECT_BEGIN() information should enable these tests via a kernel boot + * command line option. + */ +static int stackdump_tests_enabled; + +static int __init enable_unittest_stackdump(char *str) +{ + stackdump_tests_enabled = 1; + return 0; +} + +static int __init disable_unittest_stackdump(char *str) +{ + stackdump_tests_enabled = 0; + return 0; +} +early_param("of_unittest_stackdump", enable_unittest_stackdump); +early_param("no_of_unittest_stackdump", disable_unittest_stackdump); + static void __init of_unittest_find_node_by_name(void) { struct device_node *np; @@ -3047,19 +3077,26 @@ static void __init of_unittest_lifecycle(void) of_node_put(np); } + if (!stackdump_tests_enabled) + goto out_skip_stackdump_tests; + EXPECT_BEGIN(KERN_INFO, "OF: ERROR: of_node_release() detected bad of_node_put() on /testcase-data/refcount-node"); /* * refcount is now one, decrementing to zero will result in a call to * of_node_release() to free the node's memory, which should result - * in an error + * in an error. + * + * A refcount of zero will also trigger errors in + * of_unittest_check_node_linkage(), so after this block of test, + * will directly manipulate the devicetree to remove this node. + * (See the code marked as "WARNING: EVIL, EVIL, EVIL:".) */ unittest(1, "/testcase-data/refcount-node is one"); of_node_put(np); EXPECT_END(KERN_INFO, "OF: ERROR: of_node_release() detected bad of_node_put() on /testcase-data/refcount-node"); - /* * expect stack trace for subsequent of_node_put(): * __refcount_sub_and_test() calls: @@ -3138,9 +3175,20 @@ static void __init of_unittest_lifecycle(void) return; +out_skip_stackdump_tests: + /* + * Pass test since we don't yet have a way to annotate reason for an + * intentional skip until implementing the KTAP format. + */ + unittest(1, "Lifecycle stackdump tests skipped\n"); + return; + out_skip_tests: + /* skip is due to an earlier failure */ + unittest(0, "One or more lifecycle tests skipped due to previous fails\n"); +#else + unittest(1, "Lifecycle tests skipped because CONFIG_DYNAMIC=n\n"); #endif - unittest(0, "One or more lifecycle tests skipped\n"); } #ifdef CONFIG_OF_OVERLAY
Commit 74df14cd301a ("of: unittest: add node lifecycle tests") added some tests that trigger a kernel stack dump. Filtering the boot messages with scripts/dtc/of_unittest_expect detects that the stack dump is expected instead of being a test error. Test beds might interpret the stack dumps as errors, resulting in needless debugging and error reports. These test beds are likely to remove unittests due to these stack dumps. To avoid these problems, have unittest default to skip the tests that trigger a stack dump. Add a kernel cmdline option to not skip those tests. This option can be used by testers who are able to interpret the stack dumps as not an error. Signed-off-by: Frank Rowand <frowand.list@gmail.com> --- drivers/of/unittest.c | 54 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 51 insertions(+), 3 deletions(-)