| Message ID | 159363474417.929474.570677654666099808.stgit@firesoul |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | BPF selftests test runner test_progs improvement for scripting | expand |
On Wed, Jul 1, 2020 at 1:19 PM Jesper Dangaard Brouer <brouer@redhat.com> wrote: > > The program test_progs have some very useful ability to specify a list of > test name substrings for selecting which tests to run. > > This patch add the ability to list the selected test names without running > them. This is practical for seeing which tests gets selected with given > select arguments (which can also contain a exclude list via --name-blacklist). > > This output can also be used by shell-scripts in a for-loop: > > for N in $(./test_progs --list -t xdp); do \ > ./test_progs -t $N 2>&1 > result_test_${N}.log & \ > done ; wait > > This features can also be used for looking up a test number and returning > a testname. If the selection was empty then a shell EXIT_FAILURE is > returned. This is useful for scripting. e.g. like this: > > n=1; > while [ $(./test_progs --list -n $n) ] ; do \ > ./test_progs -n $n ; n=$(( n+1 )); \ > done > > Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> > --- > tools/testing/selftests/bpf/test_progs.c | 18 ++++++++++++++++++ > tools/testing/selftests/bpf/test_progs.h | 1 + > 2 files changed, 19 insertions(+) > [...] > + if (env.list_test_names) { > + if (env.succ_cnt == 0) > + env.fail_cnt = 1; > + goto out; > + } > + How about making it a failure if no tests were selected in general, regardless of --count, --list or normal case? It seems sensible that if you specified wrong selection, that's not what you wanted? > fprintf(stdout, "Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n", > env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt); > > diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h > index 0030584619c3..ec31f382e7fd 100644 > --- a/tools/testing/selftests/bpf/test_progs.h > +++ b/tools/testing/selftests/bpf/test_progs.h > @@ -67,6 +67,7 @@ struct test_env { > > bool jit_enabled; > bool get_test_cnt; > + bool list_test_names; > > struct prog_test_def *test; > FILE *stdout; > >
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 433f4dbf09ca..3345cd977c10 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -367,6 +367,7 @@ enum ARG_KEYS { ARG_VERIFIER_STATS = 's', ARG_VERBOSE = 'v', ARG_GET_TEST_CNT = 'c', + ARG_LIST_TEST_NAMES = 'l', }; static const struct argp_option opts[] = { @@ -382,6 +383,8 @@ static const struct argp_option opts[] = { "Verbose output (use -vv or -vvv for progressively verbose output)" }, { "count", ARG_GET_TEST_CNT, NULL, 0, "Get number of selected top-level tests " }, + { "list", ARG_LIST_TEST_NAMES, NULL, 0, + "List test names that would run (without running them) " }, {}, }; @@ -517,6 +520,9 @@ static error_t parse_arg(int key, char *arg, struct argp_state *state) case ARG_GET_TEST_CNT: env->get_test_cnt = true; break; + case ARG_LIST_TEST_NAMES: + env->list_test_names = true; + break; case ARGP_KEY_ARG: argp_usage(state); break; @@ -665,6 +671,12 @@ int main(int argc, char **argv) continue; } + if (env.list_test_names) { + fprintf(env.stdout, "%s\n", test->test_name); + env.succ_cnt++; + continue; + } + test->run_test(); /* ensure last sub-test is finalized properly */ if (test->subtest_name) @@ -694,6 +706,12 @@ int main(int argc, char **argv) goto out; } + if (env.list_test_names) { + if (env.succ_cnt == 0) + env.fail_cnt = 1; + goto out; + } + fprintf(stdout, "Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n", env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt); diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h index 0030584619c3..ec31f382e7fd 100644 --- a/tools/testing/selftests/bpf/test_progs.h +++ b/tools/testing/selftests/bpf/test_progs.h @@ -67,6 +67,7 @@ struct test_env { bool jit_enabled; bool get_test_cnt; + bool list_test_names; struct prog_test_def *test; FILE *stdout;
The program test_progs have some very useful ability to specify a list of test name substrings for selecting which tests to run. This patch add the ability to list the selected test names without running them. This is practical for seeing which tests gets selected with given select arguments (which can also contain a exclude list via --name-blacklist). This output can also be used by shell-scripts in a for-loop: for N in $(./test_progs --list -t xdp); do \ ./test_progs -t $N 2>&1 > result_test_${N}.log & \ done ; wait This features can also be used for looking up a test number and returning a testname. If the selection was empty then a shell EXIT_FAILURE is returned. This is useful for scripting. e.g. like this: n=1; while [ $(./test_progs --list -n $n) ] ; do \ ./test_progs -n $n ; n=$(( n+1 )); \ done Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> --- tools/testing/selftests/bpf/test_progs.c | 18 ++++++++++++++++++ tools/testing/selftests/bpf/test_progs.h | 1 + 2 files changed, 19 insertions(+)