| Message ID | 20220402170609.17790-3-alpernebiyasak@gmail.com |
|---|---|
| State | Accepted |
| Commit | ce12c47b92152e9457d3daa3ddbf53c1cc3de0bb |
| Delegated to: | Simon Glass |
| Headers | show |
| Series | patman: test_util: Prettify test report outputs for python tools | expand |
On Sat, 2 Apr 2022 at 11:06, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote: > > It's possible to request a specific test to run when trying to run a > python tool's tests. If we request a nonexistent test, the unittest > loaders generate a fake test that reports this as an error. However, we > get these fake tests even when the test exists, because test_util can > load tests from multiple places one by one and the test we want only > exists in one. > > The test_util helpers currently remove these fake tests when printing > test results, but that's more of a workaround than a proper solution. > Instead, don't even try to load the missing tests. > > Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com> > --- > > tools/patman/test_util.py | 21 +++++---------------- > 1 file changed, 5 insertions(+), 16 deletions(-) Reviewed-by: Simon Glass <sjg@chromium.org>
On Sat, 2 Apr 2022 at 11:06, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote: > > It's possible to request a specific test to run when trying to run a > python tool's tests. If we request a nonexistent test, the unittest > loaders generate a fake test that reports this as an error. However, we > get these fake tests even when the test exists, because test_util can > load tests from multiple places one by one and the test we want only > exists in one. > > The test_util helpers currently remove these fake tests when printing > test results, but that's more of a workaround than a proper solution. > Instead, don't even try to load the missing tests. > > Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com> > --- > > tools/patman/test_util.py | 21 +++++---------------- > 1 file changed, 5 insertions(+), 16 deletions(-) Reviewed-by: Simon Glass <sjg@chromium.org> Applied to u-boot-dm, thanks!
diff --git a/tools/patman/test_util.py b/tools/patman/test_util.py index 8b2220dbbafc..a4c2a2c3c0b2 100644 --- a/tools/patman/test_util.py +++ b/tools/patman/test_util.py @@ -110,19 +110,6 @@ def report_result(toolname:str, test_name: str, result: unittest.TestResult): test_name: Name of test that was run, or None for all result: A unittest.TestResult object containing the results """ - # Remove errors which just indicate a missing test. Since Python v3.5 If an - # ImportError or AttributeError occurs while traversing name then a - # synthetic test that raises that error when run will be returned. These - # errors are included in the errors accumulated by result.errors. - if test_name: - errors = [] - - for test, err in result.errors: - if ("has no attribute '%s'" % test_name) not in err: - errors.append((test, err)) - result.testsRun -= 1 - result.errors = errors - print(result) for test, err in result.errors: print(test.id(), err) @@ -184,10 +171,12 @@ def run_test_suites(result, debug, verbosity, test_preserve_dirs, processes, preserve_outdirs=test_preserve_dirs and test_name is not None, toolpath=toolpath, verbosity=verbosity) if test_name: - try: + # Since Python v3.5 If an ImportError or AttributeError occurs + # while traversing a name then a synthetic test that raises that + # error when run will be returned. Check that the requested test + # exists, otherwise these errors are included in the results. + if test_name in loader.getTestCaseNames(module): suite.addTests(loader.loadTestsFromName(test_name, module)) - except AttributeError: - continue else: suite.addTests(loader.loadTestsFromTestCase(module)) if use_concurrent and processes != 1:
It's possible to request a specific test to run when trying to run a python tool's tests. If we request a nonexistent test, the unittest loaders generate a fake test that reports this as an error. However, we get these fake tests even when the test exists, because test_util can load tests from multiple places one by one and the test we want only exists in one. The test_util helpers currently remove these fake tests when printing test results, but that's more of a workaround than a proper solution. Instead, don't even try to load the missing tests. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com> --- tools/patman/test_util.py | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-)