Patchwork [1/2] task.py: allow expected failures

login
register
mail settings
Submitter kim.hansen@prevas.dk
Date Nov. 12, 2013, 12:55 p.m.
Message ID <7edb8c00d72b4760127670f8854bcb138515ab27.1384260784.git.kiho@prevas.dk>
Download mbox | patch
Permalink /patch/290628/
State Changes Requested
Delegated to: Esben Haabendal
Headers show

Comments

kim.hansen@prevas.dk - Nov. 12, 2013, 12:55 p.m.
From: Kim Højgaard-Hansen <kiho@prevas.dk>

Make it possible to have test cases which are expected to fail
as part of a bake build e.g. oe bake world. When a task is marked
as expected to fail, the bake will continue when the task actually
fails, and abort if the task should succeed.
---
 lib/oelite/task.py | 29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)
Esben Haabendal - Nov. 28, 2013, 8 a.m.
<kim.hansen@prevas.dk> writes:

> From: Kim Højgaard-Hansen <kiho@prevas.dk>
>
> Make it possible to have test cases which are expected to fail
> as part of a bake build e.g. oe bake world. When a task is marked
> as expected to fail, the bake will continue when the task actually
> fails, and abort if the task should succeed.
> ---
>  lib/oelite/task.py | 29 ++++++++++++++++++++++-------
>  1 file changed, 22 insertions(+), 7 deletions(-)
>
> diff --git a/lib/oelite/task.py b/lib/oelite/task.py
> index 2dd08c8..49bcd20 100644
> --- a/lib/oelite/task.py
> +++ b/lib/oelite/task.py
> @@ -223,23 +223,38 @@ class OEliteTask:
>  
>          try:
>              for prefunc in self.get_prefuncs():
> -                print "running prefunc", prefunc
> +                debug("running prefunc: %", prefunc)

Please do not convert print statement to debug() calls.  I hope to
introduce a nicer way of controlling debug/info/warning/error messages,
which will most likely be based on the logging module API.  But I very
much plan on keeping support for all print statements being treated as
debug messages, as I believe it is a very nice KISS approach :)

>                  self.do_cleandirs(prefunc)
>                  wd = self.do_dirs(prefunc)
>                  if not prefunc.run(wd or cwd):
> -                    return False
> +                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                        return True

Returning True here will terminate the task and not continue.  But do we
really want to apply TASK_EXPECTED_FAILURE to prefuncs and postfuncs?  I
am not sure how to do that in a sane way.

> +                    else:
> +                        return False
>              try:
>                  if not function.run(cwd):
> -                    return False
> +                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                        return True

Again, this will abort without running the postfuncs. Is that intended?

> +                    else:
> +                        return False
>              except oebakery.FatalError:
> -                return False
> +                if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                    return True
> +                else:
> +                    return False

We might want to still see FatalError as errors when
TASK_EXPECTED_FAILURE is set.

>              for postfunc in self.get_postfuncs():
> -                print "running postfunc", postfunc
> +                debug("running postfunc %s",postfunc)
>                  self.do_cleandirs(postfunc)
>                  wd = self.do_dirs(postfunc)
>                  if not postfunc.run(wd or cwd):
> -                    return False
> -            return True
> +                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                        return True
> +                    else:
> +                        return False

We really need to consider how to handle prefuncs/postfuncs.  Let's say
you have a task with two prefuncs and two postfuncs, and have set
TASK_EXPECTED_FAILURE.

Do we expect all 5 functions two fail?  Do we expect all prefuncs and
postfuncs to succeed?  Do we allow prefuncs to both fail and succeed?

> +            if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                return False
> +            else:
> +                return True
>  
>          finally:
>              # Cleanup stdin, stdout and stderr redirection

/Esben

Patch

diff --git a/lib/oelite/task.py b/lib/oelite/task.py
index 2dd08c8..49bcd20 100644
--- a/lib/oelite/task.py
+++ b/lib/oelite/task.py
@@ -223,23 +223,38 @@  class OEliteTask:
 
         try:
             for prefunc in self.get_prefuncs():
-                print "running prefunc", prefunc
+                debug("running prefunc: %", prefunc)
                 self.do_cleandirs(prefunc)
                 wd = self.do_dirs(prefunc)
                 if not prefunc.run(wd or cwd):
-                    return False
+                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
+                        return True
+                    else:
+                        return False
             try:
                 if not function.run(cwd):
-                    return False
+                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
+                        return True
+                    else:
+                        return False
             except oebakery.FatalError:
-                return False
+                if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
+                    return True
+                else:
+                    return False
             for postfunc in self.get_postfuncs():
-                print "running postfunc", postfunc
+                debug("running postfunc %s",postfunc)
                 self.do_cleandirs(postfunc)
                 wd = self.do_dirs(postfunc)
                 if not postfunc.run(wd or cwd):
-                    return False
-            return True
+                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
+                        return True
+                    else:
+                        return False
+            if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
+                return False
+            else:
+                return True
 
         finally:
             # Cleanup stdin, stdout and stderr redirection