[meta-swupdate,02/12] swupdate: ensure scripts are added before images

In case of streamed images, it is essential to add scripts before
images to the swu archive. Otherwise the preinstall script function
arrives after the image installation step.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 classes/swupdate.bbclass | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

Hi Adrian,

On 23/10/19 23:10, Adrian Freihofer wrote:
> In case of streamed images, it is essential to add scripts before
> images to the swu archive. Otherwise the preinstall script function
> arrives after the image installation step.

I think there are not drawbacks to set the order as you proposed.
However, it is quite weak.

A shell script preinstall as you are clearly want to move forward won't
run before an artifact that is streamed. Streaming has always the
priority, and a shell script will be just temporary stored in advance,
but not executed. If you need a preinstall script that always run before
any installation, you should use a Lua embedded-script in sw-description.

> 
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
>  classes/swupdate.bbclass | 23 +++++++++++++++++++++--
>  1 file changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
> index 79c9aae..cb1122c 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -90,17 +90,31 @@ python do_swuimage () {
>      shutil.copyfile(os.path.join(workdir, "sw-description"), os.path.join(s, "sw-description"))
>      fetch = bb.fetch2.Fetch([], d)
>      list_for_cpio = ["sw-description"]
> +    list_for_cpio_scripts = []  # scripts: pre-scripts must arive before streamed images
> +    list_for_cpio_images = []
>  
>      if d.getVar('SWUPDATE_SIGNING', True):
>          list_for_cpio.append('sw-description.sig')
>  
> +    def is_script(filename):
> +        try:
> +            with open(filename, 'r') as fh:
> +                if fh.read(2) == '#!':
> +                    return True
> +        except UnicodeDecodeError:
> +            pass
> +        return False
> +
>      # Add files listed in SRC_URI to the swu file
>      for url in fetch.urls:
>          local = fetch.localpath(url)
>          filename = os.path.basename(local)
>          if (filename != 'sw-description'):
>              shutil.copyfile(local, os.path.join(s, "%s" % filename ))
> -            list_for_cpio.append(filename)
> +            if is_script(local):
> +                list_for_cpio_scripts.append(filename)
> +            else:
> +                list_for_cpio_images.append(filename)
>  
>      def add_image_to_swu(deploydir, imagename, s):
>          src = os.path.join(deploydir, imagename)
> @@ -109,7 +123,10 @@ python do_swuimage () {
>          target_imagename = os.path.basename(imagename)  # allow images in subfolders of DEPLOY_DIR_IMAGE
>          dst = os.path.join(s, target_imagename)
>          shutil.copyfile(src, dst)
> -        list_for_cpio.append(target_imagename)
> +        if is_script(dst):
> +            list_for_cpio_scripts.append(target_imagename)
> +        else:
> +            list_for_cpio_images.append(target_imagename)
>          return True
>  
>      # Search for images listed in SWUPDATE_IMAGES in the DEPLOY directory.
> @@ -137,6 +154,8 @@ python do_swuimage () {
>              if not add_image_to_swu(deploydir, image, s):
>                  bb.fatal("swupdate cannot find %s image file" % image)
>  
> +    list_for_cpio = list_for_cpio + list_for_cpio_scripts + list_for_cpio_images
> +
>      prepare_sw_description(d, s, list_for_cpio)
>  
>      line = 'for i in ' + ' '.join(list_for_cpio) + '; do echo $i;done | cpio -ov -H crc >' + os.path.join(imgdeploydir,d.getVar('IMAGE_NAME', True) + '.swu')
> 

Best regards,
Stefano Babic

Hi Stefano

Am Samstag, 26. Oktober 2019 15:29:24 UTC+2 schrieb Stefano Babic:
>
> Hi Adrian, 
>
> On 23/10/19 23:10, Adrian Freihofer wrote: 
> > In case of streamed images, it is essential to add scripts before 
> > images to the swu archive. Otherwise the preinstall script function 
> > arrives after the image installation step. 
>
> I think there are not drawbacks to set the order as you proposed. 
> However, it is quite weak. 
>
> A shell script preinstall as you are clearly want to move forward won't 
> run before an artifact that is streamed. Streaming has always the 
> priority, and a shell script will be just temporary stored in advance, 
> but not executed. If you need a preinstall script that always run before 
> any installation, you should use a Lua embedded-script in sw-description. 
>
 
Thank you for the explanation, I think that this patch should be skipped. 
It does not add any value. It might be useful, if prescripts would be 
executed before images are streamed. This was my assumption.
The mistake I made, was to change two things in one. First I tried to 
remove the force_ro flag by a shell script. Since this did not work and the 
log file told me that the shell script gets executed to late, my assumption 
was that the patch here would solve the problem. Then I added native 
support (patch you merged last week) to swupdate to remove the force_ro 
flag, which finally solved the issue. I did not realize that the shell 
script did not work even after enforcing the order in the swu file.

Regards,
Adrian
 

> > 
> > Signed-off-by: Adrian Freihofer <adrian....@siemens.com <javascript:>> 
> > --- 
> >  classes/swupdate.bbclass | 23 +++++++++++++++++++++-- 
> >  1 file changed, 21 insertions(+), 2 deletions(-) 
> > 
> > diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass 
> > index 79c9aae..cb1122c 100644 
> > --- a/classes/swupdate.bbclass 
> > +++ b/classes/swupdate.bbclass 
> > @@ -90,17 +90,31 @@ python do_swuimage () { 
> >      shutil.copyfile(os.path.join(workdir, "sw-description"), 
> os.path.join(s, "sw-description")) 
> >      fetch = bb.fetch2.Fetch([], d) 
> >      list_for_cpio = ["sw-description"] 
> > +    list_for_cpio_scripts = []  # scripts: pre-scripts must arive 
> before streamed images 
> > +    list_for_cpio_images = [] 
> >   
> >      if d.getVar('SWUPDATE_SIGNING', True): 
> >          list_for_cpio.append('sw-description.sig') 
> >   
> > +    def is_script(filename): 
> > +        try: 
> > +            with open(filename, 'r') as fh: 
> > +                if fh.read(2) == '#!': 
> > +                    return True 
> > +        except UnicodeDecodeError: 
> > +            pass 
> > +        return False 
> > + 
> >      # Add files listed in SRC_URI to the swu file 
> >      for url in fetch.urls: 
> >          local = fetch.localpath(url) 
> >          filename = os.path.basename(local) 
> >          if (filename != 'sw-description'): 
> >              shutil.copyfile(local, os.path.join(s, "%s" % filename )) 
> > -            list_for_cpio.append(filename) 
> > +            if is_script(local): 
> > +                list_for_cpio_scripts.append(filename) 
> > +            else: 
> > +                list_for_cpio_images.append(filename) 
> >   
> >      def add_image_to_swu(deploydir, imagename, s): 
> >          src = os.path.join(deploydir, imagename) 
> > @@ -109,7 +123,10 @@ python do_swuimage () { 
> >          target_imagename = os.path.basename(imagename)  # allow images 
> in subfolders of DEPLOY_DIR_IMAGE 
> >          dst = os.path.join(s, target_imagename) 
> >          shutil.copyfile(src, dst) 
> > -        list_for_cpio.append(target_imagename) 
> > +        if is_script(dst): 
> > +            list_for_cpio_scripts.append(target_imagename) 
> > +        else: 
> > +            list_for_cpio_images.append(target_imagename) 
> >          return True 
> >   
> >      # Search for images listed in SWUPDATE_IMAGES in the DEPLOY 
> directory. 
> > @@ -137,6 +154,8 @@ python do_swuimage () { 
> >              if not add_image_to_swu(deploydir, image, s): 
> >                  bb.fatal("swupdate cannot find %s image file" % image) 
> >   
> > +    list_for_cpio = list_for_cpio + list_for_cpio_scripts + 
> list_for_cpio_images 
> > + 
> >      prepare_sw_description(d, s, list_for_cpio) 
> >   
> >      line = 'for i in ' + ' '.join(list_for_cpio) + '; do echo $i;done | 
> cpio -ov -H crc >' + os.path.join(imgdeploydir,d.getVar('IMAGE_NAME', True) 
> + '.swu') 
> > 
>
> Best regards, 
> Stefano Babic 
>
> -- 
> ===================================================================== 
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk 
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany 
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de 
> <javascript:> 
> ===================================================================== 
>