| Message ID | 20191023211101.16591-2-adrian.freihofer@siemens.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | simplify swupdate config | expand |
Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > Try to make the swupdate.bbclass a little bit smarter in finding the > image files added to the swu archive. The implementation is also a bit > more Yocto like. By default, files are searched automatically whereas > MACHINE specific files get priority over files without the MACHINE in > the filename. > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more optional. > Most standard use cases are expected to "just work" without setting > this flag. NACK, because the filename need to be synchronized with the sw-description. A "more Yocto like" implementation would be a bbclass to create a sw-description from the recipe variables like the fitImage bbclass. Regards Stefan
Hi Stefan, Adrian, On 24/10/19 10:33, Stefan Herbrechtsmeier wrote: > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: >> Try to make the swupdate.bbclass a little bit smarter in finding the >> image files added to the swu archive. The implementation is also a bit >> more Yocto like. By default, files are searched automatically whereas >> MACHINE specific files get priority over files without the MACHINE in >> the filename. >> This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more optional. >> Most standard use cases are expected to "just work" without setting >> this flag. > > NACK, because the filename need to be synchronized with the > sw-description. Yes, this is true. > A "more Yocto like" implementation would be a bbclass to > create a sw-description from the recipe variables like the fitImage > bbclass. But this is maybe Yocto-like, but with some restrictions. Let me say that I am not very fond of the fitImage bbclass. The question is who is the master. If OE is the master, the description file is generated. This happens with the firImage class, but it fits just for the common cases. I have multiple cases where I add to a fitimage multiple configuration, a splash screen, a bitstream for a FPGA, ..... all these cases cannot be covered. So IMHO the master is the .its file for fitImage (I do not use the fitimage class and I build the fitImage myself starting from .its in an own recipe). The same is for sw-description, even worse: a Yocto-like implementation would generate a sw-description, but it is worse as for fitImage because there are a lot of different cases. IMHO the sw-description is the master, and the class (swupdate or whatever) just helps to fill in sw-description what is error prone, like filenames, hashes, and so on. Best regards, Stefano
Hi Stefan, Stefano Regarding the sw-description generator, I also think that this would require an unrealistic amount of code to cover all potential use cases. Let's focus on the current approach with a hand written sw-description. With the current implementation, I was just struggling with adding a file which is not a Yocto generated image. For images generated by Yocto, adding the MACHINE name to the image name is required to find the corresponding file in the deploy folder. For other artifacts appending the MACHINE name to the filename is wrong. With the current implementation adding such a file requires to add the VarFlag SWUPDATE_IMAGES_NOAPPEND_MACHINE. I spent quite some time to do the reverse engineering until I found this VarFlag. The only thing I would like to change is that Yocto automatically tries to find a file without the MACHINE appended, if the file with the MACHINE name appended does not exist. There are two reasons for keeping the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility and the very unlike case that the deploy folder contains both files. Then the SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly specify the right file. I think the new implementation would be simpler to use. I don't see any disadvantages such as a not synchronized filename. Regards, Adrian Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb Stefan Herbrechtsmeier: > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > Try to make the swupdate.bbclass a little bit smarter in finding the > > image files added to the swu archive. The implementation is also a bit > > more Yocto like. By default, files are searched automatically whereas > > MACHINE specific files get priority over files without the MACHINE in > > the filename. > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more optional. > > Most standard use cases are expected to "just work" without setting > > this flag. > > NACK, because the filename need to be synchronized with the > sw-description. A "more Yocto like" implementation would be a bbclass to > create a sw-description from the recipe variables like the fitImage > bbclass. > > Regards > Stefan >
Hi Adrian, I am just processing the discussion so far : On 24/10/19 22:37, adrian.freihofer@gmail.com wrote: > Hi Stefan, Stefano > > Regarding the sw-description generator, I also think that this would > require an unrealistic amount of code to cover all potential use cases. > Let's focus on the current approach with a hand written sw-description. > > With the current implementation, I was just struggling with adding a > file which is not a Yocto generated image. The assumption is that generated files for a SWU are found in DEPLOYDIR, while not generated files are fetched exactly as sw-description. All files in SRC_URI are added to the SWU image. > For images generated by > Yocto, adding the MACHINE name to the image name is required to find the > corresponding file in the deploy folder. For other artifacts appending > the MACHINE name to the filename is wrong. With the current > implementation adding such a file requires to add the VarFlag > SWUPDATE_IMAGES_NOAPPEND_MACHINE. Right. > I spent quite some time to do the > reverse engineering until I found this VarFlag. > > The only thing I would like to change is that Yocto automatically tries > to find a file without the MACHINE appended, if the file with the > MACHINE name appended does not exist. There are two reasons for keeping > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility and the > very unlike case that the deploy folder contains both files. Then the > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly specify the right > file. > > I think the new implementation would be simpler to use. I don't see any > disadvantages such as a not synchronized filename. > Best regards, Stefano Babic > Regards, > Adrian > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb Stefan > Herbrechtsmeier: > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > Try to make the swupdate.bbclass a little bit smarter in finding the > > image files added to the swu archive. The implementation is also a > bit > > more Yocto like. By default, files are searched automatically whereas > > MACHINE specific files get priority over files without the MACHINE in > > the filename. > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more > optional. > > Most standard use cases are expected to "just work" without setting > > this flag. > > NACK, because the filename need to be synchronized with the > sw-description. A "more Yocto like" implementation would be a > bbclass to > create a sw-description from the recipe variables like the fitImage > bbclass. > > Regards > Stefan > > -- > You received this message because you are subscribed to the Google > Groups "swupdate" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to swupdate+unsubscribe@googlegroups.com > <mailto:swupdate+unsubscribe@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>.
Hi Stefano Am Samstag, 26. Oktober 2019 15:20:13 UTC+2 schrieb Stefano Babic: > > Hi Adrian, > > I am just processing the discussion so far : > > On 24/10/19 22:37, adrian....@gmail.com <javascript:> wrote: > > Hi Stefan, Stefano > > > > Regarding the sw-description generator, I also think that this would > > require an unrealistic amount of code to cover all potential use cases. > > Let's focus on the current approach with a hand written sw-description. > > > > With the current implementation, I was just struggling with adding a > > file which is not a Yocto generated image. > > The assumption is that generated files for a SWU are found in DEPLOYDIR, > while not generated files are fetched exactly as sw-description. All > files in SRC_URI are added to the SWU image. > > > For images generated by > > Yocto, adding the MACHINE name to the image name is required to find the > > corresponding file in the deploy folder. For other artifacts appending > > the MACHINE name to the filename is wrong. With the current > > implementation adding such a file requires to add the VarFlag > > SWUPDATE_IMAGES_NOAPPEND_MACHINE. > > Right. > The problem I was facing: We use a Variable for the deployed image file name. Adding this variable to the sw-descripton did not work, because a file such as image-${MACHINE}${MACHINE} was searched for. The new implementation tries to find the file by resolving the magic done by Yocto (as before) or (that's new) without magic, if the first trial fails. Regards, Adrian > > I spent quite some time to do the > > reverse engineering until I found this VarFlag. > > > > The only thing I would like to change is that Yocto automatically tries > > to find a file without the MACHINE appended, if the file with the > > MACHINE name appended does not exist. There are two reasons for keeping > > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility and the > > very unlike case that the deploy folder contains both files. Then the > > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly specify the right > > file. > > > > I think the new implementation would be simpler to use. I don't see any > > disadvantages such as a not synchronized filename. > > > > Best regards, > Stefano Babic > > > Regards, > > Adrian > > > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb Stefan > > Herbrechtsmeier: > > > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > > Try to make the swupdate.bbclass a little bit smarter in finding > the > > > image files added to the swu archive. The implementation is also a > > bit > > > more Yocto like. By default, files are searched automatically > whereas > > > MACHINE specific files get priority over files without the MACHINE > in > > > the filename. > > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more > > optional. > > > Most standard use cases are expected to "just work" without > setting > > > this flag. > > > > NACK, because the filename need to be synchronized with the > > sw-description. A "more Yocto like" implementation would be a > > bbclass to > > create a sw-description from the recipe variables like the fitImage > > bbclass. > > > > Regards > > Stefan > > > > -- > > You received this message because you are subscribed to the Google > > Groups "swupdate" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to swup...@googlegroups.com <javascript:> > > <mailto:swup...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > -- > ===================================================================== > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de > <javascript:> > ===================================================================== >
Hi Adrian, On 27/10/19 12:58, adrian.freihofer@gmail.com wrote: > Hi Stefano > > Am Samstag, 26. Oktober 2019 15:20:13 UTC+2 schrieb Stefano Babic: > > Hi Adrian, > > I am just processing the discussion so far : > > On 24/10/19 22:37, adrian....@gmail.com <javascript:> wrote: > > Hi Stefan, Stefano > > > > Regarding the sw-description generator, I also think that this would > > require an unrealistic amount of code to cover all potential use > cases. > > Let's focus on the current approach with a hand written > sw-description. > > > > With the current implementation, I was just struggling with adding a > > file which is not a Yocto generated image. > > The assumption is that generated files for a SWU are found in > DEPLOYDIR, > while not generated files are fetched exactly as sw-description. All > files in SRC_URI are added to the SWU image. > > > For images generated by > > Yocto, adding the MACHINE name to the image name is required to > find the > > corresponding file in the deploy folder. For other artifacts > appending > > the MACHINE name to the filename is wrong. With the current > > implementation adding such a file requires to add the VarFlag > > SWUPDATE_IMAGES_NOAPPEND_MACHINE. > > Right. > > The problem I was facing: We use a Variable for the deployed image file > name. Ok, I do the same, too. > Adding this variable to the sw-descripton did not work, because a > file such as image-${MACHINE}${MACHINE} was searched for. From one of my projects, I set : images: ( { filename = "@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; type = "raw"; device = "/dev/mmcblk1p3"; sha256 = "@@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; }, and variabel substitution works. Yes, there is the weirdness with a single "@" for sha256 and "@@" for variable substitution. But it works. Can you tell me which is the exact use case that does not work ? > The new > implementation tries to find the file by resolving the magic done by > Yocto (as before) or (that's new) without magic, if the first trial fails. I am unsure if the patch just fix the issue you report, or more as one in one step. I must look at it deeper. Regards, Stefano > > Regards, > Adrian > > > > I spent quite some time to do the > > reverse engineering until I found this VarFlag. > > > > The only thing I would like to change is that Yocto automatically > tries > > to find a file without the MACHINE appended, if the file with the > > MACHINE name appended does not exist. There are two reasons for > keeping > > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility and the > > very unlike case that the deploy folder contains both files. Then the > > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly specify the > right > > file. > > > > I think the new implementation would be simpler to use. I don't > see any > > disadvantages such as a not synchronized filename. > > > > Best regards, > Stefano Babic > > > Regards, > > Adrian > > > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb Stefan > > Herbrechtsmeier: > > > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > > Try to make the swupdate.bbclass a little bit smarter in > finding the > > > image files added to the swu archive. The implementation is > also a > > bit > > > more Yocto like. By default, files are searched > automatically whereas > > > MACHINE specific files get priority over files without the > MACHINE in > > > the filename. > > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more > > optional. > > > Most standard use cases are expected to "just work" without > setting > > > this flag. > > > > NACK, because the filename need to be synchronized with the > > sw-description. A "more Yocto like" implementation would be a > > bbclass to > > create a sw-description from the recipe variables like the > fitImage > > bbclass. > > > > Regards > > Stefan > > > > -- > > You received this message because you are subscribed to the Google > > Groups "swupdate" group. > > To unsubscribe from this group and stop receiving emails from it, > send > > an email to swup...@googlegroups.com <javascript:> > > <mailto:swup...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>>. > > > -- > ===================================================================== > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > sba...@denx.de <javascript:> > ===================================================================== > > -- > You received this message because you are subscribed to the Google > Groups "swupdate" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to swupdate+unsubscribe@googlegroups.com > <mailto:swupdate+unsubscribe@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer>.
Hi Stefano Thank you for the explanation. You are right, the @@@ works, also without my patch (not this patch here). I removed this patch from my setup and went through the issues I'm having without it. - We need to add a boot container for an i.MX8. (I think we use code from NXP.) The boot container is deployed to: build/tmp/deploy/images/$MACHINE/imx-boot-tools/imx-boot-$MAHINE-mfg.bin This path has two incompatibilities: It's a sub-folder and the binary is not named as expected. Setting SWUPDATE_IMAGES = "imx-boot-tools/imx-boot-cxg3-sd.bin" fails. The file is searched in tmp/work of the recipe instead of the deploy folder because the image type is not set. But setting an image type does not really make sense to me in this case. - The images we have, do not follow the default image naming convention of Yocto. It's a legacy on our side. With my patch the file described above are found, with or without setting SWUPDATE_IMAGES_NOAPPEND_MACHINE. Without I did not find a solution. I would like to argue that Yocto tries to simplify its usage by searching files in more than one path. I tried to offer something similar here. Am Montag, 28. Oktober 2019 13:32:48 UTC+1 schrieb Stefano Babic: > > Hi Adrian, > > On 27/10/19 12:58, adrian....@gmail.com <javascript:> wrote: > > Hi Stefano > > > > Am Samstag, 26. Oktober 2019 15:20:13 UTC+2 schrieb Stefano Babic: > > > > Hi Adrian, > > > > I am just processing the discussion so far : > > > > On 24/10/19 22:37, adrian....@gmail.com <javascript:> wrote: > > > Hi Stefan, Stefano > > > > > > Regarding the sw-description generator, I also think that this > would > > > require an unrealistic amount of code to cover all potential use > > cases. > > > Let's focus on the current approach with a hand written > > sw-description. > > > > > > With the current implementation, I was just struggling with adding > a > > > file which is not a Yocto generated image. > > > > The assumption is that generated files for a SWU are found in > > DEPLOYDIR, > > while not generated files are fetched exactly as sw-description. All > > files in SRC_URI are added to the SWU image. > > > > > For images generated by > > > Yocto, adding the MACHINE name to the image name is required to > > find the > > > corresponding file in the deploy folder. For other artifacts > > appending > > > the MACHINE name to the filename is wrong. With the current > > > implementation adding such a file requires to add the VarFlag > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE. > > > > Right. > > > > The problem I was facing: We use a Variable for the deployed image file > > name. > > Ok, I do the same, too. > > > Adding this variable to the sw-descripton did not work, because a > > file such as image-${MACHINE}${MACHINE} was searched for. > > > From one of my projects, I set : > > images: ( > { > filename = > "@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > type = "raw"; > device = "/dev/mmcblk1p3"; > sha256 = > "@@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > }, > and variabel substitution works. Yes, there is the weirdness with a > single "@" for sha256 and "@@" for variable substitution. But it works. > Can you tell me which is the exact use case that does not work ? > > Until now, I did not try to write many variables into the sw-description. My approach was to have something similar like you showed in the example, in the recipe but not in the sw-description. > > The new > > implementation tries to find the file by resolving the magic done by > > Yocto (as before) or (that's new) without magic, if the first trial > fails. > > I am unsure if the patch just fix the issue you report, or more as one > in one step. I must look at it deeper. > The goal was to find files with and files without $MACHINE in the file name (without a need for setting the SWUPDATE_IMAGES_NOAPPEND_MACHINE). Files in a sub-folder are supported as well. Changing something else is at least not my intention. Regards, Adrian > > Regards, > Stefano > > > > > Regards, > > Adrian > > > > > > > I spent quite some time to do the > > > reverse engineering until I found this VarFlag. > > > > > > The only thing I would like to change is that Yocto automatically > > tries > > > to find a file without the MACHINE appended, if the file with the > > > MACHINE name appended does not exist. There are two reasons for > > keeping > > > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility and > the > > > very unlike case that the deploy folder contains both files. Then > the > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly specify the > > right > > > file. > > > > > > I think the new implementation would be simpler to use. I don't > > see any > > > disadvantages such as a not synchronized filename. > > > > > > > Best regards, > > Stefano Babic > > > > > Regards, > > > Adrian > > > > > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb Stefan > > > Herbrechtsmeier: > > > > > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > > > Try to make the swupdate.bbclass a little bit smarter in > > finding the > > > > image files added to the swu archive. The implementation is > > also a > > > bit > > > > more Yocto like. By default, files are searched > > automatically whereas > > > > MACHINE specific files get priority over files without the > > MACHINE in > > > > the filename. > > > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more > > > optional. > > > > Most standard use cases are expected to "just work" without > > setting > > > > this flag. > > > > > > NACK, because the filename need to be synchronized with the > > > sw-description. A "more Yocto like" implementation would be a > > > bbclass to > > > create a sw-description from the recipe variables like the > > fitImage > > > bbclass. > > > > > > Regards > > > Stefan > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "swupdate" group. > > > To unsubscribe from this group and stop receiving emails from it, > > send > > > an email to swup...@googlegroups.com <javascript:> > > > <mailto:swup...@googlegroups.com <javascript:>>. > > > To view this discussion on the web visit > > > > > > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com> > > > > > > > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>>. > > > > > > > -- > > > ===================================================================== > > DENX Software Engineering GmbH, Managing Director: Wolfgang > Denk > > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > Germany > > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > > sba...@denx.de <javascript:> > > > ===================================================================== > > > > -- > > You received this message because you are subscribed to the Google > > Groups "swupdate" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to swup...@googlegroups.com <javascript:> > > <mailto:swup...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > > -- > ===================================================================== > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de > <javascript:> > ===================================================================== >
Hi Adrian, On 28/10/19 23:47, adrian.freihofer@gmail.com wrote: > Hi Stefano > > Thank you for the explanation. You are right, the @@@ works, also > without my patch (not this patch here). Ok > > I removed this patch from my setup and went through the issues I'm > having without it. > - We need to add a boot container for an i.MX8. (I think we use code > from NXP.) Yes, it is - there is current no other way to build MX8. > The boot container is deployed to: > build/tmp/deploy/images/$MACHINE/imx-boot-tools/imx-boot-$MAHINE-mfg.bin Ok, I understand the problem. > This path has two incompatibilities: It's a sub-folder and the binary > is not named as expected. One way to overcome this is to add in the recipe originating imx-boot-$MAHINE-mfg.bin a link in DEPLOY_DIR, so that $DEPLOY_DIR/imx-boot-$MACHINE.bin ==> $DEPLOY_DIR/imx-boot-tools/imx-boot-$MAHINE-mfg.bin. Because images are already copied by the class in workdir, cpio will work. However, a more generic way is to extrend the search path in the swupdate class. It just searches in $DEPLOY_DIR, but we can extend in a way that images are searched in a list of directories, like in the class: SWUPDATE_IMAGE_PATH="${DEPLOY_DIR}" and in the own recipe: SWUPDATE_IMAGE_PATH+="${DEPLOY_DIR}/imx-boot-tools" > Setting SWUPDATE_IMAGES = "imx-boot-tools/imx-boot-cxg3-sd.bin" fails. Yes, the case is not covered. > The file is searched in tmp/work of the recipe instead of the deploy > folder because the image type is not set. mmmh...file is always searched in DEPLOYDIR even if the image type is not set, or it was fetched and it is one of SRC_URI. > But setting an image type does > not really make sense to me in this case. No, it is thought for another reason. For example, uImage has no $MACHINE extension, or uImage.bin. In your case, the issue is due to the search path by the class that is fixed to $DEPLOY_DIR > - The images we have, do not follow the default image naming convention > of Yocto. It's a legacy on our side. If the image (even with any weird naming convention) was in DEPLOY_DIR, it works. > > With my patch the file described above are found, with or without > setting SWUPDATE_IMAGES_NOAPPEND_MACHINE. Without I did not find a solution. > I would like to argue that Yocto tries to simplify its usage by > searching files in more than one path. Well, I think it was not Yocto, but it is like I implemented this part. Yocto gives me out-of-the-box the fetching of additional images from SRC_URI (sw-description, scripts, etc.). Other images are searched just in $DEPLOY_DIR. What about to add to your recipe something like: FILESEXTRAPATHS_prepend := "${@d.getVar('DEPLOY_DIR_IMAGE')}/imx-boot-tools:" And then: SRC_URI += "file://imx-boot-mg.bin" > I tried to offer something > similar here. > Best regards, Stefano > > Am Montag, 28. Oktober 2019 13:32:48 UTC+1 schrieb Stefano Babic: > > Hi Adrian, > > On 27/10/19 12:58, adrian....@gmail.com <javascript:> wrote: > > Hi Stefano > > > > Am Samstag, 26. Oktober 2019 15:20:13 UTC+2 schrieb Stefano Babic: > > > > Hi Adrian, > > > > I am just processing the discussion so far : > > > > On 24/10/19 22:37, adrian....@gmail.com <javascript:> wrote: > > > Hi Stefan, Stefano > > > > > > Regarding the sw-description generator, I also think that > this would > > > require an unrealistic amount of code to cover all potential > use > > cases. > > > Let's focus on the current approach with a hand written > > sw-description. > > > > > > With the current implementation, I was just struggling with > adding a > > > file which is not a Yocto generated image. > > > > The assumption is that generated files for a SWU are found in > > DEPLOYDIR, > > while not generated files are fetched exactly as > sw-description. All > > files in SRC_URI are added to the SWU image. > > > > > For images generated by > > > Yocto, adding the MACHINE name to the image name is required to > > find the > > > corresponding file in the deploy folder. For other artifacts > > appending > > > the MACHINE name to the filename is wrong. With the current > > > implementation adding such a file requires to add the VarFlag > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE. > > > > Right. > > > > The problem I was facing: We use a Variable for the deployed image > file > > name. > > Ok, I do the same, too. > > > Adding this variable to the sw-descripton did not work, because a > > file such as image-${MACHINE}${MACHINE} was searched for. > > > From one of my projects, I set : > > images: ( > { > filename = > "@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > type = "raw"; > device = "/dev/mmcblk1p3"; > sha256 = > "@@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > }, > and variabel substitution works. Yes, there is the weirdness with a > single "@" for sha256 and "@@" for variable substitution. But it works. > Can you tell me which is the exact use case that does not work ? > > Until now, I did not try to write many variables into the > sw-description. My approach was to have something similar like you > showed in the example, in the recipe but not in the sw-description. > > > > The new > > implementation tries to find the file by resolving the magic done by > > Yocto (as before) or (that's new) without magic, if the first > trial fails. > > I am unsure if the patch just fix the issue you report, or more as one > in one step. I must look at it deeper. > > The goal was to find files with and files without $MACHINE in the file > name (without a need for setting the SWUPDATE_IMAGES_NOAPPEND_MACHINE). > Files in a sub-folder are supported as well. > Changing something else is at least not my intention. > > Regards, > Adrian > > > Regards, > Stefano > > > > > Regards, > > Adrian > > > > > > > I spent quite some time to do the > > > reverse engineering until I found this VarFlag. > > > > > > The only thing I would like to change is that Yocto > automatically > > tries > > > to find a file without the MACHINE appended, if the file > with the > > > MACHINE name appended does not exist. There are two reasons for > > keeping > > > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility > and the > > > very unlike case that the deploy folder contains both files. > Then the > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly > specify the > > right > > > file. > > > > > > I think the new implementation would be simpler to use. I don't > > see any > > > disadvantages such as a not synchronized filename. > > > > > > > Best regards, > > Stefano Babic > > > > > Regards, > > > Adrian > > > > > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb Stefan > > > Herbrechtsmeier: > > > > > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > > > Try to make the swupdate.bbclass a little bit smarter in > > finding the > > > > image files added to the swu archive. The > implementation is > > also a > > > bit > > > > more Yocto like. By default, files are searched > > automatically whereas > > > > MACHINE specific files get priority over files without > the > > MACHINE in > > > > the filename. > > > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE > VarFlag more > > > optional. > > > > Most standard use cases are expected to "just work" > without > > setting > > > > this flag. > > > > > > NACK, because the filename need to be synchronized with the > > > sw-description. A "more Yocto like" implementation would > be a > > > bbclass to > > > create a sw-description from the recipe variables like the > > fitImage > > > bbclass. > > > > > > Regards > > > Stefan > > > > > > -- > > > You received this message because you are subscribed to the > Google > > > Groups "swupdate" group. > > > To unsubscribe from this group and stop receiving emails > from it, > > send > > > an email to swup...@googlegroups.com <javascript:> > > > <mailto:swup...@googlegroups.com <javascript:>>. > > > To view this discussion on the web visit > > > > > > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com>> > > > > > > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>>>. > > > > > > > -- > > > ===================================================================== > > DENX Software Engineering GmbH, Managing Director: > Wolfgang Denk > > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > Germany > > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > > sba...@denx.de <javascript:> > > > ===================================================================== > > > > -- > > You received this message because you are subscribed to the Google > > Groups "swupdate" group. > > To unsubscribe from this group and stop receiving emails from it, > send > > an email to swup...@googlegroups.com <javascript:> > > <mailto:swup...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer>>. > > > > -- > ===================================================================== > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > sba...@denx.de <javascript:> > ===================================================================== > > -- > You received this message because you are subscribed to the Google > Groups "swupdate" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to swupdate+unsubscribe@googlegroups.com > <mailto:swupdate+unsubscribe@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com?utm_medium=email&utm_source=footer>.
Hi Stefano Am Dienstag, 29. Oktober 2019 09:42:43 UTC+1 schrieb Stefano Babic: > > Hi Adrian, > > On 28/10/19 23:47, adrian....@gmail.com <javascript:> wrote: > > Hi Stefano > > > > Thank you for the explanation. You are right, the @@@ works, also > > without my patch (not this patch here). > > Ok > > > > > I removed this patch from my setup and went through the issues I'm > > having without it. > > - We need to add a boot container for an i.MX8. (I think we use code > > from NXP.) > > Yes, it is - there is current no other way to build MX8. > > > The boot container is deployed to: > > > build/tmp/deploy/images/$MACHINE/imx-boot-tools/imx-boot-$MAHINE-mfg.bin > > Ok, I understand the problem. > > > This path has two incompatibilities: It's a sub-folder and the binary > > is not named as expected. > > One way to overcome this is to add in the recipe originating > imx-boot-$MAHINE-mfg.bin a link in DEPLOY_DIR, so that > $DEPLOY_DIR/imx-boot-$MACHINE.bin ==> > $DEPLOY_DIR/imx-boot-tools/imx-boot-$MAHINE-mfg.bin. Because images are > already copied by the class in workdir, cpio will work. > > However, a more generic way is to extrend the search path in the > swupdate class. It just searches in $DEPLOY_DIR, but we can extend in a > way that images are searched in a list of directories, like > > in the class: > > SWUPDATE_IMAGE_PATH="${DEPLOY_DIR}" > > and in the own recipe: > > SWUPDATE_IMAGE_PATH+="${DEPLOY_DIR}/imx-boot-tools" > > > > Setting SWUPDATE_IMAGES = "imx-boot-tools/imx-boot-cxg3-sd.bin" fails. > > Yes, the case is not covered. > > > The file is searched in tmp/work of the recipe instead of the deploy > > folder because the image type is not set. > > mmmh...file is always searched in DEPLOYDIR even if the image type is > not set, or it was fetched and it is one of SRC_URI. > > > But setting an image type does > > not really make sense to me in this case. > > No, it is thought for another reason. For example, uImage has no > $MACHINE extension, or uImage.bin. In your case, the issue is due to the > search path by the class that is fixed to $DEPLOY_DIR > > > - The images we have, do not follow the default image naming convention > > of Yocto. It's a legacy on our side. > > If the image (even with any weird naming convention) was in DEPLOY_DIR, > it works. > > > > > With my patch the file described above are found, with or without > > setting SWUPDATE_IMAGES_NOAPPEND_MACHINE. Without I did not find a > solution. > > I would like to argue that Yocto tries to simplify its usage by > > searching files in more than one path. > > Well, I think it was not Yocto, but it is like I implemented this part. > Yocto gives me out-of-the-box the fetching of additional images from > SRC_URI (sw-description, scripts, etc.). Other images are searched just > in $DEPLOY_DIR. > > What about to add to your recipe something like: > > FILESEXTRAPATHS_prepend := > "${@d.getVar('DEPLOY_DIR_IMAGE')}/imx-boot-tools:" > > And then: > > SRC_URI += "file://imx-boot-mg.bin" > Thank you for the hint. I changed my implementation accordingly. It works. However, the implementation is much more complicated. One artifact is added as image, one as a file, even if both of them are just copied from the deploy folder. It also requires to add a task dependency. The deploy task of the files in the deploy folder must be executed before the fetch task of the swu image. Ideally, the swupdate.bbclass would support an easy way to add MACHINE specific artifacts to the cpio archive and to add MACHINE specific lines to the sw-description file. Based on my patch, it's pretty straight forward to implement something similar like the MACHINE_EXTRA_RDEPENDS variable provides for images. To add machine specific lines to the sw-description, the simplest possible implementation might be a build-time include directive. (it's simpler than a complete sw-description generator) > I tried to offer something > > similar here. > > > > In order to continue on this path, I would like to see one more of my patches merged. Are there any concerns about the next patch "swupdate: improve systemd config"? Thank you for the review and the hints. Regards, Adrian Best regards, > Stefano > > > > > Am Montag, 28. Oktober 2019 13:32:48 UTC+1 schrieb Stefano Babic: > > > > Hi Adrian, > > > > On 27/10/19 12:58, adrian....@gmail.com <javascript:> wrote: > > > Hi Stefano > > > > > > Am Samstag, 26. Oktober 2019 15:20:13 UTC+2 schrieb Stefano Babic: > > > > > > Hi Adrian, > > > > > > I am just processing the discussion so far : > > > > > > On 24/10/19 22:37, adrian....@gmail.com <javascript:> wrote: > > > > Hi Stefan, Stefano > > > > > > > > Regarding the sw-description generator, I also think that > > this would > > > > require an unrealistic amount of code to cover all potential > > use > > > cases. > > > > Let's focus on the current approach with a hand written > > > sw-description. > > > > > > > > With the current implementation, I was just struggling with > > adding a > > > > file which is not a Yocto generated image. > > > > > > The assumption is that generated files for a SWU are found in > > > DEPLOYDIR, > > > while not generated files are fetched exactly as > > sw-description. All > > > files in SRC_URI are added to the SWU image. > > > > > > > For images generated by > > > > Yocto, adding the MACHINE name to the image name is required > to > > > find the > > > > corresponding file in the deploy folder. For other artifacts > > > appending > > > > the MACHINE name to the filename is wrong. With the current > > > > implementation adding such a file requires to add the > VarFlag > > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE. > > > > > > Right. > > > > > > The problem I was facing: We use a Variable for the deployed image > > file > > > name. > > > > Ok, I do the same, too. > > > > > Adding this variable to the sw-descripton did not work, because a > > > file such as image-${MACHINE}${MACHINE} was searched for. > > > > > > From one of my projects, I set : > > > > images: ( > > { > > filename = > > > "@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > > > > type = "raw"; > > device = "/dev/mmcblk1p3"; > > sha256 = > > > "@@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > > > > }, > > and variabel substitution works. Yes, there is the weirdness with a > > single "@" for sha256 and "@@" for variable substitution. But it > works. > > Can you tell me which is the exact use case that does not work ? > > > > Until now, I did not try to write many variables into the > > sw-description. My approach was to have something similar like you > > showed in the example, in the recipe but not in the sw-description. > > > > > > > The new > > > implementation tries to find the file by resolving the magic done > by > > > Yocto (as before) or (that's new) without magic, if the first > > trial fails. > > > > I am unsure if the patch just fix the issue you report, or more as > one > > in one step. I must look at it deeper. > > > > The goal was to find files with and files without $MACHINE in the file > > name (without a need for setting the SWUPDATE_IMAGES_NOAPPEND_MACHINE). > > Files in a sub-folder are supported as well. > > Changing something else is at least not my intention. > > > > Regards, > > Adrian > > > > > > Regards, > > Stefano > > > > > > > > Regards, > > > Adrian > > > > > > > > > > I spent quite some time to do the > > > > reverse engineering until I found this VarFlag. > > > > > > > > The only thing I would like to change is that Yocto > > automatically > > > tries > > > > to find a file without the MACHINE appended, if the file > > with the > > > > MACHINE name appended does not exist. There are two reasons > for > > > keeping > > > > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward compatibility > > and the > > > > very unlike case that the deploy folder contains both files. > > Then the > > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly > > specify the > > > right > > > > file. > > > > > > > > I think the new implementation would be simpler to use. I > don't > > > see any > > > > disadvantages such as a not synchronized filename. > > > > > > > > > > Best regards, > > > Stefano Babic > > > > > > > Regards, > > > > Adrian > > > > > > > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb > Stefan > > > > Herbrechtsmeier: > > > > > > > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > > > > Try to make the swupdate.bbclass a little bit smarter > in > > > finding the > > > > > image files added to the swu archive. The > > implementation is > > > also a > > > > bit > > > > > more Yocto like. By default, files are searched > > > automatically whereas > > > > > MACHINE specific files get priority over files without > > the > > > MACHINE in > > > > > the filename. > > > > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE > > VarFlag more > > > > optional. > > > > > Most standard use cases are expected to "just work" > > without > > > setting > > > > > this flag. > > > > > > > > NACK, because the filename need to be synchronized with > the > > > > sw-description. A "more Yocto like" implementation would > > be a > > > > bbclass to > > > > create a sw-description from the recipe variables like > the > > > fitImage > > > > bbclass. > > > > > > > > Regards > > > > Stefan > > > > > > > > -- > > > > You received this message because you are subscribed to the > > Google > > > > Groups "swupdate" group. > > > > To unsubscribe from this group and stop receiving emails > > from it, > > > send > > > > an email to swup...@googlegroups.com <javascript:> > > > > <mailto:swup...@googlegroups.com <javascript:>>. > > > > To view this discussion on the web visit > > > > > > > > > > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com> > > > > > > > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com>> > > > > > > > > > > > > > > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer> > > > > > > > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > > < > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>>>. > > > > > > > > > > > > -- > > > > > > ===================================================================== > > > DENX Software Engineering GmbH, Managing Director: > > Wolfgang Denk > > > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > > Germany > > > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > > > sba...@denx.de <javascript:> > > > > > > ===================================================================== > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "swupdate" group. > > > To unsubscribe from this group and stop receiving emails from it, > > send > > > an email to swup...@googlegroups.com <javascript:> > > > <mailto:swup...@googlegroups.com <javascript:>>. > > > To view this discussion on the web visit > > > > > > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com> > > > > > > > > < > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer > > < > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer>>. > > > > > > > > > -- > > > ===================================================================== > > DENX Software Engineering GmbH, Managing Director: Wolfgang > Denk > > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > Germany > > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > > sba...@denx.de <javascript:> > > > ===================================================================== > > > > -- > > You received this message because you are subscribed to the Google > > Groups "swupdate" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to swup...@googlegroups.com <javascript:> > > <mailto:swup...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com > > < > https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > -- > ===================================================================== > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de > <javascript:> > ===================================================================== >
Hi Adrian, On 02/11/19 10:35, adrian.freihofer@gmail.com wrote: > Hi Stefano > > Am Dienstag, 29. Oktober 2019 09:42:43 UTC+1 schrieb Stefano Babic: > > Hi Adrian, > > On 28/10/19 23:47, adrian....@gmail.com <javascript:> wrote: > > Hi Stefano > > > > Thank you for the explanation. You are right, the @@@ works, also > > without my patch (not this patch here). > > Ok > > > > > I removed this patch from my setup and went through the issues I'm > > having without it. > > - We need to add a boot container for an i.MX8. (I think we use code > > from NXP.) > > Yes, it is - there is current no other way to build MX8. > > > The boot container is deployed to: > > > build/tmp/deploy/images/$MACHINE/imx-boot-tools/imx-boot-$MAHINE-mfg.bin > > > Ok, I understand the problem. > > > This path has two incompatibilities: It's a sub-folder and the > binary > > is not named as expected. > > One way to overcome this is to add in the recipe originating > imx-boot-$MAHINE-mfg.bin a link in DEPLOY_DIR, so that > $DEPLOY_DIR/imx-boot-$MACHINE.bin ==> > $DEPLOY_DIR/imx-boot-tools/imx-boot-$MAHINE-mfg.bin. Because images are > already copied by the class in workdir, cpio will work. > > However, a more generic way is to extrend the search path in the > swupdate class. It just searches in $DEPLOY_DIR, but we can extend in a > way that images are searched in a list of directories, like > > in the class: > > SWUPDATE_IMAGE_PATH="${DEPLOY_DIR}" > > and in the own recipe: > > SWUPDATE_IMAGE_PATH+="${DEPLOY_DIR}/imx-boot-tools" > > > > Setting SWUPDATE_IMAGES = "imx-boot-tools/imx-boot-cxg3-sd.bin" > fails. > > Yes, the case is not covered. > > > The file is searched in tmp/work of the recipe instead of the deploy > > folder because the image type is not set. > > mmmh...file is always searched in DEPLOYDIR even if the image type is > not set, or it was fetched and it is one of SRC_URI. > > > But setting an image type does > > not really make sense to me in this case. > > No, it is thought for another reason. For example, uImage has no > $MACHINE extension, or uImage.bin. In your case, the issue is due to > the > search path by the class that is fixed to $DEPLOY_DIR > > > - The images we have, do not follow the default image naming > convention > > of Yocto. It's a legacy on our side. > > If the image (even with any weird naming convention) was in DEPLOY_DIR, > it works. > > > > > With my patch the file described above are found, with or without > > setting SWUPDATE_IMAGES_NOAPPEND_MACHINE. Without I did not find a > solution. > > I would like to argue that Yocto tries to simplify its usage by > > searching files in more than one path. > > Well, I think it was not Yocto, but it is like I implemented this part. > Yocto gives me out-of-the-box the fetching of additional images from > SRC_URI (sw-description, scripts, etc.). Other images are searched just > in $DEPLOY_DIR. > > What about to add to your recipe something like: > > FILESEXTRAPATHS_prepend := > "${@d.getVar('DEPLOY_DIR_IMAGE')}/imx-boot-tools:" > > And then: > > SRC_URI += "file://imx-boot-mg.bin" > > Thank you for the hint. I changed my implementation accordingly. It works. ;-) > > However, the implementation is much more complicated. One artifact is > added as image, one as a file, even if both of them are just copied from > the deploy folder. Right - my first concern is not that the its is simple, but that it is very flexible and can cover even complicate cases that can happens. In fact, a SWU in most cases contains artifacts for the single MACHINE. But this is not always true. It is quite common (at least, for me..) that a SWU contains artifacts not generated for the MACHINE used to build the SWU. Some of this cases: - there is a microcontroller firmware, and it makes no sense to add it to rootfs if it is just used by SWUpdate. The FW must be part of SWU, it is generated outside OE. - Images that belong to other MACHINEs. One common case is a rescue image, where a new MCHINE is introduce to tune footprint (kernel, etc.). The recipe must take this file, it is in DEPLOY_DIR, but not in deploy/images/${MACHINE} - Files that customers decide to add the SWU because they have own handlers (in Lua) - they must be fetched. For some of these cases (external files, files from different MACHINEs) my solution was to add the fetch() task to the SWU build. So I got the general fetch mechanism from OE. For files generated by current build, I decided to check in deploy folder only. My argument was that, if something must be installed on the target, it is something to be "deployed". If a recipe generates something that should be picked up by building the SWU, this recipe must add a do_deploy() task. > It also requires to add a task dependency. The deploy task of the files > in the deploy folder must be executed before the fetch task of the swu > image. It is not very clear to me which case you are telling about, but sure, I can imagine such as cases. Surely, the recipe building the SWU depends on all images in the deploy directories, and these dependencies must be added in the own "SWU" recipe (the recipe including the swupdate class). However, I am missing the dependencies with the fetch() task, they seem orthogonal to me. Just when do_suimage runs, all artifacts, without regard if they are picked from DEPLOYDIR or if they are fetched, must be available. > > Ideally, the swupdate.bbclass would support an easy way to add MACHINE > specific artifacts to the cpio archive and to add MACHINE specific lines > to the sw-description file. > Based on my patch, it's pretty straight forward to implement something > similar like the MACHINE_EXTRA_RDEPENDS variable provides for images. I am reviewing the patches - IMHO there are parts that can be applied (and I tend to apply very soon) as they are, and some of them that need some iteration. > To add machine specific lines to the sw-description, the simplest > possible implementation might be a build-time include directive. (it's > simpler than a complete sw-description generator) Well, an include is something I have thought, too. But please consider the use cases above with artifacts not belonging to the current MACHINE. Note: of course, I have never solved the dependency when I need an artifact from another MACHINE. OE solves dependencies for the single build, not across builds. In that case, I have to force a list of bitbake commands. > > > I tried to offer something > > similar here. > > > > In order to continue on this path, I would like to see one more of my > patches merged. Are there any concerns about the next patch "swupdate: > improve systemd config"? I am reviewing them, in short I answer. > > Thank you for the review and the hints. Best regards, Stefano > Regards, > Adrian > > Best regards, > Stefano > > > > > Am Montag, 28. Oktober 2019 13:32:48 UTC+1 schrieb Stefano Babic: > > > > Hi Adrian, > > > > On 27/10/19 12:58, adrian....@gmail.com <javascript:> wrote: > > > Hi Stefano > > > > > > Am Samstag, 26. Oktober 2019 15:20:13 UTC+2 schrieb Stefano > Babic: > > > > > > Hi Adrian, > > > > > > I am just processing the discussion so far : > > > > > > On 24/10/19 22:37, adrian....@gmail.com <javascript:> > wrote: > > > > Hi Stefan, Stefano > > > > > > > > Regarding the sw-description generator, I also think that > > this would > > > > require an unrealistic amount of code to cover all > potential > > use > > > cases. > > > > Let's focus on the current approach with a hand written > > > sw-description. > > > > > > > > With the current implementation, I was just struggling > with > > adding a > > > > file which is not a Yocto generated image. > > > > > > The assumption is that generated files for a SWU are > found in > > > DEPLOYDIR, > > > while not generated files are fetched exactly as > > sw-description. All > > > files in SRC_URI are added to the SWU image. > > > > > > > For images generated by > > > > Yocto, adding the MACHINE name to the image name is > required to > > > find the > > > > corresponding file in the deploy folder. For other > artifacts > > > appending > > > > the MACHINE name to the filename is wrong. With the > current > > > > implementation adding such a file requires to add the > VarFlag > > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE. > > > > > > Right. > > > > > > The problem I was facing: We use a Variable for the deployed > image > > file > > > name. > > > > Ok, I do the same, too. > > > > > Adding this variable to the sw-descripton did not work, > because a > > > file such as image-${MACHINE}${MACHINE} was searched for. > > > > > > From one of my projects, I set : > > > > images: ( > > { > > filename = > > > "@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > > > > type = "raw"; > > device = "/dev/mmcblk1p3"; > > sha256 = > > > "@@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@"; > > > > > }, > > and variabel substitution works. Yes, there is the weirdness > with a > > single "@" for sha256 and "@@" for variable substitution. But > it works. > > Can you tell me which is the exact use case that does not work ? > > > > Until now, I did not try to write many variables into the > > sw-description. My approach was to have something similar like you > > showed in the example, in the recipe but not in the sw-description. > > > > > > > The new > > > implementation tries to find the file by resolving the magic > done by > > > Yocto (as before) or (that's new) without magic, if the first > > trial fails. > > > > I am unsure if the patch just fix the issue you report, or > more as one > > in one step. I must look at it deeper. > > > > The goal was to find files with and files without $MACHINE in the > file > > name (without a need for setting the > SWUPDATE_IMAGES_NOAPPEND_MACHINE). > > Files in a sub-folder are supported as well. > > Changing something else is at least not my intention. > > > > Regards, > > Adrian > > > > > > Regards, > > Stefano > > > > > > > > Regards, > > > Adrian > > > > > > > > > > I spent quite some time to do the > > > > reverse engineering until I found this VarFlag. > > > > > > > > The only thing I would like to change is that Yocto > > automatically > > > tries > > > > to find a file without the MACHINE appended, if the file > > with the > > > > MACHINE name appended does not exist. There are two > reasons for > > > keeping > > > > the SWUPDATE_IMAGES_NOAPPEND_MACHINE. Backward > compatibility > > and the > > > > very unlike case that the deploy folder contains both > files. > > Then the > > > > SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitly > > specify the > > > right > > > > file. > > > > > > > > I think the new implementation would be simpler to > use. I don't > > > see any > > > > disadvantages such as a not synchronized filename. > > > > > > > > > > Best regards, > > > Stefano Babic > > > > > > > Regards, > > > > Adrian > > > > > > > > Am Donnerstag, 24. Oktober 2019 10:33:57 UTC+2 schrieb > Stefan > > > > Herbrechtsmeier: > > > > > > > > Am 23.10.19 um 23:10 schrieb Adrian Freihofer: > > > > > Try to make the swupdate.bbclass a little bit > smarter in > > > finding the > > > > > image files added to the swu archive. The > > implementation is > > > also a > > > > bit > > > > > more Yocto like. By default, files are searched > > > automatically whereas > > > > > MACHINE specific files get priority over files > without > > the > > > MACHINE in > > > > > the filename. > > > > > This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE > > VarFlag more > > > > optional. > > > > > Most standard use cases are expected to "just work" > > without > > > setting > > > > > this flag. > > > > > > > > NACK, because the filename need to be synchronized > with the > > > > sw-description. A "more Yocto like" implementation > would > > be a > > > > bbclass to > > > > create a sw-description from the recipe variables > like the > > > fitImage > > > > bbclass. > > > > > > > > Regards > > > > Stefan > > > > > > > > -- > > > > You received this message because you are subscribed > to the > > Google > > > > Groups "swupdate" group. > > > > To unsubscribe from this group and stop receiving emails > > from it, > > > send > > > > an email to swup...@googlegroups.com <javascript:> > > > > <mailto:swup...@googlegroups.com <javascript:>>. > > > > To view this discussion on the web visit > > > > > > > > > > https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com>> > > > > > > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com>>> > > > > > > > > > > > > > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>> > > > > > > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer> > > > > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/f6b7f1ce-82b9-4f1f-a4bd-3125456a6cf1%40googlegroups.com?utm_medium=email&utm_source=footer>>>>. > > > > > > > > > > > > -- > > > > > > ===================================================================== > > > DENX Software Engineering GmbH, Managing Director: > > Wolfgang Denk > > > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 > Groebenzell, > > Germany > > > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > > > sba...@denx.de <javascript:> > > > > > > ===================================================================== > > > > > > -- > > > You received this message because you are subscribed to the > Google > > > Groups "swupdate" group. > > > To unsubscribe from this group and stop receiving emails > from it, > > send > > > an email to swup...@googlegroups.com <javascript:> > > > <mailto:swup...@googlegroups.com <javascript:>>. > > > To view this discussion on the web visit > > > > > > https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com>> > > > > > > > > > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer> > > > > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/3adc963c-6b22-4c0a-8451-4d555aa93cc4%40googlegroups.com?utm_medium=email&utm_source=footer>>>. > > > > > > > > > -- > > > ===================================================================== > > DENX Software Engineering GmbH, Managing Director: > Wolfgang Denk > > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > Germany > > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > > sba...@denx.de <javascript:> > > > ===================================================================== > > > > -- > > You received this message because you are subscribed to the Google > > Groups "swupdate" group. > > To unsubscribe from this group and stop receiving emails from it, > send > > an email to swup...@googlegroups.com <javascript:> > > <mailto:swup...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com> > > > > <https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com?utm_medium=email&utm_source=footer > <https://groups.google.com/d/msgid/swupdate/a4fe4890-5fa1-4cff-85a2-a962944c9967%40googlegroups.com?utm_medium=email&utm_source=footer>>. > > > -- > ===================================================================== > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: > sba...@denx.de <javascript:> > ===================================================================== > > -- > You received this message because you are subscribed to the Google > Groups "swupdate" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to swupdate+unsubscribe@googlegroups.com > <mailto:swupdate+unsubscribe@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/swupdate/a1f89eb7-f412-47a4-85a8-ecb917a0c1af%40googlegroups.com > <https://groups.google.com/d/msgid/swupdate/a1f89eb7-f412-47a4-85a8-ecb917a0c1af%40googlegroups.com?utm_medium=email&utm_source=footer>.
diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass index 1d74eef..79c9aae 100644 --- a/classes/swupdate.bbclass +++ b/classes/swupdate.bbclass @@ -7,10 +7,30 @@ # in field. # See also http://sbabic.github.io/swupdate/ # +# To use this class, add swupdate to the inherit clause of the update image bb file. +# The generated output file is an swu archive ready to be uploaded to a device running +# swupdate. # -# To use, add swupdate to the inherit clause and set -# set the images (all of them must be found in deploy directory) -# that are part of the compound image. +# Files listed in the SRC_URI variable are added the the swu archive. +# +# For each entry in the SWUPDATE_IMAGES variable an image file is searched for in the +# ${DEPLOY_DIR_IMAGE} folder and added to the swu archive. Different types of entries +# are supported: +# * image name(s) and fstype(s): +# Example: +# SWUPDATE_IMAGES = "core-image-full-cmdline" +# SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz" +# For this example either a file core-image-full-cmdline-${MACHINE}.ext4.gz or a file +# core-image-full-cmdline.ext4.gz gets added the swu archive. Optionally the variable +# SWUPDATE_IMAGES_NOAPPEND_MACHINE allows to explicitley define if the MACHINE name +# must be part of the image file name or not. +# * image file name(s) +# Example: +# SWUPDATE_IMAGES = "core-image-full-cmdline.ext4.gz" +# If SWUPDATE_IMAGES_FSTYPES is not defined for an entry in SWUPDATE_IMAGES or the +# corresponding image files cannot be found in the ${DEPLOY_DIR_IMAGE} folder, an +# image file with exactly the name as specified in SWUPDATE_IMAGES is searched for. + inherit swupdate-common.bbclass S = "${WORKDIR}/${PN}" @@ -74,6 +94,7 @@ python do_swuimage () { if d.getVar('SWUPDATE_SIGNING', True): list_for_cpio.append('sw-description.sig') + # Add files listed in SRC_URI to the swu file for url in fetch.urls: local = fetch.localpath(url) filename = os.path.basename(local) @@ -81,30 +102,40 @@ python do_swuimage () { shutil.copyfile(local, os.path.join(s, "%s" % filename )) list_for_cpio.append(filename) -# SWUPDATE_IMAGES refers to images in the DEPLOY directory -# If they are not there, additional file can be added -# by fetching from URLs + def add_image_to_swu(deploydir, imagename, s): + src = os.path.join(deploydir, imagename) + if not os.path.isfile(src): + return False + target_imagename = os.path.basename(imagename) # allow images in subfolders of DEPLOY_DIR_IMAGE + dst = os.path.join(s, target_imagename) + shutil.copyfile(src, dst) + list_for_cpio.append(target_imagename) + return True + + # Search for images listed in SWUPDATE_IMAGES in the DEPLOY directory. deploydir = d.getVar('DEPLOY_DIR_IMAGE', True) imgdeploydir = d.getVar('IMGDEPLOYDIR', True) - for image in images: fstypes = (d.getVarFlag("SWUPDATE_IMAGES_FSTYPES", image, True) or "").split() - if not fstypes: - fstypes = [""] - - for fstype in fstypes: - - appendmachine = d.getVarFlag("SWUPDATE_IMAGES_NOAPPEND_MACHINE", image, True) - if appendmachine == None: - imagebase = image + '-' + d.getVar('MACHINE', True) - else: - imagebase = image - - imagename = imagebase + fstype - src = os.path.join(deploydir, "%s" % imagename) - dst = os.path.join(s, "%s" % imagename) - shutil.copyfile(src, dst) - list_for_cpio.append(imagename) + if fstypes: + noappend_machine = d.getVarFlag("SWUPDATE_IMAGES_NOAPPEND_MACHINE", image, True) + if noappend_machine == False: # Search for a file explicitely with MACHINE + imagebases = [ image + '-' + d.getVar('MACHINE', True) ] + elif noappend_machine == True: # Search for a file explicitely without MACHINE + imagebases = [ image ] + else: # None, means auto mode. Just try to find an image file with MACHINE or without MACHINE + imagebases = [ image + '-' + d.getVar('MACHINE', True), image ] + for fstype in fstypes: + image_found = False + for imagebase in imagebases: + image_found = add_image_to_swu(deploydir, imagebase + fstype, s) + if image_found: + break + if not image_found: + bb.fatal("swupdate cannot find image file: %s" % os.path.join(deploydir, imagebase + fstype)) + else: # Allow also complete entries like "image.ext4.gz" in SWUPDATE_IMAGES + if not add_image_to_swu(deploydir, image, s): + bb.fatal("swupdate cannot find %s image file" % image) prepare_sw_description(d, s, list_for_cpio)
Try to make the swupdate.bbclass a little bit smarter in finding the image files added to the swu archive. The implementation is also a bit more Yocto like. By default, files are searched automatically whereas MACHINE specific files get priority over files without the MACHINE in the filename. This makes the SWUPDATE_IMAGES_NOAPPEND_MACHINE VarFlag more optional. Most standard use cases are expected to "just work" without setting this flag. Add support for files in a sub-folder of DEPLOY_DIR_IMAGE. Add documentation to the header of the swupdate.bbclass file. Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> --- classes/swupdate.bbclass | 77 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 54 insertions(+), 23 deletions(-)