[meta-swupdate,1/3] classes: introduce new bbclass to handle package version generation

Introduce new class that provides a functionality to record the version
of the component in the format, that could be used during sw-description
file construction.

Once this class is inherited by a recipe - it would generate a file in
image deployment folder, which would contain a version of the package
(either SRCREV or PV) together with a package name.

Signed-off-by: Andrey Zhizhikin <andrey.zhizhikin@leica-geosystems.com>
---
 classes/swupdate-ver-collector.bbclass | 30 ++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
 create mode 100644 classes/swupdate-ver-collector.bbclass

Hi Andrey,

On 17/01/20 22:30, Andrey Zhizhikin wrote:
> Introduce new class that provides a functionality to record the version
> of the component in the format, that could be used during sw-description
> file construction.
> 
> Once this class is inherited by a recipe - it would generate a file in
> image deployment folder, which would contain a version of the package
> (either SRCREV or PV) together with a package name.
> 
> Signed-off-by: Andrey Zhizhikin <andrey.zhizhikin@leica-geosystems.com>
> ---
>  classes/swupdate-ver-collector.bbclass | 30 ++++++++++++++++++++++++++
>  1 file changed, 30 insertions(+)
>  create mode 100644 classes/swupdate-ver-collector.bbclass
> 
> diff --git a/classes/swupdate-ver-collector.bbclass b/classes/swupdate-ver-collector.bbclass
> new file mode 100644
> index 0000000..70b1a94
> --- /dev/null
> +++ b/classes/swupdate-ver-collector.bbclass
> @@ -0,0 +1,30 @@
> +#
> +# Version Collector class for SWUpdate component
> +#
> +# This class should be inherited by components, which are later included in SWU
> +# file and required to be versioned there (via sw-description's 'version' parameter).
> +#
> +# This class would produce a file in deployment folder with the name of the
> +# component and would list a pair:
> +# <name of component>     <version>
> +#
> +# Files produced by multiple components would later be combined into sw-versions
> +# file, which is used to udpate <pkg-ver></pkg-ver> tags in sw-description file
> +# and could also be taken directly into the target rootfs and verified by swupdate
> +# in runtime.
> +#
> +
> +do_generate_package_ver_for_swu() {
> +        # Create folder where version file should be present
> +        install -d ${DEPLOY_DIR_IMAGE}/swupdate-versions
> +
> +	# If the SRCREV is set - use it to report the version, otherwise report PV
> +	if [ "${SRCREV}" != "AUTOINC" ] && [ "${SRCREV}" != "INVALID" ]; then
> +		echo "${BPN}    ${SRCREV}" > ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version
> +	else
> +		bbnote "Recipe does not provide valid SRCREV, use PV as reported version"
> +		echo "${BPN}    ${PV}" > ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version
> +	fi
> +}
> +
> +addtask generate_package_ver_for_swu before do_build after do_compile

Not sure if this can cover all cases - see my comment for 0/3. I will
suggest that we let the possibility to add a VERSIOn directly into the
recipe, in case SRCREV or PV are not enough.

Best regards,
Stefano Babic

On Sun, Jan 19, 2020 at 4:09 PM Stefano Babic <sbabic@denx.de> wrote:
>
> Hi Andrey,
>
> On 17/01/20 22:30, Andrey Zhizhikin wrote:
> > Introduce new class that provides a functionality to record the version
> > of the component in the format, that could be used during sw-description
> > file construction.
> >
> > Once this class is inherited by a recipe - it would generate a file in
> > image deployment folder, which would contain a version of the package
> > (either SRCREV or PV) together with a package name.
> >
> > Signed-off-by: Andrey Zhizhikin <andrey.zhizhikin@leica-geosystems.com>
> > ---
> >  classes/swupdate-ver-collector.bbclass | 30 ++++++++++++++++++++++++++
> >  1 file changed, 30 insertions(+)
> >  create mode 100644 classes/swupdate-ver-collector.bbclass
> >
> > diff --git a/classes/swupdate-ver-collector.bbclass b/classes/swupdate-ver-collector.bbclass
> > new file mode 100644
> > index 0000000..70b1a94
> > --- /dev/null
> > +++ b/classes/swupdate-ver-collector.bbclass
> > @@ -0,0 +1,30 @@
> > +#
> > +# Version Collector class for SWUpdate component
> > +#
> > +# This class should be inherited by components, which are later included in SWU
> > +# file and required to be versioned there (via sw-description's 'version' parameter).
> > +#
> > +# This class would produce a file in deployment folder with the name of the
> > +# component and would list a pair:
> > +# <name of component>     <version>
> > +#
> > +# Files produced by multiple components would later be combined into sw-versions
> > +# file, which is used to udpate <pkg-ver></pkg-ver> tags in sw-description file
> > +# and could also be taken directly into the target rootfs and verified by swupdate
> > +# in runtime.
> > +#
> > +
> > +do_generate_package_ver_for_swu() {
> > +        # Create folder where version file should be present
> > +        install -d ${DEPLOY_DIR_IMAGE}/swupdate-versions
> > +
> > +     # If the SRCREV is set - use it to report the version, otherwise report PV
> > +     if [ "${SRCREV}" != "AUTOINC" ] && [ "${SRCREV}" != "INVALID" ]; then
> > +             echo "${BPN}    ${SRCREV}" > ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version
> > +     else
> > +             bbnote "Recipe does not provide valid SRCREV, use PV as reported version"
> > +             echo "${BPN}    ${PV}" > ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version
> > +     fi
> > +}
> > +
> > +addtask generate_package_ver_for_swu before do_build after do_compile
>
> Not sure if this can cover all cases - see my comment for 0/3. I will
> suggest that we let the possibility to add a VERSIOn directly into the
> recipe, in case SRCREV or PV are not enough.

Agree. It would be beneficial if a user can define an arbitrary
version via additional variable, which could be weakly defined in this
class. In case if user does not provide any version he/she deliberetly
would like to see in SWU file - either SRCREV or PV would be used
instead.

I think something like this should be fine:
SWU_VER ??= "${SRCREV}"

What do you think about this?

>
> Best regards,
> Stefano Babic
>
> --
> =====================================================================
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
> =====================================================================

Hi Andrey,

On 19/01/20 20:48, Andrey Zhizhikin wrote:
> On Sun, Jan 19, 2020 at 4:09 PM Stefano Babic <sbabic@denx.de> wrote:
>>
>> Hi Andrey,
>>
>> On 17/01/20 22:30, Andrey Zhizhikin wrote:
>>> Introduce new class that provides a functionality to record the version
>>> of the component in the format, that could be used during sw-description
>>> file construction.
>>>
>>> Once this class is inherited by a recipe - it would generate a file in
>>> image deployment folder, which would contain a version of the package
>>> (either SRCREV or PV) together with a package name.
>>>
>>> Signed-off-by: Andrey Zhizhikin <andrey.zhizhikin@leica-geosystems.com>
>>> ---
>>>  classes/swupdate-ver-collector.bbclass | 30 ++++++++++++++++++++++++++
>>>  1 file changed, 30 insertions(+)
>>>  create mode 100644 classes/swupdate-ver-collector.bbclass
>>>
>>> diff --git a/classes/swupdate-ver-collector.bbclass b/classes/swupdate-ver-collector.bbclass
>>> new file mode 100644
>>> index 0000000..70b1a94
>>> --- /dev/null
>>> +++ b/classes/swupdate-ver-collector.bbclass
>>> @@ -0,0 +1,30 @@
>>> +#
>>> +# Version Collector class for SWUpdate component
>>> +#
>>> +# This class should be inherited by components, which are later included in SWU
>>> +# file and required to be versioned there (via sw-description's 'version' parameter).
>>> +#
>>> +# This class would produce a file in deployment folder with the name of the
>>> +# component and would list a pair:
>>> +# <name of component>     <version>
>>> +#
>>> +# Files produced by multiple components would later be combined into sw-versions
>>> +# file, which is used to udpate <pkg-ver></pkg-ver> tags in sw-description file
>>> +# and could also be taken directly into the target rootfs and verified by swupdate
>>> +# in runtime.
>>> +#
>>> +
>>> +do_generate_package_ver_for_swu() {
>>> +        # Create folder where version file should be present
>>> +        install -d ${DEPLOY_DIR_IMAGE}/swupdate-versions
>>> +
>>> +     # If the SRCREV is set - use it to report the version, otherwise report PV
>>> +     if [ "${SRCREV}" != "AUTOINC" ] && [ "${SRCREV}" != "INVALID" ]; then
>>> +             echo "${BPN}    ${SRCREV}" > ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version
>>> +     else
>>> +             bbnote "Recipe does not provide valid SRCREV, use PV as reported version"
>>> +             echo "${BPN}    ${PV}" > ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version
>>> +     fi
>>> +}
>>> +
>>> +addtask generate_package_ver_for_swu before do_build after do_compile
>>
>> Not sure if this can cover all cases - see my comment for 0/3. I will
>> suggest that we let the possibility to add a VERSIOn directly into the
>> recipe, in case SRCREV or PV are not enough.
> 
> Agree. It would be beneficial if a user can define an arbitrary
> version via additional variable, which could be weakly defined in this
> class.

Right.

> In case if user does not provide any version he/she deliberetly
> would like to see in SWU file - either SRCREV or PV would be used
> instead.
> 
> I think something like this should be fine:
> SWU_VER ??= "${SRCREV}"
> 
> What do you think about this?

The only important thing for the name is that there should be no
conflict with official OE variable and that the new variable will be
documented in building-with-yocto.rst.

Best regards,
Stefano Babic

Hi Andrey,

IMHO a solution without an extra class which needs to be inherited by 
several recipes would be better.
The information exported by your swupdate-ver-collector.bbclass is already 
exported by Yocto.
In the manifest file of the image all installed packages are listed. 
Further information about a package
is in the pkgdata folder. Example:

The folder build/tmp/work/aarch64-ccp-linux/acl/2.2.52-r0/pkgdata/runtime 
contains a file acl. It looks like:

PN: acl
PV: 2.2.52
PR: r0
PKGV: 2.2.52
PKGR: r0
LICENSE_acl: GPLv2+
...

Maybe adding something like the following could help you the get the 
information into swupdate-common.bbclass

python get_pkg_pv () {
    image_manifest = d.getVar('IMAGE_MANIFEST')
    f = open(image_manifest, 'r')
    for line in f:
        pkg = line.split()[0]
        bb.debug(1, 'Fetching packagedata for %s' % pkg)
        pkgdata_dir = d.getVar("PKGDATA_DIR", True)
        try:
            pkgdata = 
oe.packagedata.read_pkgdatafile("%s/runtime-reverse/%s" % (pkgdata_dir, 
pkg))
        except Exception as e:
            bb.warn("Failed to read pkgdata file %s/runtime-reverse/%s: %s: 
%s" % (pkgdata_dir, pkg, e.__class__, str(e)))
            continue
        package_pv = [value for key, value in pkgdata.items() if 
key.startswith("PV")][0]


This would probably also solve the issues with the append file.

Regards,
Adrian

Am Freitag, 17. Januar 2020 22:30:27 UTC+1 schrieb Andrey Zhizhikin:
>
> Introduce new class that provides a functionality to record the version 
> of the component in the format, that could be used during sw-description 
> file construction. 
>
> Once this class is inherited by a recipe - it would generate a file in 
> image deployment folder, which would contain a version of the package 
> (either SRCREV or PV) together with a package name. 
>
> Signed-off-by: Andrey Zhizhikin <andrey....@leica-geosystems.com 
> <javascript:>> 
> --- 
>  classes/swupdate-ver-collector.bbclass | 30 ++++++++++++++++++++++++++ 
>  1 file changed, 30 insertions(+) 
>  create mode 100644 classes/swupdate-ver-collector.bbclass 
>
> diff --git a/classes/swupdate-ver-collector.bbclass 
> b/classes/swupdate-ver-collector.bbclass 
> new file mode 100644 
> index 0000000..70b1a94 
> --- /dev/null 
> +++ b/classes/swupdate-ver-collector.bbclass 
> @@ -0,0 +1,30 @@ 
> +# 
> +# Version Collector class for SWUpdate component 
> +# 
> +# This class should be inherited by components, which are later included 
> in SWU 
> +# file and required to be versioned there (via sw-description's 'version' 
> parameter). 
> +# 
> +# This class would produce a file in deployment folder with the name of 
> the 
> +# component and would list a pair: 
> +# <name of component>     <version> 
> +# 
> +# Files produced by multiple components would later be combined into 
> sw-versions 
> +# file, which is used to udpate <pkg-ver></pkg-ver> tags in 
> sw-description file 
> +# and could also be taken directly into the target rootfs and verified by 
> swupdate 
> +# in runtime. 
> +# 
> + 
> +do_generate_package_ver_for_swu() { 
> +        # Create folder where version file should be present 
> +        install -d ${DEPLOY_DIR_IMAGE}/swupdate-versions 
> + 
> +        # If the SRCREV is set - use it to report the version, otherwise 
> report PV 
> +        if [ "${SRCREV}" != "AUTOINC" ] && [ "${SRCREV}" != "INVALID" ]; 
> then 
> +                echo "${BPN}    ${SRCREV}" > 
> ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version 
> +        else 
> +                bbnote "Recipe does not provide valid SRCREV, use PV as 
> reported version" 
> +                echo "${BPN}    ${PV}" > 
> ${DEPLOY_DIR_IMAGE}/swupdate-versions/${BPN}.version 
> +        fi 
> +} 
> + 
> +addtask generate_package_ver_for_swu before do_build after do_compile 
> -- 
> 2.17.1 
>
>