mbox series

[RFC,0/6] generic way to deprecate machines

Message ID 20171108022828.7242-1-f4bug@amsat.org
Headers show
Series generic way to deprecate machines | expand

Message

Philippe Mathieu-Daudé Nov. 8, 2017, 2:28 a.m. UTC
Hi,

This series intends to provide a simple and common way to deprecate
machines between releases. It may be extended to deprecate devices.

I started with the machines list from:

  https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines

Comment welcomed :)

Regards,

Phil.

Alistair Francis (1):
  hw/arm: deprecate the EP108 board

Philippe Mathieu-Daudé (5):
  machine: add a deprecated_reason property
  hw/arm: deprecate the Gumstix boards
  hw/ppc: deprecate the PReP machine
  hw/i386: deprecate the pc-0.10 and pc-0.11 machines
  hw/i386: deprecate the "isapc" machine

 include/hw/boards.h  | 1 +
 hw/arm/gumstix.c     | 2 ++
 hw/arm/xlnx-zcu102.c | 6 +++++-
 hw/i386/pc_piix.c    | 3 +++
 hw/ppc/prep.c        | 1 +
 vl.c                 | 7 +++++++
 6 files changed, 19 insertions(+), 1 deletion(-)

Comments

Daniel P. Berrangé Nov. 8, 2017, 9:10 a.m. UTC | #1
On Tue, Nov 07, 2017 at 11:28:22PM -0300, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series intends to provide a simple and common way to deprecate
> machines between releases. It may be extended to deprecate devices.
> 
> I started with the machines list from:
> 
>   https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines
> 
> Comment welcomed :)
> 
> Regards,
> 
> Phil.
> 
> Alistair Francis (1):
>   hw/arm: deprecate the EP108 board
> 
> Philippe Mathieu-Daudé (5):
>   machine: add a deprecated_reason property
>   hw/arm: deprecate the Gumstix boards
>   hw/ppc: deprecate the PReP machine
>   hw/i386: deprecate the pc-0.10 and pc-0.11 machines
>   hw/i386: deprecate the "isapc" machine
> 
>  include/hw/boards.h  | 1 +
>  hw/arm/gumstix.c     | 2 ++
>  hw/arm/xlnx-zcu102.c | 6 +++++-
>  hw/i386/pc_piix.c    | 3 +++
>  hw/ppc/prep.c        | 1 +
>  vl.c                 | 7 +++++++
>  6 files changed, 19 insertions(+), 1 deletion(-)

Unless listed in the qemu-doc.texi "Deprecated features" appendix,
things are not considered deprecated...

Regards,
Daniel
Thomas Huth May 4, 2018, 5:15 p.m. UTC | #2
On 08.11.2017 03:28, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series intends to provide a simple and common way to deprecate
> machines between releases. It may be extended to deprecate devices.

*ping*

Philippe, I just discovered your findings wrt to the Gumstix machines on
https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines ...
sounds reasonable, so I think we should really continue here and mark
those as deprecated. And pc-0.10 and pc-0.11, too.

Do you have some spare time to respin your patch series?

 Thomas
Philippe Mathieu-Daudé May 8, 2018, 3:47 p.m. UTC | #3
Hi Thomas,

On 05/04/2018 02:15 PM, Thomas Huth wrote:
> On 08.11.2017 03:28, Philippe Mathieu-Daudé wrote:
>>
>> This series intends to provide a simple and common way to deprecate
>> machines between releases. It may be extended to deprecate devices.
> 
> *ping*
> 
> Philippe, I just discovered your findings wrt to the Gumstix machines on
> https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines ...
> sounds reasonable, so I think we should really continue here and mark
> those as deprecated. And pc-0.10 and pc-0.11, too.

Ironically I have been using the Gumstix machines quite a lot for the SD
'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
reach the Linux userland since the kernel crashes), and plan to add SD
integration tests via Avocado.

This raises:

- What will happens if I add tests downloading running on their compiled
u-boot
(https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
and the company decides to remove this old directory?
Since sometimes old open-source software are hard to rebuild with recent
compilers, should we consider to use a public storage to keep
open-source (signed) blobs we can use for integration testing?

Avocado has a 'vmimage library' which could be extended, adding support
for binary url + detached gpg signatures from some QEMU maintainers?

(I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
Avocado SuperIO tests, which aren't guaranteed to stay downloadable
forever).

- Maybe I have to volunteer to update the Gumstix section from Orphan
'Odd Fixes' to keep those tests.

> Do you have some spare time to respin your patch series?

I'll make some.

Regards,

Phil.

> 
>  Thomas
>
Eduardo Habkost May 11, 2018, 1:55 p.m. UTC | #4
(CCing Cleber and avocado-devel in case they have suggestions)

On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
[...]
> Ironically I have been using the Gumstix machines quite a lot for the SD
> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
> reach the Linux userland since the kernel crashes), and plan to add SD
> integration tests via Avocado.
> 
> This raises:
> 
> - What will happens if I add tests downloading running on their compiled
> u-boot
> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
> and the company decides to remove this old directory?
> Since sometimes old open-source software are hard to rebuild with recent
> compilers, should we consider to use a public storage to keep
> open-source (signed) blobs we can use for integration testing?

I think a maintained repository of images for testing would be
nice to have.  We need to be careful to comply with the license
of the software being distributed, though.

If the images are very small (like u-boot.bin above), it might be
OK to carry them in qemu.git, just like the images in pc-bios.

> 
> Avocado has a 'vmimage library' which could be extended, adding support
> for binary url + detached gpg signatures from some QEMU maintainers?

Requiring a signature makes the binaries hard to replace.  Any
specific reason to suggest gpg signatures instead of just a
(e.g.) sha256 hash?

> 
> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
> forever).

Question for the Avocado folks: how this is normally handled in
avocado/avocado-vt?  Do you maintain a repository for guest
images, or you always point to their original sources?
Cleber Rosa May 11, 2018, 2:27 p.m. UTC | #5
On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
> (CCing Cleber and avocado-devel in case they have suggestions)
> 
> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
> [...]
>> Ironically I have been using the Gumstix machines quite a lot for the SD
>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>> reach the Linux userland since the kernel crashes), and plan to add SD
>> integration tests via Avocado.
>>
>> This raises:
>>
>> - What will happens if I add tests downloading running on their compiled
>> u-boot
>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>> and the company decides to remove this old directory?
>> Since sometimes old open-source software are hard to rebuild with recent
>> compilers, should we consider to use a public storage to keep
>> open-source (signed) blobs we can use for integration testing?
> 
> I think a maintained repository of images for testing would be
> nice to have.  We need to be careful to comply with the license
> of the software being distributed, though.
> 
> If the images are very small (like u-boot.bin above), it might be
> OK to carry them in qemu.git, just like the images in pc-bios.
> 
>>
>> Avocado has a 'vmimage library' which could be extended, adding support
>> for binary url + detached gpg signatures from some QEMU maintainers?
> 
> Requiring a signature makes the binaries hard to replace.  Any
> specific reason to suggest gpg signatures instead of just a
> (e.g.) sha256 hash?
> 
>>
>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>> forever).
> 
> Question for the Avocado folks: how this is normally handled in
> avocado/avocado-vt?  Do you maintain a repository for guest
> images, or you always point to their original sources?
> 

For pure Avocado, the vmimage library attempts to fetch, by default, the
latest version of a guest image directly from the original sources.
Say, a Fedora image will be downloaded by default from the Fedora
servers.  Because of that, we don't pay too much attention to the
availability of specific (old?) versions of guest images.

For Avocado-VT, there are the JeOS images[1], which we keep on a test
"assets" directory.  We have a lot of storage/bandwidth availability, so
it can be used for other assets proven to be necessary for tests.

As long as distribution rights and licensing are not issues, we can
definitely use the same server for kernels, u-boot images and what not.

[1] - https://avocado-project.org/data/assets/
Alistair Francis May 18, 2018, 6:16 p.m. UTC | #6
On Fri, May 11, 2018 at 7:27 AM, Cleber Rosa <crosa@redhat.com> wrote:
>
>
> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>> (CCing Cleber and avocado-devel in case they have suggestions)
>>
>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>> [...]
>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>> integration tests via Avocado.
>>>
>>> This raises:
>>>
>>> - What will happens if I add tests downloading running on their compiled
>>> u-boot
>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>> and the company decides to remove this old directory?
>>> Since sometimes old open-source software are hard to rebuild with recent
>>> compilers, should we consider to use a public storage to keep
>>> open-source (signed) blobs we can use for integration testing?
>>
>> I think a maintained repository of images for testing would be
>> nice to have.  We need to be careful to comply with the license
>> of the software being distributed, though.
>>
>> If the images are very small (like u-boot.bin above), it might be
>> OK to carry them in qemu.git, just like the images in pc-bios.
>>
>>>
>>> Avocado has a 'vmimage library' which could be extended, adding support
>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>
>> Requiring a signature makes the binaries hard to replace.  Any
>> specific reason to suggest gpg signatures instead of just a
>> (e.g.) sha256 hash?
>>
>>>
>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>> forever).
>>
>> Question for the Avocado folks: how this is normally handled in
>> avocado/avocado-vt?  Do you maintain a repository for guest
>> images, or you always point to their original sources?
>>
>
> For pure Avocado, the vmimage library attempts to fetch, by default, the
> latest version of a guest image directly from the original sources.
> Say, a Fedora image will be downloaded by default from the Fedora
> servers.  Because of that, we don't pay too much attention to the
> availability of specific (old?) versions of guest images.
>
> For Avocado-VT, there are the JeOS images[1], which we keep on a test
> "assets" directory.  We have a lot of storage/bandwidth availability, so
> it can be used for other assets proven to be necessary for tests.
>
> As long as distribution rights and licensing are not issues, we can
> definitely use the same server for kernels, u-boot images and what not.
>
> [1] - https://avocado-project.org/data/assets/

Is it possible to add something to the landing page at
https://avocado-project.org ?

The Palo Alto Network routers block the avocado-project.org page as
they classify it as blank. Something on the root URL would help fix
this.

Alistair

>
> --
> Cleber Rosa
> [ Sr Software Engineer - Virtualization Team - Red Hat ]
> [ Avocado Test Framework - avocado-framework.github.io ]
> [  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]
>
Cleber Rosa May 22, 2018, 4:17 p.m. UTC | #7
On 05/18/2018 02:16 PM, Alistair Francis wrote:
> On Fri, May 11, 2018 at 7:27 AM, Cleber Rosa <crosa@redhat.com> wrote:
>>
>>
>> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>>> (CCing Cleber and avocado-devel in case they have suggestions)
>>>
>>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>>> [...]
>>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>>> integration tests via Avocado.
>>>>
>>>> This raises:
>>>>
>>>> - What will happens if I add tests downloading running on their compiled
>>>> u-boot
>>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>>> and the company decides to remove this old directory?
>>>> Since sometimes old open-source software are hard to rebuild with recent
>>>> compilers, should we consider to use a public storage to keep
>>>> open-source (signed) blobs we can use for integration testing?
>>>
>>> I think a maintained repository of images for testing would be
>>> nice to have.  We need to be careful to comply with the license
>>> of the software being distributed, though.
>>>
>>> If the images are very small (like u-boot.bin above), it might be
>>> OK to carry them in qemu.git, just like the images in pc-bios.
>>>
>>>>
>>>> Avocado has a 'vmimage library' which could be extended, adding support
>>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>>
>>> Requiring a signature makes the binaries hard to replace.  Any
>>> specific reason to suggest gpg signatures instead of just a
>>> (e.g.) sha256 hash?
>>>
>>>>
>>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>>> forever).
>>>
>>> Question for the Avocado folks: how this is normally handled in
>>> avocado/avocado-vt?  Do you maintain a repository for guest
>>> images, or you always point to their original sources?
>>>
>>
>> For pure Avocado, the vmimage library attempts to fetch, by default, the
>> latest version of a guest image directly from the original sources.
>> Say, a Fedora image will be downloaded by default from the Fedora
>> servers.  Because of that, we don't pay too much attention to the
>> availability of specific (old?) versions of guest images.
>>
>> For Avocado-VT, there are the JeOS images[1], which we keep on a test
>> "assets" directory.  We have a lot of storage/bandwidth availability, so
>> it can be used for other assets proven to be necessary for tests.
>>
>> As long as distribution rights and licensing are not issues, we can
>> definitely use the same server for kernels, u-boot images and what not.
>>
>> [1] - https://avocado-project.org/data/assets/
> 
> Is it possible to add something to the landing page at
> https://avocado-project.org ?
> 

Done!
- Cleber.

> The Palo Alto Network routers block the avocado-project.org page as
> they classify it as blank. Something on the root URL would help fix
> this.
> 
> Alistair
>
Alistair Francis May 22, 2018, 4:34 p.m. UTC | #8
On Tue, May 22, 2018 at 9:17 AM, Cleber Rosa <crosa@redhat.com> wrote:
>
>
> On 05/18/2018 02:16 PM, Alistair Francis wrote:
>> On Fri, May 11, 2018 at 7:27 AM, Cleber Rosa <crosa@redhat.com> wrote:
>>>
>>>
>>> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>>>> (CCing Cleber and avocado-devel in case they have suggestions)
>>>>
>>>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>>>> [...]
>>>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>>>> integration tests via Avocado.
>>>>>
>>>>> This raises:
>>>>>
>>>>> - What will happens if I add tests downloading running on their compiled
>>>>> u-boot
>>>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>>>> and the company decides to remove this old directory?
>>>>> Since sometimes old open-source software are hard to rebuild with recent
>>>>> compilers, should we consider to use a public storage to keep
>>>>> open-source (signed) blobs we can use for integration testing?
>>>>
>>>> I think a maintained repository of images for testing would be
>>>> nice to have.  We need to be careful to comply with the license
>>>> of the software being distributed, though.
>>>>
>>>> If the images are very small (like u-boot.bin above), it might be
>>>> OK to carry them in qemu.git, just like the images in pc-bios.
>>>>
>>>>>
>>>>> Avocado has a 'vmimage library' which could be extended, adding support
>>>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>>>
>>>> Requiring a signature makes the binaries hard to replace.  Any
>>>> specific reason to suggest gpg signatures instead of just a
>>>> (e.g.) sha256 hash?
>>>>
>>>>>
>>>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>>>> forever).
>>>>
>>>> Question for the Avocado folks: how this is normally handled in
>>>> avocado/avocado-vt?  Do you maintain a repository for guest
>>>> images, or you always point to their original sources?
>>>>
>>>
>>> For pure Avocado, the vmimage library attempts to fetch, by default, the
>>> latest version of a guest image directly from the original sources.
>>> Say, a Fedora image will be downloaded by default from the Fedora
>>> servers.  Because of that, we don't pay too much attention to the
>>> availability of specific (old?) versions of guest images.
>>>
>>> For Avocado-VT, there are the JeOS images[1], which we keep on a test
>>> "assets" directory.  We have a lot of storage/bandwidth availability, so
>>> it can be used for other assets proven to be necessary for tests.
>>>
>>> As long as distribution rights and licensing are not issues, we can
>>> definitely use the same server for kernels, u-boot images and what not.
>>>
>>> [1] - https://avocado-project.org/data/assets/
>>
>> Is it possible to add something to the landing page at
>> https://avocado-project.org ?
>>
>
> Done!

Awesome! It looks good and this should help everyone behind a proxy.

Alistair

> - Cleber.
>
>> The Palo Alto Network routers block the avocado-project.org page as
>> they classify it as blank. Something on the root URL would help fix
>> this.
>>
>> Alistair
>>
Philippe Mathieu-Daudé June 17, 2019, 5:17 a.m. UTC | #9
Hi Cleber,

On 5/11/18 4:27 PM, Cleber Rosa wrote:
> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>> (CCing Cleber and avocado-devel in case they have suggestions)
>>
>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>> [...]
>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>> integration tests via Avocado.
>>>
>>> This raises:
>>>
>>> - What will happens if I add tests downloading running on their compiled
>>> u-boot
>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>> and the company decides to remove this old directory?
>>> Since sometimes old open-source software are hard to rebuild with recent
>>> compilers, should we consider to use a public storage to keep
>>> open-source (signed) blobs we can use for integration testing?
>>
>> I think a maintained repository of images for testing would be
>> nice to have.  We need to be careful to comply with the license
>> of the software being distributed, though.
>>
>> If the images are very small (like u-boot.bin above), it might be
>> OK to carry them in qemu.git, just like the images in pc-bios.
>>
>>>
>>> Avocado has a 'vmimage library' which could be extended, adding support
>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>
>> Requiring a signature makes the binaries hard to replace.  Any
>> specific reason to suggest gpg signatures instead of just a
>> (e.g.) sha256 hash?
>>
>>>
>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>> forever).
>>
>> Question for the Avocado folks: how this is normally handled in
>> avocado/avocado-vt?  Do you maintain a repository for guest
>> images, or you always point to their original sources?
>>
> 
> For pure Avocado, the vmimage library attempts to fetch, by default, the
> latest version of a guest image directly from the original sources.
> Say, a Fedora image will be downloaded by default from the Fedora
> servers.  Because of that, we don't pay too much attention to the
> availability of specific (old?) versions of guest images.
> 
> For Avocado-VT, there are the JeOS images[1], which we keep on a test
> "assets" directory.  We have a lot of storage/bandwidth availability, so
> it can be used for other assets proven to be necessary for tests.
> 
> As long as distribution rights and licensing are not issues, we can
> definitely use the same server for kernels, u-boot images and what not.
> 
> [1] - https://avocado-project.org/data/assets/

1/ How do we check for distribution rights?

Is it OK for:
- a Debian/Fedora image
- a compiled Linux kernel (for a Debian/Fedora release)

2/ Who to ask to add files to this assets directory?

3/ Can we use a 'webarchive' directory structure?

Such /site/date/original_site_path/file

4/ What are the chances that this website disappears? :S

(Someone has to pay for it, and the bandwidth...)

Thanks,

Phil.
Philippe Mathieu-Daudé Aug. 16, 2019, 10:51 a.m. UTC | #10
ping? :)

On 6/17/19 7:17 AM, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
> 
> On 5/11/18 4:27 PM, Cleber Rosa wrote:
>> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>>> (CCing Cleber and avocado-devel in case they have suggestions)
>>>
>>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>>> [...]
>>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>>> integration tests via Avocado.
>>>>
>>>> This raises:
>>>>
>>>> - What will happens if I add tests downloading running on their compiled
>>>> u-boot
>>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>>> and the company decides to remove this old directory?
>>>> Since sometimes old open-source software are hard to rebuild with recent
>>>> compilers, should we consider to use a public storage to keep
>>>> open-source (signed) blobs we can use for integration testing?
>>>
>>> I think a maintained repository of images for testing would be
>>> nice to have.  We need to be careful to comply with the license
>>> of the software being distributed, though.
>>>
>>> If the images are very small (like u-boot.bin above), it might be
>>> OK to carry them in qemu.git, just like the images in pc-bios.
>>>
>>>>
>>>> Avocado has a 'vmimage library' which could be extended, adding support
>>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>>
>>> Requiring a signature makes the binaries hard to replace.  Any
>>> specific reason to suggest gpg signatures instead of just a
>>> (e.g.) sha256 hash?
>>>
>>>>
>>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>>> forever).
>>>
>>> Question for the Avocado folks: how this is normally handled in
>>> avocado/avocado-vt?  Do you maintain a repository for guest
>>> images, or you always point to their original sources?
>>>
>>
>> For pure Avocado, the vmimage library attempts to fetch, by default, the
>> latest version of a guest image directly from the original sources.
>> Say, a Fedora image will be downloaded by default from the Fedora
>> servers.  Because of that, we don't pay too much attention to the
>> availability of specific (old?) versions of guest images.
>>
>> For Avocado-VT, there are the JeOS images[1], which we keep on a test
>> "assets" directory.  We have a lot of storage/bandwidth availability, so
>> it can be used for other assets proven to be necessary for tests.
>>
>> As long as distribution rights and licensing are not issues, we can
>> definitely use the same server for kernels, u-boot images and what not.
>>
>> [1] - https://avocado-project.org/data/assets/
> 
> 1/ How do we check for distribution rights?
> 
> Is it OK for:
> - a Debian/Fedora image
> - a compiled Linux kernel (for a Debian/Fedora release)
> 
> 2/ Who to ask to add files to this assets directory?
> 
> 3/ Can we use a 'webarchive' directory structure?
> 
> Such /site/date/original_site_path/file
> 
> 4/ What are the chances that this website disappears? :S
> 
> (Someone has to pay for it, and the bandwidth...)
> 
> Thanks,
> 
> Phil.
Cleber Rosa Aug. 19, 2019, 1:10 p.m. UTC | #11
On Fri, Aug 16, 2019 at 12:51:06PM +0200, Philippe Mathieu-Daudé wrote:
> ping? :)
> 
> On 6/17/19 7:17 AM, Philippe Mathieu-Daudé wrote:
> > Hi Cleber,
> > 
> > On 5/11/18 4:27 PM, Cleber Rosa wrote:
> >> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
> >>> (CCing Cleber and avocado-devel in case they have suggestions)
> >>>
> >>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
> >>> [...]
> >>>> Ironically I have been using the Gumstix machines quite a lot for the SD
> >>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
> >>>> reach the Linux userland since the kernel crashes), and plan to add SD
> >>>> integration tests via Avocado.
> >>>>
> >>>> This raises:
> >>>>
> >>>> - What will happens if I add tests downloading running on their compiled
> >>>> u-boot
> >>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
> >>>> and the company decides to remove this old directory?
> >>>> Since sometimes old open-source software are hard to rebuild with recent
> >>>> compilers, should we consider to use a public storage to keep
> >>>> open-source (signed) blobs we can use for integration testing?
> >>>
> >>> I think a maintained repository of images for testing would be
> >>> nice to have.  We need to be careful to comply with the license
> >>> of the software being distributed, though.
> >>>
> >>> If the images are very small (like u-boot.bin above), it might be
> >>> OK to carry them in qemu.git, just like the images in pc-bios.
> >>>
> >>>>
> >>>> Avocado has a 'vmimage library' which could be extended, adding support
> >>>> for binary url + detached gpg signatures from some QEMU maintainers?

Yes, although I believe a stronger distinction between the images
originally targeted by avocado.utils.vmimage and those other images
should probably exist.  One of the reasons that make me think so is
that the images obtained through vmimage *should* be configurable by
the accompanying avocado.utils.cloudinit library.

> >>>
> >>> Requiring a signature makes the binaries hard to replace.  Any
> >>> specific reason to suggest gpg signatures instead of just a
> >>> (e.g.) sha256 hash?
> >>>
> >>>>
> >>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
> >>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
> >>>> forever).
> >>>
> >>> Question for the Avocado folks: how this is normally handled in
> >>> avocado/avocado-vt?  Do you maintain a repository for guest
> >>> images, or you always point to their original sources?
> >>>
> >>
> >> For pure Avocado, the vmimage library attempts to fetch, by default, the
> >> latest version of a guest image directly from the original sources.
> >> Say, a Fedora image will be downloaded by default from the Fedora
> >> servers.  Because of that, we don't pay too much attention to the
> >> availability of specific (old?) versions of guest images.
> >>
> >> For Avocado-VT, there are the JeOS images[1], which we keep on a test
> >> "assets" directory.  We have a lot of storage/bandwidth availability, so
> >> it can be used for other assets proven to be necessary for tests.
> >>
> >> As long as distribution rights and licensing are not issues, we can
> >> definitely use the same server for kernels, u-boot images and what not.
> >>
> >> [1] - https://avocado-project.org/data/assets/
> > 
> > 1/ How do we check for distribution rights?
> > 
> > Is it OK for:
> > - a Debian/Fedora image
> > - a compiled Linux kernel (for a Debian/Fedora release)
> >

I think released images are just fine.

FIY, with regards to custom iamges: the Avocado-VT JeOS images are
based on Fedora, and can be recreated with the kickstart files that
are also in-tree.  Making the custom binary file reproducible is,
AFAICT good enough.

> > 2/ Who to ask to add files to this assets directory?
> >

I'd be happy to setup a secure upload mechanism, say, ftp+ssl, so that
"image maintainers" can upload new images.

> > 3/ Can we use a 'webarchive' directory structure?
> > 
> > Such /site/date/original_site_path/file
> >

I don't see why not.

> > 4/ What are the chances that this website disappears? :S
> > 
> > (Someone has to pay for it, and the bandwidth...)

Of course the best scenario would be to rely on some bigger
institution, say an University decides to host this archive of images,
and others decide to mirror them.  Until then, if it's decided to push
forward the use of the "avocado-project.org" site, the chances of it
going away are basically if I fail to have $5/month to keep it
running, so hopefully (I mean it) this won't be an issue :)

And thans for the ping, although I was really in need of a brick thrown
at my head to wake me up on this thread! :)

- Cleber.

> > 
> > Thanks,
> > 
> > Phil.