diff mbox series

[08/10] ci, docker: update CentOS and OpenSUSE Python to non-EOL versions

Message ID 20230222143752.466090-9-pbonzini@redhat.com
State New
Headers show
Series improvement to Python detection, preparation for dropping 3.6 | expand

Commit Message

Paolo Bonzini Feb. 22, 2023, 2:37 p.m. UTC 
Python 3.6 is at end-of-life.  libvirt-ci now supports overrides for
targets and package mappings, therefore QEMU can use newer versions
provided by CentOS 8 (Python 3.8) and OpenSUSE 15.3 (Python 3.9).

Packages that the distro does not provide are included in the image
via PyPI; lcitool knows how to do that whenever the rpm mapping
is empty.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/testing.rst                        |  6 ++
 tests/docker/dockerfiles/centos8.docker       | 22 +++---
 tests/docker/dockerfiles/opensuse-leap.docker | 22 +++---
 tests/docker/dockerfiles/ubuntu2004.docker    |  2 +-
 tests/lcitool/mappings.yml                    | 77 +++++++++++++++++++
 tests/lcitool/targets/centos-stream-8.yml     |  3 +
 tests/lcitool/targets/opensuse-leap-153.yml   |  3 +
 7 files changed, 113 insertions(+), 22 deletions(-)
 create mode 100644 tests/lcitool/mappings.yml
 create mode 100644 tests/lcitool/targets/centos-stream-8.yml
 create mode 100644 tests/lcitool/targets/opensuse-leap-153.yml

Comments

Daniel P. Berrangé Feb. 22, 2023, 3:14 p.m. UTC | #1 
On Wed, Feb 22, 2023 at 03:37:50PM +0100, Paolo Bonzini wrote:
> Python 3.6 is at end-of-life.  libvirt-ci now supports overrides for
> targets and package mappings, therefore QEMU can use newer versions
> provided by CentOS 8 (Python 3.8) and OpenSUSE 15.3 (Python 3.9).
> 
> Packages that the distro does not provide are included in the image
> via PyPI; lcitool knows how to do that whenever the rpm mapping
> is empty.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  docs/devel/testing.rst                        |  6 ++
>  tests/docker/dockerfiles/centos8.docker       | 22 +++---
>  tests/docker/dockerfiles/opensuse-leap.docker | 22 +++---
>  tests/docker/dockerfiles/ubuntu2004.docker    |  2 +-
>  tests/lcitool/mappings.yml                    | 77 +++++++++++++++++++
>  tests/lcitool/targets/centos-stream-8.yml     |  3 +
>  tests/lcitool/targets/opensuse-leap-153.yml   |  3 +
>  7 files changed, 113 insertions(+), 22 deletions(-)
>  create mode 100644 tests/lcitool/mappings.yml
>  create mode 100644 tests/lcitool/targets/centos-stream-8.yml
>  create mode 100644 tests/lcitool/targets/opensuse-leap-153.yml

> 
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 648b7aa09137..67798dc63def 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -481,6 +481,12 @@ the mapping to the ``libvirt-ci`` project as well:
>     contains the ``mappings.yml`` update.  Then add the prerequisite and
>     run ``make lcitool-refresh``.
>  
> +For enterprise distros that default to old, end-of-life versions of the
> +Python runtime, QEMU uses a separate set of mappings that work with more
> +recent versions.  These can be found in ``tests/lcitool/mappings.yml``.
> +These should not be a problem unless the dependencies you are adding
> +is a Python library.

I'd suggest changing this last sentence to

 It should not be required to modify this file unless the new
 dependency being added is a Python library.


Either way,

  Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

With regards,
Daniel
diff mbox series

Patch

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 648b7aa09137..67798dc63def 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -481,6 +481,12 @@  the mapping to the ``libvirt-ci`` project as well:
    contains the ``mappings.yml`` update.  Then add the prerequisite and
    run ``make lcitool-refresh``.
 
+For enterprise distros that default to old, end-of-life versions of the
+Python runtime, QEMU uses a separate set of mappings that work with more
+recent versions.  These can be found in ``tests/lcitool/mappings.yml``.
+These should not be a problem unless the dependencies you are adding
+is a Python library.
+
 
 Adding new OS distros
 ^^^^^^^^^^^^^^^^^^^^^
diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker
index fbc953c6dccc..3c74be09a693 100644
--- a/tests/docker/dockerfiles/centos8.docker
+++ b/tests/docker/dockerfiles/centos8.docker
@@ -82,7 +82,6 @@  RUN dnf distro-sync -y && \
         lzo-devel \
         make \
         mesa-libgbm-devel \
-        meson \
         ncurses-devel \
         nettle-devel \
         ninja-build \
@@ -94,13 +93,12 @@  RUN dnf distro-sync -y && \
         pixman-devel \
         pkgconfig \
         pulseaudio-libs-devel \
-        python3 \
-        python3-PyYAML \
-        python3-numpy \
-        python3-pillow \
-        python3-pip \
-        python3-sphinx \
-        python3-sphinx_rtd_theme \
+        python38 \
+        python38-PyYAML \
+        python38-numpy \
+        python38-pip \
+        python38-setuptools \
+        python38-wheel \
         rdma-core-devel \
         rpm \
         sed \
@@ -128,8 +126,14 @@  RUN dnf distro-sync -y && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
+RUN /usr/bin/pip3.8 install \
+                    meson==0.63.2 \
+                    pillow \
+                    sphinx \
+                    sphinx-rtd-theme
+
 ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
 ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV PYTHON "/usr/bin/python3.8"
diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
index 4b2c02d6abfb..5b8dbf2b83dd 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -89,16 +89,9 @@  RUN zypper update -y && \
            pam-devel \
            pcre-devel-static \
            pkgconfig \
-           python3-Pillow \
-           python3-PyYAML \
-           python3-Sphinx \
-           python3-base \
-           python3-numpy \
-           python3-opencv \
-           python3-pip \
-           python3-setuptools \
-           python3-sphinx_rtd_theme \
-           python3-wheel \
+           python39-base \
+           python39-pip \
+           python39-setuptools \
            rdma-core-devel \
            rpm \
            sed \
@@ -129,10 +122,15 @@  RUN zypper update -y && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-RUN /usr/bin/pip3 install meson==0.56.0
+RUN /usr/bin/pip3.9 install \
+                    PyYAML \
+                    meson==0.63.2 \
+                    pillow \
+                    sphinx \
+                    sphinx-rtd-theme
 
 ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
 ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV PYTHON "/usr/bin/python3.9"
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 13ab0b688726..5b27b89f1c72 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -138,7 +138,7 @@  RUN export DEBIAN_FRONTEND=noninteractive && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-RUN /usr/bin/pip3 install meson==0.56.0
+RUN /usr/bin/pip3 install meson==0.63.2
 
 ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
 ENV LANG "en_US.UTF-8"
diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml
new file mode 100644
index 000000000000..e4719e45516c
--- /dev/null
+++ b/tests/lcitool/mappings.yml
@@ -0,0 +1,77 @@ 
+mappings:
+  flake8:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  meson:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3:
+    CentOSStream8: python38
+    OpenSUSELeap153: python39-base
+
+  python3-PyYAML:
+    CentOSStream8: python38-PyYAML
+    OpenSUSELeap153:
+
+  python3-devel:
+    CentOSStream8: python38-devel
+    OpenSUSELeap153: python39-devel
+
+  python3-docutils:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-numpy:
+    CentOSStream8: python38-numpy
+    OpenSUSELeap153:
+
+  python3-opencv:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-pillow:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-pip:
+    CentOSStream8: python38-pip
+    OpenSUSELeap153: python39-pip
+
+  python3-pillow:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-selinux:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-setuptools:
+    CentOSStream8: python38-setuptools
+    OpenSUSELeap153: python39-setuptools
+
+  python3-sphinx:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-sphinx-rtd-theme:
+    CentOSStream8:
+    OpenSUSELeap153:
+
+  python3-venv:
+    CentOSStream8: python38
+    OpenSUSELeap153: python39-base
+
+  python3-wheel:
+    CentOSStream8: python38-wheel
+    OpenSUSELeap153: python39-pip
+
+pypi_mappings:
+  # Request more recent version
+  meson:
+    default: meson==0.63.2
+
+  # Drop packages that need devel headers
+  python3-numpy:
+    OpenSUSELeap153:
diff --git a/tests/lcitool/targets/centos-stream-8.yml b/tests/lcitool/targets/centos-stream-8.yml
new file mode 100644
index 000000000000..6b11160fd1dc
--- /dev/null
+++ b/tests/lcitool/targets/centos-stream-8.yml
@@ -0,0 +1,3 @@ 
+paths:
+  pip3: /usr/bin/pip3.8
+  python: /usr/bin/python3.8
diff --git a/tests/lcitool/targets/opensuse-leap-153.yml b/tests/lcitool/targets/opensuse-leap-153.yml
new file mode 100644
index 000000000000..683016e0077a
--- /dev/null
+++ b/tests/lcitool/targets/opensuse-leap-153.yml
@@ -0,0 +1,3 @@ 
+paths:
+  pip3: /usr/bin/pip3.9
+  python: /usr/bin/python3.9