diff mbox series

[2/2] tests: update avocado installation to use mkvenv

Message ID 20230605075823.48871-3-pbonzini@redhat.com
State New
Headers show
Series Fix venv issues with Avocado by reverting to an older version | expand

Commit Message

Paolo Bonzini June 5, 2023, 7:58 a.m. UTC 
The recent changes to Python handling mean that there is a single
virtual environment for all the build, instead of a separate one for testing.
Because this virtual environment will often have system site packages
available, it makes sense to use mkvenv.py to install avocado, which will
avoid using PyPI if a new-enough version is available in the system.

However, requiring a specific version of avocado will cause conflicts with
any avocado plugins installed on the host:

   $ make check-venv
   make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
     GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
     VENVPIP install -e /home/berrange/src/virt/qemu/python/
     VENVPIP install -r /home/berrange/src/virt/qemu/tests/requirements.txt
   ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
   make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'

so the requirements should use a ">=" constraint and the versions of
Avocado and pycdlib should be limited to what distros provide
in the system packages.  Only Fedora has Avocado, and more specifically
version 92.0, while the following distros have pycdlib:

   CentOS Stream 8             1.11.0
   CentOS Stream 9             1.11.0
   Fedora 37                   1.13.0
   Fedora 38                   1.14.0
   Ubuntu 22.04                1.11.0
   Debian bookworm             1.12.0

So the current minimal versions specified by tests/requirements.txt are
okay.  Move them to the check-venv target and add the corresponding
constraints to python/tests/minreqs.txt as well.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 python/tests/minreqs.txt | 3 ++-
 tests/Makefile.include   | 8 ++++++--
 tests/requirements.txt   | 9 ---------
 3 files changed, 8 insertions(+), 12 deletions(-)
 delete mode 100644 tests/requirements.txt
diff mbox series

Patch

diff --git a/python/tests/minreqs.txt b/python/tests/minreqs.txt
index 1ce72cef6d8..6c9336ad119 100644
--- a/python/tests/minreqs.txt
+++ b/python/tests/minreqs.txt
@@ -23,7 +23,8 @@  distlib==0.3.6
 fusepy==2.0.4
 
 # Test-runners, utilities, etc.
-avocado-framework==90.0
+avocado-framework==88.1
+pycdlib==1.11.0
 
 # Linters
 flake8==5.0.4
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8294a44816c..82240d631fe 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -1,5 +1,8 @@ 
 # -*- Mode: makefile -*-
 
+AVOCADO_MIN_VERSION=88.1
+PYCDLIB_MIN_VERSION=1.11.0
+
 .PHONY: check-help
 check-help:
 	@echo "Regression testing targets:"
@@ -90,7 +93,7 @@  distclean-tcg: $(DISTCLEAN_TCG_TARGET_RULES)
 TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
 
 TESTS_VENV_TOKEN=$(BUILD_DIR)/pyvenv/tests.group
-TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
+TESTS_VENV_REQ=$(SRC_PATH)/python/tests/minreqs.txt
 TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 ifndef AVOCADO_TESTS
 	AVOCADO_TESTS=tests/avocado
@@ -112,7 +115,8 @@  quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
 
 $(TESTS_VENV_TOKEN): $(TESTS_VENV_REQ)
 	$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
-	$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
+	$(quiet-@)$(PYTHON) $(SRC_PATH)/python/scripts/mkvenv.py ensure --diagnose avocado --online \
+	      'avocado-framework>=$(AVOCADO_MIN_VERSION)' 'pycdlib>=$(PYCDLIB_MIN_VERSION)'
 	$(call quiet-command, touch $@)
 
 $(TESTS_RESULTS_DIR):
diff --git a/tests/requirements.txt b/tests/requirements.txt
deleted file mode 100644
index 07e713ef5ac..00000000000
--- a/tests/requirements.txt
+++ /dev/null
@@ -1,9 +0,0 @@ 
-# Add Python module requirements, one per line, to be installed
-# in the qemu build_dir/pyvenv Python virtual environment. For more info,
-# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-#
-# Note that qemu.git/python/ is implicitly installed to this venv when
-# 'make check-venv' is run, and will persist until configure is run
-# again.
-avocado-framework==88.1
-pycdlib==1.11.0