qemu/python/scripts, branch master

python: mkvenv: remove ensure command

2024-06-08T08:33:39Z

This was used to bootstrap the venv with a TOML parser, after which ensuregroup is used. Now that we expect it to be present as a system package (either tomli or, for Python 3.11, tomllib), it is not needed anymore. Note that this means that, when implemented, the hypothetical "isolated" mode that does not use any system packages will only work with Python 3.11+. Signed-off-by: Paolo Bonzini

Revert "python: use vendored tomli"

2024-06-08T08:33:39Z

Now that Ubuntu 20.04 is not included anymore, there is no need to ship it as part of QEMU; Ubuntu 22.04 includes it and Leap users anyway need to install all the required dependencies from PyPI. This mostly reverts commit ec77ee7634de123b7c899739711000fd21dab68b, with just some changes to the wording. Signed-off-by: Paolo Bonzini

buildsys: Bump known good meson version to v1.2.3

2023-11-24T15:21:55Z

We need meson v1.2.3 to build QEMU on macOS Sonoma. It also builds fine all our CI jobs (as tested by also bumping "accepted" in pythondeps.toml), so let's use it as our "good enough" packaged wheel. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1939 Suggested-by: Paolo Bonzini Signed-off-by: Philippe Mathieu-Daudé Message-ID: <20231109160504.93677-2-philmd@linaro.org> Signed-off-by: Paolo Bonzini

Revert "mkvenv: work around broken pip installations on Debian 10"

2023-09-07T11:32:37Z

Debian 10 has Python 3.7, so it is not possible to use it anymore now that Python 3.8 is required. Signed-off-by: Paolo Bonzini

mkvenv: assume presence of importlib.metadata

2023-09-07T11:32:37Z

importlib.metadata is included in Python 3.8, so there is no need to fallback to either importlib-metadata or pkgresources when generating console script shims. Signed-off-by: Paolo Bonzini

Python: Drop support for Python 3.7

2023-09-07T11:32:37Z

Debian 10 is not anymore a supported distro, since Debian 12 was released on June 10, 2023. Our supported build platforms as of today all support at least 3.8 (and all of them except for Ubuntu 20.04 support 3.9): openSUSE Leap 15.5: 3.6.15 (3.11.2) CentOS Stream 8: 3.6.8 (3.8.13, 3.9.16, 3.11.4) CentOS Stream 9: 3.9.17 (3.11.4) Fedora 37: 3.11.4 Fedora 38: 3.11.4 Debian 11: 3.9.2 Debian 12: 3.11.2 Alpine 3.14, 3.15: 3.9.16 Alpine 3.16, 3.17: 3.10.10 Ubuntu 20.04 LTS: 3.8.10 Ubuntu 22.04 LTS: 3.10.12 NetBSD 9.3: 3.9.13* FreeBSD 12.4: 3.9.16 FreeBSD 13.1: 3.9.18 OpenBSD 7.2: 3.9.17 Note: NetBSD does not appear to have a default meta-package, but offers several options, the lowest of which is 3.7.15. However, "python39" appears to be a pre-requisite to one of the other packages we request in tests/vm/netbsd. Since it is safe under our supported platform policy, bump our minimum supported version of Python to 3.8. The two most interesting features to have by default include: - the importlib.metadata module, whose lack is responsible for over 100 lines of code in mkvenv.py - improvements to asyncio, for example asyncio.CancelledError inherits from BaseException rather than Exception In addition, code can now use the assignment operator ':=' Because mypy now learns about importlib.metadata, a small change to mkvenv.py is needed to pass type checking. Signed-off-by: Paolo Bonzini

Revert "tests: Use separate virtual environment for avocado"

2023-08-28T07:55:48Z

This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9. ensuregroup allows to specify both the acceptable versions of avocado, and a locked version to be used when avocado is not installed as a system pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and reuse the distro package on Fedora and CentOS Stream (the only distros where it's available). ensuregroup's usage of "(>=..., <=...)" constraints when evaluating the distro package, and "==" constraints when installing it from PyPI, makes it possible to avoid conflicts between the known-good version and a package plugins included in the distro. This is because package plugins have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in an error: avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. But at the same time, if the venv does not include a system distribution of avocado then we can install a known-good version and stick to LTS releases. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663 Signed-off-by: Paolo Bonzini

python: use vendored tomli

2023-08-28T07:55:48Z

Debian only introduced tomli in the bookworm release. Use a vendored wheel to avoid requiring a package that is only in bullseye-backports and is also absent in Ubuntu 20.04. While at it, fix an issue in the vendor.py scripts which does not add a newline after each package and hash. Signed-off-by: Paolo Bonzini

python: mkvenv: add ensuregroup command

2023-08-28T07:55:36Z

Introduce a new subcommand that retrieves the packages to be installed from a TOML file. This allows being more flexible in using the system version of a package, while at the same time using a known-good version when installing the package. This is important for packages that sometimes have backwards-incompatible changes or that depend on specific versions of their dependencies. Compared to JSON, TOML is more human readable and easier to edit. A parser is available in 3.11 but also available as a small (12k) package for older versions, tomli. While tomli is bundled with pip, this is only true of recent versions of pip. Of all the supported OSes pretty much only FreeBSD has a recent enough version of pip while staying on Python <3.11. So we cannot use the same trick that is in place for distlib. Signed-off-by: Paolo Bonzini

python: mkvenv: introduce TOML-like representation of dependencies

2023-08-28T07:55:05Z

We would like to place all Python dependencies in the same file, so that we can add more information without having long and complex command lines. The plan is to have a TOML file with one entry per package, for example [avocado] avocado-framework = { accepted = "(>=88.1, <93.0)", installed = "88.1", canary = "avocado" } Each TOML section will thus be a dictionary of dictionaries. Modify mkvenv.py's workhorse function, _do_ensure, to already operate on such a data structure. The "ensure" subcommand is modified to separate the depspec into a name and a version part, and use the result (plus the --diagnose argument) to build a dictionary for each command line argument. Signed-off-by: Paolo Bonzini