aboutsummaryrefslogtreecommitdiffstats
path: root/lib/_emerge/depgraph.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* _dynamic_deps_preload: Fix settings referenceZac Medico2024-02-241-1/+1
| | | | | | | | | | Use the settings reference from the config pool as intended to prevent a KeyError similar to bug 924319 but triggered by emerge --dynamic-deps. Fixes: f9ea958018c0 ("MetadataRegen: Use EbuildMetadataPhase deallocate_config") Bug: https://bugs.gentoo.org/925350 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: Skip installed packages with emptytree in depgraph selectionGábor Oszkár Dénes2024-02-231-0/+13
| | | | | | | | | | | | | | Running emerge with emptytree tries to find the best match for every atom it needs to install. Sometimes the best matches would be already installed packages (with `operation=nomerge`), but these packages would be silently skipped with full emptytree installation. This change makes sure that emerge attempts to install every package. If the package has unmet requirements, emerge will complain. Bug: https://bugs.gentoo.org/651018 Signed-off-by: Gábor Oszkár Dénes <gaboroszkar@protonmail.com> Closes: https://github.com/gentoo/portage/pull/1272 Signed-off-by: Sam James <sam@gentoo.org>
* MetadataRegen: Use EbuildMetadataPhase deallocate_configZac Medico2024-02-211-0/+11
| | | | | | | | | | | For EbuildMetadataPhase consumers like MetadataRegen and depgraph, store a pool of config instances in a config_pool list, and return instaces to the list when the deallocate_config future is done. Bug: https://bugs.gentoo.org/924319 Fixes: c95fc64abf96 ("EbuildPhase: async_check_locale") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* */*: rerun black w/ 24.1.0Zac Medico2024-01-281-26/+33
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: Fix SyntaxWarning: 'str' object is not callableZac Medico2024-01-171-2/+4
| | | | | Fixes: a5078e577493 ("emerge: backtrack consistently regardless of --fetchonly") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: backtrack consistently regardless of --fetchonlyZac Medico2024-01-161-5/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | Make the _accept_blocker_conflicts method always return True when backtracking is enabled, so that backtracking results will be identical regardless of options that _accept_blocker_conflicts treats specially. This way, conflicts will only be accepted when backtracking is disabled or all backtracking tries have been exhausted. Make --nodeps imply --backtrack=0, since backtracking is only useful with dependencies. Make _eliminate_rebuilds return early if there are slot conflicts, since the merge list is invalid anyway, and its possible that state inconsistencies could trigger unexpected exceptions as in bug 922038. Make the KeyError from bug 922038 a warning, and include relevant information to help trace the inconsistency back to the _eliminate_rebuilds method or some other source like _solve_non_slot_operator_slot_conflicts. Bug: https://bugs.gentoo.org/161422 Bug: https://bugs.gentoo.org/607252 Bug: https://bugs.gentoo.org/675748 Bug: https://bugs.gentoo.org/922038 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _calc_depclean: add dep_check actionZac Medico2024-01-081-1/+6
| | | | | | | | | | | | | | | | | | | | | Add a dep_check action which can be used to check the dependencies of all installed packages. The plan is for depgraph to use this action to check for broken dependencies prior to the merge order calculation. The new frozen_config parameter will allow depgraph to pass a shared frozen config to _calc_depclean. The result of the dep_check action becomes stale as soon as there is any change to the installed packages. So, in order to account for dependencies that may become broken or satisfied during the process of updating installed packages, the merge order calculation will need to refresh the dep_check calculation for every merge order choice that it makes. This refresh will need to be optimized to identify the portion of the graph that would become stale due to a given change, so that it can avoid unnecessary repetition of work. Bug: https://bugs.gentoo.org/921333 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib/_emerge: codespell fixesSam James2024-01-051-1/+1
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* emerge: enable "avoid spamming too much info about unused binpkgs" againEli Schwartz2023-12-271-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In commit a6853d5493b7bed37e60c2e3b7536b209500ba3f this code in _show_ignored_binaries_respect_use was refactored to handle pkg.root. But it was refactored incorrectly -- the storage/lookup key started off as: ``` seen[pkg.cpv] ``` and was refactored so the storage key was: ``` seen[pkg.root][pkg.cpv] ``` and the lookup key was ``` seen[(pkg.root, pkg.cpv)] ``` As a result, we never detected a previously seen USE flags combo, and the logic to avoid spamming too much info was a no-op; the info was spammed, instead. Note that even though we have more code than before this patch, we actually have less code. The black code formatter decided that since the line length was decreased, this entire code block should be reformatted, murdering the diff view and dividing less code across *more* lines. This is not my fault and I refuse to be held accountable for it -- if you git blame this and do not know what happened, understand that it happens despite my objections. Signed-off-by: Eli Schwartz <eschwartz93@gmail.com> Closes: https://github.com/gentoo/portage/pull/1219 Signed-off-by: Sam James <sam@gentoo.org>
* depclean: Strengthen IDEPEND in unmerge orderZac Medico2023-12-261-1/+3
| | | | | | | | | | Increase priority of IDEPEND so that it is stronger than RDEPEND in unmerge order calculations. This causes IDEPEND to be unmerged afterwards when packages are involved in RDEPEND cycles. Bug: https://bugs.gentoo.org/916135 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: Use strip_libc_deps in _eliminate_rebuildsZac Medico2023-12-181-12/+21
| | | | | | | | | | The included test case will fail without strip_libc_deps because the "injected" >=sys-libs/glibc-2.37 dependency will be interpreted as a dependency change, triggering unwanted reinstall of app-misc/A-1. Bug: https://bugs.gentoo.org/915494 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ebuild: inject implicit libc RDEPENDSam James2023-12-101-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Inject >=${LIBC_PROVIDER}-${VERSION_OF_LIBC_PROVIDER} into RDEPEND. We already try to upgrade the virtual/libc provider and its deps aggressively but that's not so helpful if there's a binpkg for, say, one of its deps available built against a newer glibc - which ends up breaking the system because our glibc isn't new enough (symbol versioning). This is a long-standing problem for binpkg users and one of the last showstoppers for being able to use them reliably. Note that this applies to source installs too, although it matters less there; it'll make downgrading libc a bit harder for people who want to do that, but users are already prevented from doing that (pkg_* check) for glibc, and I don't really see it as a bad thing to effectively propagate this to other libcs. To fully solve the problem, we should arguably do at least one of the following: 1) Implicit >= on anything (not just libc) which installs ELF (at least a PROVIDES); 2) Implement the suggestion in bug #753500 based on analysis of used versioned symbols (*). But libc is really the critical one and the one where things explode pretty badly, especially combined with us trying to Do The Right Thing for non-binpkg cases (aggressively upgrading libc before anything else). The other cases don't matter so much as they get upgraded later and by that point, the library is usually done. (It's not really clear if downgrading musl works and even if it is supported, I'm not really sure it's a supported case at all, so I'm not bothering to carve out an exception here. It'd make this far less elegant and I don't see any benefit to doing so.) (*) util-linux is one of the examples I have in mind here as justification for either point. Bug: https://bugs.gentoo.org/753500 Bug: https://bugs.gentoo.org/913628 Signed-off-by: Sam James <sam@gentoo.org>
* DepPriority{Normal,Satisfied}Range: weaken _ignore_runtime for cross rootZac Medico2023-12-061-10/+36
| | | | | | | | | | | | | | | | | | | | | When the dependency parent is for a cross root (ROOT != /) package, weaken _ignore_runtime in order to tolerate runtime cycles that are less problematic for cross root packages. The included test case fails with this error without the fix: * Error: circular dependencies: (dev-libs/gmp-6.3.0:0/10.4::test_repo, binary scheduled for merge to '/tmp/tmp25nwdjn7/cross_root/') depends on (sys-devel/gcc-13.2.1_p20230826:0/0::test_repo, binary scheduled for merge to '/tmp/tmp25nwdjn7/cross_root/') (runtime) (dev-libs/gmp-6.3.0:0/10.4::test_repo, binary scheduled for merge to '/tmp/tmp25nwdjn7/cross_root/') (runtime_slot_op) It might be possible to break this cycle by applying the following change: - dev-libs/gmp-6.3.0 (Change USE: -cxx) Bug: https://bugs.gentoo.org/919174 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: avoid spamming too much info about unused binpkgs, reduxEli Schwartz2023-12-041-6/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Specifically: - many packages can be participating in the recursive deptree, but not get selected for actual merging, and may not even be installed at all; these were still reported as changed-deps although the packages themselves were not shown in show_merge_list Previously done for mismatched-USE in commit: bb82666b48e18f448661a1a8bf6a39b773cc4b1c Before: ``` Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 17.62 s. [binary U ] dev-python/ensurepip-pip-23.3.1-1 [23.2.1] [binary U ] sys-devel/binutils-2.40-r9-1 [2.40-r5] USE="-debuginfod%" [binary U ] dev-python/trove-classifiers-2023.11.14-1 [2023.10.18] [ebuild U ] dev-python/traitlets-5.13.0 [5.11.2] PYTHON_TARGETS="(-pypy3)" [ebuild U ] dev-python/pip-23.3.1 [23.2.1] [ebuild U ] dev-python/pycurl-7.45.2-r1 [7.45.2] [ebuild U ] dev-python/cachetools-5.3.2 [5.3.1] [ebuild U ] dev-python/pytest-7.4.3 [7.4.2] [ebuild U ] dev-python/executing-2.0.1 [1.2.0] PYTHON_TARGETS="(-pypy3) -python3_12%" [ebuild U ] dev-python/virtualenv-20.24.7 [20.24.5] !!! The following binary packages have been ignored due to changed dependencies: kde-plasma/kde-cli-tools-5.27.9::gentoo kde-plasma/kde-cli-tools-5.27.8::gentoo NOTE: The --binpkg-changed-deps=n option will prevent emerge from ignoring these binary packages if possible. Using --binpkg-changed-deps=y will silence this warning. ``` After: ``` Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 17.62 s. [binary U ] dev-python/ensurepip-pip-23.3.1-1 [23.2.1] [binary U ] sys-devel/binutils-2.40-r9-1 [2.40-r5] USE="-debuginfod%" [binary U ] dev-python/trove-classifiers-2023.11.14-1 [2023.10.18] [ebuild U ] dev-python/traitlets-5.13.0 [5.11.2] PYTHON_TARGETS="(-pypy3)" [ebuild U ] dev-python/pip-23.3.1 [23.2.1] [ebuild U ] dev-python/pycurl-7.45.2-r1 [7.45.2] [ebuild U ] dev-python/cachetools-5.3.2 [5.3.1] [ebuild U ] dev-python/pytest-7.4.3 [7.4.2] [ebuild U ] dev-python/executing-2.0.1 [1.2.0] PYTHON_TARGETS="(-pypy3) -python3_12%" [ebuild U ] dev-python/virtualenv-20.24.7 [20.24.5] ``` Signed-off-by: Eli Schwartz <eschwartz93@gmail.com> Closes: https://github.com/gentoo/portage/pull/1194 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: Handle empty leaves during runtime cycle topological sortZac Medico2023-11-291-2/+2
| | | | | Fixes: 31832c7faf5b ("Optimize runtime cycle ignore_priority leaf selection loop for topological sort") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Optimize runtime cycle ignore_priority leaf selection loop for topological sortZac Medico2023-11-291-19/+12
| | | | | | | | | | | | | Since increasing ignore_priority can only lead to a larger selection of leaf nodes, there is no need to increase ignore_priority to search for smaller groups of leaf nodes. It was the "only harvest one node at a time" part of commit 3487594cd8f4 that caused the test case to succeed. Fixes: 3487594cd8f4 ("Increase ignore_priority during topological sort for runtime cycle") Bug: https://bugs.gentoo.org/917259 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Prefer installed leaves in runtime cycle topological sortZac Medico2023-11-281-0/+6
| | | | | | | | | | In order to avoid possibly merging a package too early, prefer installed leaves in runtime cycle topological sort. This fixes an AlternativesGzipTestCase failure that arose after 2e298ea7ba36 caused leaves to be selected in a slightly different order. Bug: https://bugs.gentoo.org/917259 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Increase ignore_priority during topological sort for runtime cycleZac Medico2023-11-281-7/+20
| | | | | | | | | | | | Fix AlternativesGzipTestCase by increasing ignore_priority in order to find smaller groups of leaf nodes during topological sort for runtime cycles. This causes some changes in merge order for MergeOrderTestCase, but they appear to be acceptable since they prevent temporarily unsatisfied RDEPEND by relying on satisfied installed build-time dependencies. Bug: https://bugs.gentoo.org/917259 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* find_smallest_cycle: Optimize to traverse fewer nodesZac Medico2023-11-271-3/+33
| | | | | | | | | | | | | | If gather_deps is traversing a cycle that is greater than or equal to the size of the current smallest_cycle, then abort early. Also abort early if we traverse a node encountered in a previous cycle for the same ignore_priority, since that means the two cycles are identical. On my laptop, this brings the emerge -pe @world time down to 3m28.884s, compared to 10m44.268s with portage-3.0.55. Bug: https://bugs.gentoo.org/918682 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "find_smallest_cycle: Increase ignore_priority to find smaller cycles"Zac Medico2023-11-221-4/+3
| | | | | | | | | | This reverts commit 9206d5a75ecd2d9ae0fe63e57d28aa8061b5927e. The len(smallest_cycle) == 1 loop termination condition is not optimal and it's too expensive as reported in bug 917660. Bug: https://bugs.gentoo.org/917660 Bug: https://bugs.gentoo.org/917259 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* find_smallest_cycle: Increase ignore_priority to find smaller cyclesZac Medico2023-11-181-3/+4
| | | | | | | | | | | | | | | Fix AlternativesGzipTestCase by increasing ignore_priority in order to find smaller cycles. This causes some changes in merge order for MergeOrderTestCase, but they appear to be acceptable since they prevent temporarily unsatisfied RDEPEND by relying on satisfied installed build-time dependencies. Add a test case for bug 690436, since the change to merge order in MergeOrderTestCase is related (see commit 680276cc4d4f). Bug: https://bugs.gentoo.org/690436 Bug: https://bugs.gentoo.org/917259 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: fix _show_ignored_binaries_respect_use with incomplete depgraphSam James2023-11-061-1/+3
| | | | | | | | | I've gone for the simpler solution of just using an empty tuple where the merge list is empty to preserve prior behaviour with what we do (or do not) display wrt skipped binaries. Bug: https://bugs.gentoo.org/916614 Signed-off-by: Sam James <sam@gentoo.org>
* emerge: account for EROOT in _show_ignored_binaries_respect_useSam James2023-11-061-4/+9
| | | | | | | Zac suggested this when reviewing the fix for bug #916336. Bug: https://bugs.gentoo.org/916336 Signed-off-by: Sam James <sam@gentoo.org>
* emerge: fix binpkg-respect-use notice with blockersSam James2023-11-061-1/+5
| | | | | | | | Items in _dynamic_config._displayed_list might be blockers, so filter those out. Bug: https://bugs.gentoo.org/916336 Fixes: bb82666b48e18f448661a1a8bf6a39b773cc4b1c Signed-off-by: Sam James <sam@gentoo.org>
* emerge: avoid spamming too much info about unused binpkgsEli Schwartz2023-10-221-11/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Specifically: - the binhost might have multiple binpkgs for the same cpv, same USE flags, but with different build ids, in which case the same line is repeated twice - many packages can be participating in the recursive deptree, but not get selected for actual merging, and may not even be installed at all; these were still reported as mismatched-USE although the packages themselves were not shown in show_merge_list Both cases are useless noise. Prune them before printing a status report. Before: ``` Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 17.29 s. [binary U ] sys-apps/hwdata-0.374-1 [0.373] [binary r U ] dev-libs/libsodium-1.0.19-r1-1 [1.0.18_p20220618] [binary U ] net-dns/libidn2-2.3.4-r1-1 [2.3.4] [ebuild R ] dev-python/fs-2.4.16 PYTHON_TARGETS="(-python3_12%*)" [ebuild U ] sys-apps/iproute2-6.5.0 [6.4.0] [binary U ] media-video/ffmpeg-6.0-r9-2 [6.0-r6] [ebuild rR ] app-editors/vim-9.0.1627 The following packages are causing rebuilds: (dev-libs/libsodium-1.0.19-r1-1:0/26::gentoo, binary scheduled for merge) causes rebuilds for: (app-editors/vim-9.0.1627:0/0::gentoo, ebuild scheduled for merge) !!! The following binary packages have been ignored due to non matching USE: =app-shells/bash-5.1_p16-r6 -plugins =sys-apps/util-linux-2.38.1-r2 -caps split-usr =sys-apps/util-linux-2.38.1-r2 -caps split-usr =sys-process/procps-3.3.17-r2 split-usr =sys-process/procps-3.3.17-r2 split-usr -systemd =app-editors/vim-9.0.1627 -python =app-editors/vim-9.0.1627 -python =dev-lang/python-3.11.5 -lto -pgo =dev-lang/python-3.11.5 -lto -pgo =dev-lang/python-3.11.5 -lto -pgo =media-fonts/noto-20230831 X =net-misc/networkmanager-1.42.6-r2 bluetooth gtk-doc ppp =net-misc/networkmanager-1.42.6-r2 bluetooth gtk-doc ppp =net-misc/networkmanager-1.42.6-r2 bluetooth gtk-doc ppp =net-misc/networkmanager-1.42.6-r2 bluetooth gtk-doc ppp =sys-devel/gdb-13.2-r1 -lzma nls -source-highlight -xxhash -zstd =x11-base/xorg-server-21.1.8-r2 -xvfb =media-libs/libsdl2-2.28.3 gles2 pulseaudio wayland =media-libs/libsdl2-2.28.3 gles2 pipewire pulseaudio wayland =sys-devel/clang-16.0.6 abi_x86_32 =sys-devel/clang-16.0.6 -xml =app-arch/zstd-1.5.5 split-usr =app-arch/zstd-1.5.5 split-usr =sys-apps/sandbox-2.38 -abi_x86_32 =dev-python/certifi-3021.3.16-r3 -python_targets_python3_12 =dev-python/certifi-3021.3.16-r3 -python_targets_python3_12 =dev-libs/libffi-3.4.4-r1 abi_x86_32 =sys-libs/ncurses-6.4_p20230401 abi_x86_32 -gpm split-usr =sys-libs/readline-8.1_p2-r1 split-usr =dev-db/sqlite-3.43.2 -icu =dev-libs/icu-73.2 abi_x86_32 =sys-libs/gpm-1.20.7-r5 abi_x86_32 split-usr =app-crypt/pinentry-1.2.1-r3 keyring qt5 wayland =dev-libs/glib-2.76.4 sysprof =sys-apps/systemd-253.11 split-usr =dev-libs/libxml2-2.11.5 abi_x86_32 =dev-libs/libxml2-2.11.5 -icu =media-libs/mesa-23.1.8 video_cards_intel video_cards_nouveau =media-libs/mesa-23.1.8 video_cards_intel video_cards_nouveau =media-libs/mesa-23.1.8 video_cards_intel video_cards_nouveau =media-libs/mesa-23.1.8 video_cards_intel video_cards_nouveau =media-libs/mesa-23.1.8 video_cards_intel video_cards_nouveau =x11-libs/libxkbcommon-1.5.0 wayland =x11-libs/libxkbcommon-1.5.0 wayland =x11-libs/libxkbcommon-1.5.0 wayland =sys-devel/llvm-16.0.6 abi_x86_32 =sys-devel/llvm-16.0.6 -xml =x11-misc/xdg-utils-1.1.3_p20210805-r1 gnome =dev-libs/libpcre2-10.42-r1 split-usr =sys-auth/polkit-122-r1 kde =net-libs/nghttp2-1.57.0 -systemd -xml =sys-libs/glibc-2.37-r7 -multilib -stack-realign -systemd =app-crypt/gpgme-1.21.0 -qt5 =sys-devel/clang-runtime-16.0.6 -abi_x86_32 =media-sound/mpg123-1.31.3-r1 pulseaudio =dev-libs/libpcre-8.45-r2 split-usr =net-wireless/wpa_supplicant-2.10-r3 qt5 =media-libs/gst-plugins-base-1.20.6 wayland =media-libs/gst-plugins-base-1.20.6 theora wayland =media-libs/gst-plugins-base-1.20.6 wayland =net-libs/libproxy-0.5.3 kde =gnome-base/gvfs-1.50.6 gnome-online-accounts keyring =gnome-base/gvfs-1.50.6 gnome-online-accounts keyring =sys-libs/libblockdev-2.28-r1 introspection =dev-libs/libdbusmenu-16.04.0-r2 gtk =dev-libs/libdbusmenu-16.04.0-r2 gtk =dev-python/css-parser-1.0.9 -python_targets_python3_12 =dev-qt/qtbase-6.5.2-r2 nls wayland =dev-qt/qtwebengine-6.5.2-r1 pulseaudio qml screencast =dev-qt/qtwebengine-6.5.2-r1 pulseaudio qml screencast =dev-qt/qtwebchannel-6.5.2-r2 qml =app-arch/unzip-6.0_p27-r1 -natspec =app-text/liblangtag-0.6.4 introspection =app-text/liblangtag-0.6.4 introspection =sys-apps/groff-1.22.4 -X =virtual/tmpfiles-0-r5 -systemd =sys-apps/systemd-utils-253.11-r1 split-usr =virtual/libcrypt-2-r1 -abi_x86_32 =virtual/udev-217-r7 -systemd =net-misc/openssh-9.4_p1-r1 -X =dev-libs/libusb-1.0.26 split-usr =media-video/pipewire-0.3.80 bluetooth sound-server =media-video/pipewire-0.3.80 bluetooth sound-server =x11-misc/mate-notification-daemon-1.26.0-r1 wayland =x11-misc/mate-notification-daemon-1.26.0-r1 wayland =x11-misc/mate-notification-daemon-1.26.0-r1 wayland =kde-plasma/plasma-workspace-5.27.8-r1 screencast =kde-plasma/plasma-workspace-5.27.8-r1 screencast =kde-plasma/plasma-workspace-5.27.8-r1 screencast =sys-devel/llvm-15.0.7-r3 abi_x86_32 NOTE: The --binpkg-respect-use=n option will prevent emerge from ignoring these binary packages if possible. Using --binpkg-respect-use=y will silence this warning. Would you like to merge these packages? [Yes/No] ``` After: ``` Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 17.25 s. [binary U ] sys-apps/hwdata-0.374-1 [0.373] [binary r U ] dev-libs/libsodium-1.0.19-r1-1 [1.0.18_p20220618] [binary U ] net-dns/libidn2-2.3.4-r1-1 [2.3.4] [ebuild R ] dev-python/fs-2.4.16 PYTHON_TARGETS="(-python3_12%*)" [ebuild U ] sys-apps/iproute2-6.5.0 [6.4.0] [binary U ] media-video/ffmpeg-6.0-r9-2 [6.0-r6] [ebuild rR ] app-editors/vim-9.0.1627 The following packages are causing rebuilds: (dev-libs/libsodium-1.0.19-r1-1:0/26::gentoo, binary scheduled for merge) causes rebuilds for: (app-editors/vim-9.0.1627:0/0::gentoo, ebuild scheduled for merge) !!! The following binary packages have been ignored due to non matching USE: =sys-apps/iproute2-6.5.0 split-usr =app-editors/vim-9.0.1627 -python NOTE: The --binpkg-respect-use=n option will prevent emerge from ignoring these binary packages if possible. Using --binpkg-respect-use=y will silence this warning. Would you like to merge these packages? [Yes/No] ``` Signed-off-by: Eli Schwartz <eschwartz93@gmail.com> Closes: https://github.com/gentoo/portage/pull/1148 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: Skip _eliminate_rebuilds for --emptytreeZac Medico2023-10-141-0/+3
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Detect and handle unnecessary package reinstallZac Medico2023-10-121-30/+176
| | | | | | | | | | | | | | | | | Compare package rebuilds/reinstalls to installed packages of the same exact version, and eliminate unecessary rebuilds/reinstalls triggered solely by the @__auto_slot_operator_replace_installed__ set. This is careful to obey the user's wishes if they have explicitly requested for a package to be rebuilt or reinstalled for some reason. The SonameSkipUpdateTestCase::testSonameSkipUpdateNoPruneRebuilds test case shows that the new depgraph._eliminate_rebuilds method eliminates a backtracking run that is needed for the testSonameSkipUpdate test case to succeed via prune_rebuilds backtracking which was added for bug 439688. Bug: https://bugs.gentoo.org/915494 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: Increase default number of maximum backtrack attempts from 10 to 20Florian Schmaus2023-10-121-1/+1
| | | | | | | | | Experience shows that 10 is too low and that in some cases a slighlty higher number results in a successful dependency calculation. Signed-off-by: Florian Schmaus <flow@gentoo.org> Closes: https://github.com/gentoo/portage/pull/1127 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: show backtrack information after dep resolutionFlorian Schmaus2023-10-021-6/+15
| | | | | | | | | To improve user's awareness of the backtracking performed by portage and the related limit, show briefly how much backtracking was done. Signed-off-by: Florian Schmaus <flow@gentoo.org> Closes: https://github.com/gentoo/portage/pull/1107 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: increase reverse dep cache sizeDaniel Harding2023-09-211-1/+1
| | | | | | | | | | | To avoid blowing out the cache and destroying performance, increase the maximum size of the memoization cache for the depgraph _slot_opererator_check_reverse_dependencies() method from 100 to 1000. Bug: https://bugs.gentoo.org/883071 Signed-off-by: Daniel Harding <dharding@living180.net> Closes: https://github.com/gentoo/portage/pull/1092 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: improve reverse dep cachingDaniel Harding2023-09-211-1/+8
| | | | | | | | | | | | | | | Avoid applying @functools.lru_cache() to the depgraph._slot_operator_check_reverse_depencencies() method, as applying the lru_cache decorator to a method can prevent the class instances from being garbage collected due to self being stored in the cache as part of the args key[1]. Instead, set up a per-instance cache in the __init__() method. [1] https://stackoverflow.com/q/33672412/3573385 Bug: https://bugs.gentoo.org/883071 Signed-off-by: Daniel Harding <dharding@living180.net> Signed-off-by: Sam James <sam@gentoo.org>
* Run `pyupgrade --py39-plus`Sam James2023-07-261-13/+13
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* depgraph.py: fix wrong type annotationsBerin Aniesh2023-07-111-7/+11
| | | | | | | | Fixes: 7e8a730b85b77f9495f4999ac173dc726777203f Bug: https://bugs.gentoo.org/910171 Signed-off-by: Berin Aniesh <berinaniesh@gmail.com> Closes: https://github.com/gentoo/portage/pull/1067 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph.py: fix "no ebuilds/binpkgs" messageOskari Pirhonen2023-07-091-1/+1
| | | | | | | | | | | | | | | | | | | The "there are no binary packages to satisfy" was being unconditionally output for packages that could not be found. Fix the logic for choosing between the "binary packages" and "ebuilds" form of the message. This is a temporary stopgap as alluded to by me in the bug, but the tl;dr is that some entries in the `myopts` dict have "y"/"n" values whereas some are True/unset, and this discrepancy should be sorted out. [sam: Add NEWS and Fixes, although the change in that commit _shouldn't_ have been wrong, it is because of a quirk for now...] Bug: https://bugs.gentoo.org/909853 Signed-off-by: Oskari Pirhonen <xxc3ncoredxx@gmail.com> Closes: https://github.com/gentoo/portage/pull/1065 Fixes: 0b21a5a392bd84c07b94373991f59108fbe98516 Signed-off-by: Sam James <sam@gentoo.org>
* Revert "depgraph: Don't ignore downgrades as missed_updates"Sam James2023-06-211-1/+3
| | | | | | | | | This reverts commit 44afa8445dc46464200fe46c1e09e0c7475067bf. This is just so we can make a release for the other fixes in master like the Perl bits and keep a linear history. Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: Do not allow slotted deps to be satisfied by wrong slotsYiFei Zhu2023-06-161-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This may be part of what caused the "perl rebuild bug". The "priority" of perl, when seen by perl modules, may have "satisfied" set to an installed perl of a wrong slot. Compounding this factor with bug #756199 where find_smallest_cycle would select a single node, in this case because the dependency of perl is satisfied and the priority then gets ignored, the "cycle" becomes a perl module alone and gets rebuilt early. I also updated the test from the previous patch to account for this change. No other tests seems affected. For a larger scale test, I reproduced this initially with a stage3 chroot from a Jan 1 2022 stage3 snapshot, and testing in an equivalent dockerfile would work too: FROM gentoo/stage3:amd64-openrc-20220101 RUN emerge-webrsync COPY . /portage Before this patch (USE flags omitted): # cd /portage && bin/emerge -puDN @world 2>&1 | grep -i perl [ebuild U ] app-admin/perl-cleaner-2.30-r1 [2.30] [ebuild rR ] virtual/perl-File-Temp-0.231.100 [ebuild rR ] dev-perl/Locale-gettext-1.70.0-r1 [ebuild rR ] dev-perl/MIME-Charset-1.12.2-r1 [ebuild rR ] dev-perl/Module-Build-0.423.100 [ebuild rR ] dev-perl/Text-CharWidth-0.40.0-r2 [ebuild U ] dev-lang/perl-5.36.0-r2 [5.34.0-r3] [ebuild N ] virtual/perl-CPAN-2.330.0 [ebuild U ] virtual/perl-ExtUtils-MakeMaker-7.640.0 [7.620.0] [ebuild U ] virtual/perl-File-Spec-3.840.0 [3.800.0] [...] After this patch: # cd /portage && bin/emerge -puDN @world 2>&1 | grep -i perl [ebuild U ] app-admin/perl-cleaner-2.30-r1 [2.30] [ebuild U ] dev-lang/perl-5.36.0-r2:0/5.36 [5.34.0-r3:0/5.34] [ebuild N ] virtual/perl-CPAN-2.330.0 [ebuild U ] virtual/perl-ExtUtils-MakeMaker-7.640.0 [7.620.0] [ebuild U ] virtual/perl-Data-Dumper-2.184.0 [2.179.0] [ebuild U ] virtual/perl-File-Spec-3.840.0 [3.800.0] [ebuild U ] virtual/perl-Test-Harness-3.440.0-r1 [3.430.0] [ebuild rR ] dev-perl/Pod-Parser-1.630.0-r1 [ebuild rR ] dev-perl/Text-CharWidth-0.40.0-r2 [ebuild rR ] dev-perl/Text-WrapI18N-0.60.0-r2 [...] Bug: https://bugs.gentoo.org/463976 Bug: https://bugs.gentoo.org/592880 Bug: https://bugs.gentoo.org/596664 Bug: https://bugs.gentoo.org/631490 Bug: https://bugs.gentoo.org/764365 Bug: https://bugs.gentoo.org/793992 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Closes: https://github.com/gentoo/portage/pull/1055 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: Don't ignore downgrades as missed_updatesYiFei Zhu2023-06-161-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Missed updates can also come in the form of package downgrades, when, for example, there are keyword changes. They can cause rebuilds, and these rebuilds may be not possible due to reasons such as keywords or masks. In this case, prior to this patch, portage would cancel the downgrade, but the rebuilds would be requested endlessly, because bug 439688's backtrack code does not trigger. To reproduce, on an ACCEPT_KEYWORDS=~amd64 machine, emerge =dev-libs/openssl=3.0.9, dev-util/rustup, and something else that depends on openssl. Then un-accept ~amd64 for openssl and rustup. Prior to this patch, a @world upgrade would cause: These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild rR ] dev-libs/libevent-2.1.12-r1:0/2.1-7::gentoo [ebuild rR ] net-misc/rsync-3.2.7-r2::gentoo [...] Total: 71 packages (71 reinstalls), Size of downloads: 0 KiB There are no packages marked "R", only "rR". There are no section labeled "The following packages are causing rebuilds:" either. After this patch, we have: These are the packages that would be merged, in order: Calculating dependencies... done! Total: 0 packages, Size of downloads: 0 KiB WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict: dev-libs/openssl:0 (dev-libs/openssl-1.1.1u:0/1.1::gentoo, ebuild scheduled for merge) dev-libs/openssl:0/3= required by (dev-util/rustup-1.25.2:0/0::gentoo, installed) I also updated the test from the previous patch to account for this change. No other tests seems affected. Bug: https://bugs.gentoo.org/439688 Bug: https://bugs.gentoo.org/622270 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Closes: https://github.com/gentoo/portage/pull/1053 Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: drop unnecessary spinner importSam James2023-06-141-5/+4
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: type annotationsBerin Aniesh2023-06-141-4/+39
| | | | | | Signed-off-by: Berin Aniesh <berinaniesh@gmail.com> Closes: https://github.com/gentoo/portage/pull/1054 Signed-off-by: Sam James <sam@gentoo.org>
* emerge: depgraph: fix comment typoSam James2023-06-141-1/+1
| | | | | | Reported by Arfrever at https://github.com/gentoo/portage/commit/4c25c0d7af7ad71fccbfafe1e5019116c691968e#r45851837. Signed-off-by: Sam James <sam@gentoo.org>
* Convert %-formats to fstringsOskari Pirhonen2023-05-011-150/+86
| | | | | | | | Files under lib/_emerge Signed-off-by: Oskari Pirhonen <xxc3ncoredxx@gmail.com> Closes: https://github.com/gentoo/portage/pull/1013 Signed-off-by: Sam James <sam@gentoo.org>
* emerge: add --update-if-installedSam James2023-02-211-1/+16
| | | | | | | | | | | | | | | This adds a new emerge option '--update-if-installed'. The use case for such an option is as follows: - User finds out libfoo-1.2 is buggy. - They want to upgrade all their systems if libfoo is installed. - They don't want to install libfoo if it's not already installed. Unfortunately, --update fails this last point, hence the need for a new option. Closes: https://github.com/gentoo/portage/pull/988 Signed-off-by: Sam James <sam@gentoo.org>
* emerge: add --onlydeps-with-ideps=<y|n> option (bug 890777)Tom Gillespie2023-02-171-1/+2
| | | | | | | | | | | | | | | | | | Add --onlydeps-with-ideps option in order to include install-time dependencies with --onlydeps and --onlydeps-with-rdeps=n. The dependencies that get pulled in are those that are necessary for emerge --nodeps to succeed when run after the equivalent --onlydeps. The default --onlydeps --onlydeps-with-rdeps=n behavior is unchanged. This also adds a new test file test_onlydeps_ideps.py that is derived from test_onlydeps_minimal.py and tests the behavior for EAPI={7,8}. Additional tests have been added to test_onlydeps_minimal.py to ensure that the behavior at EAPI=0 remains unchanged. Bug: https://bugs.gentoo.org/890777 Signed-off-by: Tom Gillespie <tgbugs@gmail.com> Closes: https://github.com/gentoo/portage/pull/979 Signed-off-by: Sam James <sam@gentoo.org>
* */*: rerun black w/ 23.1.0Sam James2023-02-031-70/+1
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* */*: run Black again (post-pyupgrade)Sam James2023-01-101-3/+1
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* */*: run pyupgrade --py37-plus againSam James2023-01-101-1/+1
| | | | | | After running flynt, it finds a few more bits. Signed-off-by: Sam James <sam@gentoo.org>
* */*: run Black post-flyntSam James2023-01-101-48/+16
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* */*: convert to f-strings (run flynt)Sam James2023-01-101-176/+164
| | | | Signed-off-by: Sam James <sam@gentoo.org>
* depgraph: Print dependency resolution timeMichał Górny2023-01-101-0/+6
| | | | | | | | | | Print the wall clock time it took emerge to calculate the dependency graph if spinner is used (i.e. we're in verbose mode). Requested-by: Maciej Barć <xgqt@gentoo.org> Signed-off-by: Michał Górny <mgorny@gentoo.org> Closes: https://github.com/gentoo/portage/pull/972 Signed-off-by: Sam James <sam@gentoo.org>
* emerge: depgraph: Correctly detect installation from path for binpkgsSheng Yu2023-01-101-3/+11
| | | | | | | | Bug: https://bugs.gentoo.org/873202 Bug: https://bugs.gentoo.org/890291 Signed-off-by: Sheng Yu <syu.os@protonmail.com> Closes: https://github.com/gentoo/portage/pull/973 Signed-off-by: Sam James <sam@gentoo.org>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.