Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=2.3.9 2021-08-29T12:23:26Z 2021-08-29T12:23:26Z David Kalnischkies david@kalnischkies.de 2021-08-29T11:50:31Z urn:sha1:5f6bbfa53c32ec30aff6a2bc8c412616049eab18 If you install dpkg on an empty status file with all recommends and suggests apt wants to install 4000+ packages. The deepest chain seemingly being 236 steps long. And dpkg isn't even the worst (~259). That is a problem as libapt has a hardcoded recursion limit for MarkInstall and friends … set to 100. We are saved by the fact that chains without suggests are much shorter (dpkg has 5, max seems ~43), but I ignored Conflicts in these chains, which typically trigger upgrades, so if two of the worst are chained together we suddenly get dangerously close to the limit still. So, lets just increase the limit into oblivion as it is really just a safety measure we should not be running into to begin with. MarkPackage was running years without it after all. 3000 is picked as a nice number as any other and because it is roughly the half of the stack crashs I saw previously in this branch. 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T13:55:09Z urn:sha1:c7e368aafe099dcd966cf5994ae7fb418d268278 fullyExplored is needed to keep track of having explored all providers of a package name, while Marked is tracking if we have explored a given real package (along its chosen version), so we should stop MarkPackage from exploring a (real) package if it is marked and let fullyExplored only guard the looping over the individual dependencies. The testcase is deceptively simple, but in practice only an ecosystem like rust who makes heavy use of cyclic dependency relations intermixed with versioned provides actually triggers this as seen by the buggy code being in use for four months in Debian and Ubuntu development releases. (easier to trigger if most packages are marked manual installed) Note that the testcase is successful already due to the earlier changes as we exit the recursion eventually and all packages are marked as they need to be already, but this fix does work standalone as well. Closes: #992993 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T17:49:43Z urn:sha1:e4701219cf821d24f7f48ed6e4d8123c11d47c8b MarkInstall has the same depth limit, so lets use this arbitrary limit to avoid trying to hard as that usually means we will never stop – at least not until we crash, which is not a very good error case. 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T17:22:26Z urn:sha1:d7e9f0779043814d6e7a4141170fa7f18cd90803 We can't have external users as its marked hidden and as we don't even have users outside of pkgDepCache itself we can remove the function completely from the public eye. That adds many new parameters, but it also allows to add a few more like avoiding constantly checking the debug config value. Gbp-Dch: Ignore 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T11:24:06Z urn:sha1:b104891d015d0b8e152d8ac723027f824077948a The old code is a bit longer and does a sort (N*log(N)) + find (at most N) It is replaced by max_element (N) + remove_if (N). The practical difference is minimal as the N we operate on is rather small, but the replacement is hopefully easier to understand at a glance as well. Gbp-Dch: Ignore 2021-04-26T11:00:24Z David Kalnischkies david@kalnischkies.de 2021-03-18T18:08:48Z urn:sha1:6f01e7cc0c6f231711b3b81a81beb3775f0a855a They are kinda costly, so it makes more sense to keep them around in private storage rather than generate them all the time in the MarkPackage method. We do keep them lazy through as we have that implemented already. 2021-04-26T11:00:24Z David Kalnischkies david@kalnischkies.de 2021-03-18T16:37:49Z urn:sha1:d6f3458badf2cfea3ca7de7632ae31daff5742be An interactive tool like aptitude needs these flags current far more often than we do as a user can see them in apt only in one very well defined place – the autoremove display block – so we don't need to run it up to four times while a normal "apt install" is processed as that is just busywork. The effect on runtime is minimal, as a single run doesn't take too long anyhow, but it cuts down tremendously on debug output at the expense of requiring some manual handholding. This is opt-in so that aptitude doesn't need to change nor do we need to change our own tools like "apt list" where it is working correctly as intended. A special flag and co is needed as we want to prevent the ActionGroup inside pkgDepCache::Init to be inhibited already so we need to insert ourselves while the DepCache is still in the process of being built. This is also the reason why the debug output in some tests changed to all unmarked, but that is fine as the marking could have been already obsoleted by the actions taken, just inhibited by a proper action group. 2021-04-26T11:00:24Z David Kalnischkies david@kalnischkies.de 2021-03-18T13:40:31Z urn:sha1:9a54e70c1040379fb06827bacb461c61e341e694 The autoremove algorithm would mark a package previously after exploring it once, but it could have been that it ignored some providers due to them not satisfying the (versioned) dependency. A later dependency which they might satisfy would encounter the package as already marked and hence doesn't explore the providers anymore leaving us with internal errors (as in the contrived new testcase). This is resolved by introducing a new flag denoting if we explored every provider already and only skip exploring if that is true, which sounds bad but is really not such a common occurrence that it seems noticeable in practice. It also helps us marking virtual packages as explored now which would previously be tried each time they are encountered mostly hiding this problem for the (far more common) fully virtual package. 2021-04-25T12:23:13Z David Kalnischkies david@kalnischkies.de 2021-03-17T23:47:16Z urn:sha1:acc5502e7bd4bee178b8da3198a376d9001ab414 An out-of-tree kernel module which doesn't see many new versions can pile up a considerable amount of packages if it is depended on via another packages (e.g.: v4l2loopback-utils recommends v4l2loopback-modules) which in turn can prevent the old kernels from being removed if they happen to have a dependency on the images. To prevent this we check if a provider is a versioned kernel package (like an out-of-tree module) and if so check if that module package is part of the protected kernel set – if not it is probably good to go. We only do this if at least one provider is from a protected kernel though so that the dependency remains satisfied (this can happen e.g. if the module is currently not buildable against a protected kernel). 2021-02-12T11:53:57Z Julian Andres Klode julian.klode@canonical.com 2021-02-12T11:53:57Z urn:sha1:3b198616423daaef69c938fbcc5dd11a1e8f866c