Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=2.3.9 2021-09-06T14:51:27Z 2021-09-06T14:51:27Z Julian Andres Klode julian.klode@canonical.com 2021-09-06T14:51:27Z urn:sha1:e664bdcfb287e7456f084e0335b7c68bab89d6e7 When an item has been delayed and the queue is cycled to start it, we did not properly report an error from the cycling, and we would then fail in the assert(), causing all errors to be lost. Propagate the error instead and make the assert a warning. 2021-08-29T12:23:26Z David Kalnischkies david@kalnischkies.de 2021-08-29T11:50:31Z urn:sha1:5f6bbfa53c32ec30aff6a2bc8c412616049eab18 If you install dpkg on an empty status file with all recommends and suggests apt wants to install 4000+ packages. The deepest chain seemingly being 236 steps long. And dpkg isn't even the worst (~259). That is a problem as libapt has a hardcoded recursion limit for MarkInstall and friends … set to 100. We are saved by the fact that chains without suggests are much shorter (dpkg has 5, max seems ~43), but I ignored Conflicts in these chains, which typically trigger upgrades, so if two of the worst are chained together we suddenly get dangerously close to the limit still. So, lets just increase the limit into oblivion as it is really just a safety measure we should not be running into to begin with. MarkPackage was running years without it after all. 3000 is picked as a nice number as any other and because it is roughly the half of the stack crashs I saw previously in this branch. 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T13:55:09Z urn:sha1:c7e368aafe099dcd966cf5994ae7fb418d268278 fullyExplored is needed to keep track of having explored all providers of a package name, while Marked is tracking if we have explored a given real package (along its chosen version), so we should stop MarkPackage from exploring a (real) package if it is marked and let fullyExplored only guard the looping over the individual dependencies. The testcase is deceptively simple, but in practice only an ecosystem like rust who makes heavy use of cyclic dependency relations intermixed with versioned provides actually triggers this as seen by the buggy code being in use for four months in Debian and Ubuntu development releases. (easier to trigger if most packages are marked manual installed) Note that the testcase is successful already due to the earlier changes as we exit the recursion eventually and all packages are marked as they need to be already, but this fix does work standalone as well. Closes: #992993 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T17:49:43Z urn:sha1:e4701219cf821d24f7f48ed6e4d8123c11d47c8b MarkInstall has the same depth limit, so lets use this arbitrary limit to avoid trying to hard as that usually means we will never stop – at least not until we crash, which is not a very good error case. 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T17:22:26Z urn:sha1:d7e9f0779043814d6e7a4141170fa7f18cd90803 We can't have external users as its marked hidden and as we don't even have users outside of pkgDepCache itself we can remove the function completely from the public eye. That adds many new parameters, but it also allows to add a few more like avoiding constantly checking the debug config value. Gbp-Dch: Ignore 2021-08-28T20:21:35Z David Kalnischkies david@kalnischkies.de 2021-08-28T11:24:06Z urn:sha1:b104891d015d0b8e152d8ac723027f824077948a The old code is a bit longer and does a sort (N*log(N)) + find (at most N) It is replaced by max_element (N) + remove_if (N). The practical difference is minimal as the N we operate on is rather small, but the replacement is hopefully easier to understand at a glance as well. Gbp-Dch: Ignore 2021-07-28T11:08:35Z Julian Andres Klode julian.klode@canonical.com 2021-07-28T10:43:56Z urn:sha1:4001af8920389e2bb2672b673b181c4e92515872 This is very basic support on the testing side, we just test the debug output but not how long it actually took. Would be nice to check time really. 2021-07-28T11:08:35Z Julian Andres Klode julian.klode@canonical.com 2021-07-27T15:14:23Z urn:sha1:51ee79e385d8ccb2f32195cc01f3f9c943197603 Fix the typo, and use the helper function to convert it, so we do not end up with 5 seconds encoded as 0s and 5*10^6 microseconds. 2021-07-28T11:08:35Z Julian Andres Klode julian.klode@canonical.com 2021-07-27T14:40:38Z urn:sha1:48a7ece8d0e065240153a85089f544383e16137f This yields more accurate delays and avoids issues with clock skew. 2021-07-28T11:08:32Z Julian Andres Klode julian.klode@canonical.com 2021-07-13T10:21:23Z urn:sha1:7e88def0548a92112f8295686c2a12379e4028be If there is an item with fetchAfter at the top of a queue, reduce sleep() timeout so we can detect it and start it, by calling Cycle() on the queue in the next iteration. For some reasons we have to call select() with a 0s timeout if we just marked an item as ready. Oh well. Previous versions of this patch only called global Bump() after a timeout from select(); this was unfortunately incorrect - it meant that we never bumped a queue that did not start yet while other queues were running, potentially significantly delaying retries.