Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.3 2016-09-02T15:16:36Z 2016-09-02T15:16:36Z Julian Andres Klode jak@debian.org 2016-06-15T21:13:43Z urn:sha1:2a440328ea19e9646a93f847dd9eff21e03ad16d Employ a priority queue instead of a normal queue to hold the items; and only add items to the running pipeline if their priority is the same or higher than the priority of items in the queue. The priorities are designed for a 3 stage pipeline system: In stage 1, all Release files and .diff/Index files are fetched. This allows us to determine what files remain to be fetched, and thus ensures a usable progress reporting. In stage 2, all Pdiff patches are fetched, so we can apply them in parallel with fetching other files in stage 3. In stage 3, all other files are fetched (complete index files such as Contents, Packages). Performance improvements, mainly from fetching the pdiff patches before complete files, so they can be applied in parallel: For the 01 Sep 2016 03:35:23 UTC -> 02 Sep 2016 09:25:37 update of Debian unstable and testing with Contents and appstream for amd64 and i386, update time reduced from 37 seconds to 24-28 seconds. Previously, apt would first download new DEP11 icon tarballs and metadata files, causing the CPU to be idle. By fetching the diffs in stage 2, we can now patch our contents and Packages files while we are downloading the DEP11 stuff. 2016-08-29T07:22:25Z David Kalnischkies david@kalnischkies.de 2016-08-28T10:58:20Z urn:sha1:4ff5e237d5685be187a75c563b86e80ea3e7cc01 Without randomizing the order in which we download the index files we leak needlessly information to the mirrors of which architecture is native or foreign on this system. More importantly, we leak the order in which description translations will be used which in most cases will e.g. have the native tongue first. Note that the leak effect in practice is limited as apt detects if a file it wants to download is already available in the latest version from a previous download and does not query the server in such cases. Combined with the fact that Translation files are usually updated infrequently and not all at the same time, so a mirror can never be sure if it got asked about all files the user wants. 2016-08-26T22:31:03Z Julian Andres Klode jak@debian.org 2016-08-26T22:31:03Z urn:sha1:6a68315e938eb2611806658828ecea86805822e7 2016-08-26T20:24:25Z Julian Andres Klode jak@debian.org 2016-08-25T14:25:00Z urn:sha1:6f1f3c9afdb6ade6a7be110b90c8fc9e603254cf This is needed on BSD where root's default group is wheel, not root. 2016-08-26T20:24:24Z Julian Andres Klode jak@debian.org 2016-08-25T12:24:49Z urn:sha1:2ed62ba6abcad809d1898a40950f86217af73812 This fixes issues with chroots, but the goal here was to get the test suite working on systems without dpkg. 2016-08-26T20:17:54Z Julian Andres Klode jak@debian.org 2016-08-23T17:41:58Z urn:sha1:8757a0fdeee00ea6a7cc717188a0e129ad8a553c This allows other vendors to use different paths, or to build your own APT in /opt for testing. Note that this uses + 1 in some places, as the paths we receive are absolute, but we need to strip of the initial /. 2016-08-26T20:17:13Z Julian Andres Klode jak@debian.org 2016-08-23T18:19:11Z urn:sha1:0fb16c3e678044d6d06ba8a6199b1e96487ee0d8 The C.UTF-8 locale is not portable, so we need to use C, otherwise we crash on other systems. We can use std::locale::classic() for that, which might also be a bit cheaper than using locale("C"). 2016-08-24T08:24:41Z David Kalnischkies david@kalnischkies.de 2016-08-24T07:47:48Z urn:sha1:2e2865ae53a65c00dd55a892d5b48458f3110366 The bugreport shows a segfault caused by the code not doing the correct magical dance to remove an item from inside a queue in all cases. We could try hard to fix this, but it is actually better and also easier to perform these checks (which cause instant failure) earlier so that they haven't entered queue(s) yet, which in return makes cleanup trivial. The result is that we actually end up failing "too early" as if we wouldn't be careful download errors would be logged before that process was even started. Not a problem for the acquire system, but likely to confuse users and programs alike if they see the download process producing errors before apt was technically allowed to do an acquire (it didn't, so no violation, but it looks like it to the untrained eye). Closes: 835195 2016-08-17T05:55:46Z David Kalnischkies david@kalnischkies.de 2016-08-16T05:47:44Z urn:sha1:77e274f5ad23d79294f28ecc9868fc6f534214a4 In af81ab9030229b4ce6cbe28f0f0831d4896fda01 by-hash got implemented as a special compression type for our usual index files like Packages. Missing in this scheme was the special .diff/Index index file containing the info about individual patches for this index file. Deriving from the index file class directly we inherent the compression handling infrastructure and in this way also by-hash nearly for free. Closes: #824926 2016-08-17T05:55:45Z David Kalnischkies david@kalnischkies.de 2016-08-15T08:51:07Z urn:sha1:a99bb943e97883af6f063af9eb14e20b7a7320fc The URI we later want to modify to get the file via by-hash was unset in case a file was only available uncompressed (which is usually not the case) causing an acquire error.