Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.4_rc2 2017-01-28T13:06:29Z 2017-01-28T13:06:29Z Julian Andres Klode jak@debian.org 2017-01-25T02:38:59Z urn:sha1:7b78e8bef1fc9de22d826db1db9df25f97d3710c Since the introduction of by-hash, two differently named files might have the same real URL. In our case, the files icons-64x64.tar.gz and icons-128x128.tar.gz of empty tarballs. APT would try to merge them and end with weird errors because it completed the first download and enters the second stage for decompressing and verifying. After that it would queue a new item to copy the original file to the location, but that copy item would be in the wrong stage, causing it to use the hashes for the decompressed item. Closes: #838441 2017-01-19T02:57:45Z David Kalnischkies david@kalnischkies.de 2017-01-19T02:57:45Z urn:sha1:7ca83492e802967f183babf06ab541b1b51f1703 If apt renames a file to .FAILED it leaves its namespace and is never touched again – expect since 1.1~exp4 in which "apt clean" will remove those files. The usefulness of these files rapidly degrades if you don't keep the update log itself (together with debug output in the best case) through and on 99% of all system they will be kept around forever just to collect dust over time and eat up space. With this commit an update call will remove all FAILED files of previous runs, so that the FAILED files you have on disk are always only the ones related to the last apt run stopping apt from hoarding files. Closes: 846476 2016-12-16T12:50:00Z David Kalnischkies david@kalnischkies.de 2016-11-25T14:15:01Z urn:sha1:78db35195eddcd156130fff9ea3e895b30cbf9c3 Note: This is a warning about disabling a security feature. It is supposed to be scary as we are disabling a security feature and we can't just be silent about it! Downloads really shouldn't happen any longer as root to decrease the attack surface – but if a warning causes that much uproar, consider what an error would do… The old WARNING message: | W: Can't drop privileges for downloading as file 'foobar' couldn't be | accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) is frequently (incorrectly) considered to be an error message indicating that the download didn't happen which isn't the case, it was performed, but without all the security features enabled we could have used if run from some other place… The word "unsandboxed" is chosen as the term 'sandbox(ed)' is a common encounter in feature lists/changelogs and more people are hopefully able to make the connection to 'security' than it is the case for 'privilege dropping' which is more correct, but far less known. Closes: #813786 LP: #1522675 2016-11-24T23:15:13Z David Kalnischkies david@kalnischkies.de 2016-11-13T01:29:46Z urn:sha1:5832913a49d4f7c75527264a935cc0ce00627f1d In ad9416611ab83f7799f2dcb4bf7f3ef30e9fe6f8 we fall back to asking the original mirror (e.g. a redirector) if we do not get the expected result. This works for the indexes, but patches are a different beast and much simpler. Adding this fallback code here seems like overkill as they are usually right along their Index file, so actually forward the relevant settings to the patch items which fixes pdiff support combined with a redirector and partial mirrors as in such a situation the pdiff patches would be 404 and the complete index would be downloaded. 2016-09-02T15:16:36Z Julian Andres Klode jak@debian.org 2016-06-15T21:13:43Z urn:sha1:2a440328ea19e9646a93f847dd9eff21e03ad16d Employ a priority queue instead of a normal queue to hold the items; and only add items to the running pipeline if their priority is the same or higher than the priority of items in the queue. The priorities are designed for a 3 stage pipeline system: In stage 1, all Release files and .diff/Index files are fetched. This allows us to determine what files remain to be fetched, and thus ensures a usable progress reporting. In stage 2, all Pdiff patches are fetched, so we can apply them in parallel with fetching other files in stage 3. In stage 3, all other files are fetched (complete index files such as Contents, Packages). Performance improvements, mainly from fetching the pdiff patches before complete files, so they can be applied in parallel: For the 01 Sep 2016 03:35:23 UTC -> 02 Sep 2016 09:25:37 update of Debian unstable and testing with Contents and appstream for amd64 and i386, update time reduced from 37 seconds to 24-28 seconds. Previously, apt would first download new DEP11 icon tarballs and metadata files, causing the CPU to be idle. By fetching the diffs in stage 2, we can now patch our contents and Packages files while we are downloading the DEP11 stuff. 2016-08-26T22:31:03Z Julian Andres Klode jak@debian.org 2016-08-26T22:31:03Z urn:sha1:6a68315e938eb2611806658828ecea86805822e7 2016-08-26T20:24:25Z Julian Andres Klode jak@debian.org 2016-08-25T14:25:00Z urn:sha1:6f1f3c9afdb6ade6a7be110b90c8fc9e603254cf This is needed on BSD where root's default group is wheel, not root. 2016-08-26T20:17:13Z Julian Andres Klode jak@debian.org 2016-08-23T18:19:11Z urn:sha1:0fb16c3e678044d6d06ba8a6199b1e96487ee0d8 The C.UTF-8 locale is not portable, so we need to use C, otherwise we crash on other systems. We can use std::locale::classic() for that, which might also be a bit cheaper than using locale("C"). 2016-08-24T19:49:34Z David Kalnischkies david@kalnischkies.de 2016-08-24T19:49:34Z urn:sha1:1044354995513348f4836772fe77068585091d6b Improve-Upon: 2e2865ae53a65c00dd55a892d5b48458f3110366 Reported-By: Julian Andres Klode Gbp-Dch: Ignore 2016-08-24T08:24:41Z David Kalnischkies david@kalnischkies.de 2016-08-24T07:47:48Z urn:sha1:2e2865ae53a65c00dd55a892d5b48458f3110366 The bugreport shows a segfault caused by the code not doing the correct magical dance to remove an item from inside a queue in all cases. We could try hard to fix this, but it is actually better and also easier to perform these checks (which cause instant failure) earlier so that they haven't entered queue(s) yet, which in return makes cleanup trivial. The result is that we actually end up failing "too early" as if we wouldn't be careful download errors would be logged before that process was even started. Not a problem for the acquire system, but likely to confuse users and programs alike if they see the download process producing errors before apt was technically allowed to do an acquire (it didn't, so no violation, but it looks like it to the untrained eye). Closes: 835195