Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.2.4 2016-02-26T14:17:14Z 2016-02-26T14:17:14Z Julian Andres Klode jak@debian.org 2016-02-26T14:17:14Z urn:sha1:2f4e4070a5247abac86b2ca4ae24a8400da3c42e Reported-By: Helmut Grohne on IRC 2016-02-11T22:13:47Z David Kalnischkies david@kalnischkies.de 2016-02-11T21:54:49Z urn:sha1:6fd4b4c0b693b52cb8b593b76e5b60f77e500454 pkgAcqChangelog has the default behaviour of downloading a changelog to a temporary directory (inside /tmp, not /tmp directly), which is cleaned up on shutdown, but this can be overridden to store the changelog more permanently – but that caries a permission problem. For changelog we can 'easily' solve this by always downloading to a temporary directory and only move it out of there on done. 2016-02-11T20:07:56Z David Kalnischkies david@kalnischkies.de 2016-02-11T20:07:56Z urn:sha1:b5aba9096e371a5f8612aff05384aca54ccc5acd If pkgAcqChangelog is told to acquire the changelog for a version it will check first if this version is installed on the disk and if so will use the local changelog in /usr/share/doc (possibily/likely gz compressed) instead of downloading the file from the web. An option is provided to disable this, which is enabled by default for the Ubuntu vendor as they truncate the local changelogs – and for apts --print-uris action. 2016-01-08T16:51:23Z David Kalnischkies david@kalnischkies.de 2016-01-08T16:51:23Z urn:sha1:ef3c549e00b2a0487ddee0aeb70e3a29f76c2fbb The code already deals with compressed leftovers, but forgot the uncompressed files. The opertunity is picked to reorder this code and add debug messages about the actions taken as well as produce such a leftover file in the associated testcase. 2016-01-08T14:40:01Z David Kalnischkies david@kalnischkies.de 2016-01-08T14:30:05Z urn:sha1:4e6219da0dd1e68fad7db972f7ddd76598645228 With the addition of the $HASH-Download field in the .diff/Index we got the size of the compressed patches for 'free', so if that information is available we can use it for a more fitting calculation of the size requirements of the patches vs. the complete file. Note that this predicts a too small size in the transition case in which the information isn't available for all patches, but figuring this out would be a lot of code for practically nothing as only one update can ever be in such a transition phase. 2016-01-08T14:40:01Z David Kalnischkies david@kalnischkies.de 2016-01-07T19:32:09Z urn:sha1:0179cfa83cf0042235eda41db7f35c420781c63e Downloading and storing are two different operations were different compression types can be preferred. For downloading we provide the choice via Acquire::CompressionTypes::Order as there is a choice to be made between download size and speed – and limited by whats available in the repository. Storage on the other hand has all compressions currently supported by apt available and to reduce runtime of tools accessing these files the compression type should be a low-cost format in terms of decompression. apt traditionally stores its indexes uncompressed on disk, but has options to keep them compressed. Now that apt downloads additional files we also deal with files which simply can't be stored uncompressed as they are just too big (like Contents for apt-file). Traditionally they are downloaded in a low-cost format (gz) as repositories do not provide other formats, but there might be even lower-cost formats and for download we could introduce higher-cost in the repositories. Downloading an entire index potentially requires recompression to another format, so an update takes potentially longer – but big files are usually updated via pdiffs which has to de- and re-compress anyhow and does it on the fly anyhow, so there is no extra time needed and in general it seems to be benefitial to invest the time in update to save time later on file access. 2016-01-08T14:40:01Z David Kalnischkies david@kalnischkies.de 2016-01-05T23:05:24Z urn:sha1:4e3c5633b1e74b4f58b95f339cfbbf4cbf21ab3e There is no reason to enforce that the file we start the bootstrap with is compressed with a compressor which is available online. This allows us to change the on-disk format as well as deals with repositories adding/removing support for a specific compressor. 2016-01-08T14:40:01Z David Kalnischkies david@kalnischkies.de 2016-01-05T23:08:04Z urn:sha1:1866240123a57de8f693b0ba01d8b709f027282d If we store files compressed in lists/ and the file switched compression formats we happened to retain the "old" format, but by default the cleanup process catched this oversight and removed the file. [The initial situation described doesn't arise as we store no files by default compressed and even with apt-file configuring Contents files, we don't really have that problem as there is just .gz files for those.] We solve this by just removing any uncompressed as well as compressed (we support) file just before we move the 'new' version of the file in. 2016-01-08T14:40:01Z David Kalnischkies david@kalnischkies.de 2016-01-03T18:23:30Z urn:sha1:9bd2313a5c7523501bcec398877489c5a1fc1415 Adding a new compressor method meant adding a new method as well – even if that boilt down to just linking to our generalized decompressor with a new name. That is unneeded busywork if we can instead just call the generalized decompressor and let it figure out which compressor to use based on the filenames rather than by program name. For compatibility we ship still 'gzip', 'bzip2' and co, but they are just links to our "new" 'store' method. 2015-12-27T16:04:33Z David Kalnischkies david@kalnischkies.de 2015-12-27T16:04:33Z urn:sha1:a628ca5256b4a2f3ae300697b17adf150b6e17b0 Debian has a Packages file for arch:all already, but the arch:any files contain arch:all packages as well, so downloading it would be a total waste of resources. Getting this solved is on the list of things to do, but it is also the hardest part – for index targets like Contents the situation is much easier and less server/client implementations are involved so we might not want to stall them. A repository can now declare via: No-Support-for-Architecture-all: Packages that even if an arch:all Packages exists, it shouldn't be downloaded, so that support for Contents files can be added now. See also 1dd20368486820efb6ef4476ad739e967174bec4 for the implementation of downloading arch:all index targets, which this is limiting. The field uses the name of the target from the apt configuration for simplicity and is negative by design as this field is intended to be supported/needed only for a "short" time (one or two Debian releases). While this commit theoretically supports any target, its expected to only see "Packages" as a value in reality.