Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.1.exp1 2014-06-18T10:41:11Z 2014-06-18T10:41:11Z David Kalnischkies david@kalnischkies.de 2014-06-13T07:10:11Z urn:sha1:ac2c559b33db7a2dbec3a95760ee2497baf7017a It still doesn't reflect the size the cache has on the disk compared to what is given as total size (90 vs 103 MB), but by counting all structs in we are at least a bit closer to the reality. Git-Dch: ignore 2014-06-18T10:41:11Z David Kalnischkies david@kalnischkies.de 2014-06-12T10:22:45Z urn:sha1:4ad8619bb1f0bf777d17c568bb7a6cf7f30aac34 We had a wild mixture of (unsigned) int, long and long long here without much sense, so this commit adds a few typedefs to get some sense in the typesystem and ensures that a ID isn't sometimes computed as int, stored as long and compared with a long long… as this could potentially bite us later on as the size of the archive only increases over time. 2014-06-18T10:41:11Z David Kalnischkies david@kalnischkies.de 2014-06-11T18:42:16Z urn:sha1:e8a7b0b28ca01cd8c2c1bee0d83e5997b40de689 It also makes the size configureable, so it can be adapted in the future without the need for an abi break - and even by users… The increase was long overdue as it gives a >10% decrease in runtime of e.g. 'apt-get check -s'. Some (useless) benchmark with 69933 groups and 187796 packages without a pre-built cache: time apt-get check -so APT::Cache-HashTableSize=1 → 20m time apt-get check -so APT::Cache-HashTableSize=1000 → 6,41s time apt-get check -so APT::Cache-HashTableSize=2000 → 5,64s (old) time apt-get check -so APT::Cache-HashTableSize=3000 → 5,30s time apt-get check -so APT::Cache-HashTableSize=5000 → 5,08s time apt-get check -so APT::Cache-HashTableSize=6000 → 5,05s time apt-get check -so APT::Cache-HashTableSize=7000 → 5,02s time apt-get check -so APT::Cache-HashTableSize=8000 → 5,00s time apt-get check -so APT::Cache-HashTableSize=9000 → 4,98s time apt-get check -so APT::Cache-HashTableSize=10000 → 4,96s (new) time apt-get check -so APT::Cache-HashTableSize=15000 → 4,90s time apt-get check -so APT::Cache-HashTableSize=20000 → 4,86s time apt-get check -so APT::Cache-HashTableSize=30000 → 4,77s time apt-get check -so APT::Cache-HashTableSize=40000 → 4,74s time apt-get check -so APT::Cache-HashTableSize=50000 → 4,73s time apt-get check -so APT::Cache-HashTableSize=60000 → 4,71s The gap increases further for operations which have more package lookups. Factor 5 was chosen as higher values do not provide any really significant timing advantage anymore compared to the memory increase in my testing and there is always the possibility to increase it now if that changes. (also most users will not have 3 releases and 4 architectures in the cache, so theirs will be much smaller and faster). 2014-06-18T08:13:01Z Michael Vogt mvo@debian.org 2014-06-18T08:13:01Z urn:sha1:da029b0aaebdc64a3a9f6b7012213539421c934b Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h apt-pkg/deb/debmetaindex.h apt-pkg/pkgcache.cc test/integration/test-apt-ftparchive-src-cachedb 2014-06-18T07:35:53Z Michael Vogt mvo@ubuntu.com 2014-06-18T07:35:53Z urn:sha1:b686f453b14e11a69ecbd368509f7eaf1596c6e0 This is a internal struct not a external interface so the actual breakage should be small. 2014-05-10T10:50:00Z David Kalnischkies david@kalnischkies.de 2014-05-10T10:50:00Z urn:sha1:7a223b933cab0447438ca2e964576da39078eaf4 The cache heavily depends on the architecture(s) it is build for, especially if you move from single- to multiarch. Adding a new architecture to dpkg therefore has to be detected and must invalidate the cache so that we don't operate on incorrect data. The incorrect data will prevent us from doing otherwise sensible actions (it doesn't allow bad things to happen) and the recovery is simple and automatic in most cases, so this hides pretty well and is also not as serious as it might sound at first. Closes: 745036 2014-05-09T11:06:27Z David Kalnischkies kalnischkies@gmail.com 2013-08-18T20:20:25Z urn:sha1:ffe3c68e494efc1c2c4d748fa9d69e150867e5c2 It seems unlikely for now that proper archives will carry multiple Description-* stanzas in the Packages (or Translation-*) file, but sometimes apt eats its own output as shown by the usage of the CD team and it would be interesting to let apt output multiple translations e.g. in 'apt-cache show'. 2014-03-13T12:58:46Z David Kalnischkies david@kalnischkies.de 2014-03-12T16:55:08Z urn:sha1:68134bda8868a8614edf137aad7e9be61cd2a6ee In #737085 we see that apt can be confused if informations about versions only differ slightly. This commit adds a way of at least adding a few more data points with the next abi break to help a bit with it. Git-Dch: Ignore 2014-03-13T12:58:45Z David Kalnischkies david@kalnischkies.de 2014-03-05T21:11:25Z urn:sha1:453b82a388013e522b3a1b9fcd6ed0810dab1f4f Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use) 2014-03-13T12:58:45Z David Kalnischkies david@kalnischkies.de 2014-03-01T14:11:42Z urn:sha1:655122418d714f342b5d9789f45f8035f3fe8b9a Reported-By: gcc -Wunused-parameter Git-Dch: Ignore