apt/apt-pkg/pkgcachegen.h, branch 2.7.12

Compare SHA256 to check if versions are really the same

2023-08-02T10:04:32Z

If we know both SHA256, and they're different, the packages are. This approach stores the SHA256 only at runtime, avoiding the overhead of storing it on-disk, because when we update repositories we update all of them anyhow. Note that pkgCacheGenerator is hidden, so we can just modify its ABI, hooray. Closes: #931175 LP: #2029268

Avoid undefined pointer arithmetic while growing mmap

2021-02-04T10:00:00Z

The undefined behaviour sanitizer complains with: runtime error: addition of unsigned offset to 0x… overflowed to 0x… Compilers and runtime do the right thing in any case and it is a codepath that can (and ideally should) be avoided for speed reasons alone, but fixing it can't hurt (too much).

Do not require libxxhash-dev for including pkgcachegen.h

2020-12-17T15:20:48Z

Use XXH3 for cache, hash table hashing

2020-12-15T12:47:22Z

XXH3 is faster than both our CRC32c implementation as well as DJB hash for hash table hashing, so meh, let's switch to it.

Make map_pointer typesafe

2020-02-24T17:29:07Z

Instead of just using uint32_t, which would allow you to assign e.g. a map_pointer to a map_pointer, use our own smarter struct that has strict type checking. We allow creating a map_pointer from a nullptr, and we allow comparing map_pointer to nullptr, which also deals with comparisons against 0 which are often used, as 0 will be implictly converted to nullptr.

Wrap AllocateInMap with a templated version

2020-02-24T17:09:49Z

Replace map_pointer_t with map_pointer

2020-02-24T16:08:34Z

This is a first step to a type safe cache, adding typing information everywhere. Next, we'll replace map_pointer implementation with a type safe one.

Use a 32-bit djb VersionHash instead of CRC-16

2020-02-18T12:39:26Z

Remove includes of (md5|sha1|sha2).h headers

2020-01-14T12:10:36Z

Remove it everywhere, except where it is still needed.

Avoid extra out-of-cache hash table deduplication for package names

2020-01-08T10:13:27Z

We were de-duplicating package name strings in StoreString, but also deduplicating most of them by them being in groups, so we had extra hash table lookups that could be avoided in NewGroup(). To continue deduplicating names across binary packages and source packages, insert groups for source packages as well. This is also a good first step in allowing efficient lookup of packages by source package - we can extend Group later by a list of SourceVersion objects, or alternatively, simply add a by-source chain into pkgCache::Version. This change improves performance by about 10% (913 to 814 ms), while having no significant overhead on the cache size: --- before +++ after @@ -1,7 +1,7 @@ -Total package names: 109536 (2.191 k) -Total package structures: 118689 (4.748 k) +Total package names: 119642 (2.393 k) +Total package structures: 118687 (4.747 k) Normal packages: 83309 - Pure virtual packages: 3365 + Pure virtual packages: 3363 Single virtual packages: 17811 Mixed virtual packages: 1973 Missing: 12231 @@ -10,21 +10,21 @@ Total distinct descriptions: 149291 (3.583 k) Total dependencies: 484135/156650 (12,2 M) Total ver/file relations: 57421 (1.378 k) Total Desc/File relations: 18219 (437 k) -Total Provides mappings: 29963 (719 k) +Total Provides mappings: 29959 (719 k) Total globbed strings: 226993 (5.332 k) Total slack space: 26,8 k -Total space accounted for: 38,1 M +Total space accounted for: 38,3 M Total buckets in PkgHashTable: 50503 - Unused: 5727 - Used: 44776 - Utilization: 88.6601% - Average entries: 2.65073 + Unused: 5728 + Used: 44775 + Utilization: 88.6581% + Average entries: 2.65074 Longest: 60 Shortest: 1 Total buckets in GrpHashTable: 50503 - Unused: 5727 - Used: 44776 - Utilization: 88.6601% - Average entries: 2.44631 - Longest: 10 + Unused: 4649 + Used: 45854 + Utilization: 90.7946% + Average entries: 2.60919 + Longest: 11 Shortest: 1