summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Release 2.9.142.9.14Julian Andres Klode2024-11-225-6/+45
| | | | | Oh well I accidentally committed everything from my work email today but I'm off today ...
* apt-key: Prefer gpgv-sq if it existsJulian Andres Klode2024-11-221-0/+1
| | | | | This allows you to test out gpgv-sq by just installing it, which is nice.
* Merge branch 'fix/partialfilemirror' into 'main'Julian Andres Klode2024-11-2217-286/+602
|\ | | | | | | | | Support uncompressed indexes from partial file:/ mirrors See merge request apt-team/apt!235
| * Look at non by-hash paths in copy and file methodsDavid Kalnischkies2024-11-228-102/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ideally copy and file mirrors would support by-hash as well, but its harder to setup and maintain especially if you want to cache an online mirror who has by-hash enabled. We can avoid unneeded roundtrips (in the best case) and entire cache misses (in the usual worst case) by "just" telling methods that the URI we passed it has the requested file perhaps also in other paths. This is done in pseudo-relative paths as we would otherwise need to teach redirection code to rewrite those URIs as well. A method like http can easily ignore this value and await explicit instructions to look at that file, but inspecting the path in local sources via file or copy is (comparatively) free, so we just do it immediately. If that ends up being the wrong version of the file as by-hash would have protected us from we are in this feature branch now falling back to other mirrors which are like the ones online and in support of by-hash.
| * Allow HashSum mismatches to fallback to other mirrorsDavid Kalnischkies2024-11-223-48/+94
| | | | | | | | | | | | | | | | | | | | | | Perhaps a common error in the past nowadays thanks to by-hash we rarely get mismatches due to our calling behaviour and more because the mirror is genuinely bad especially if this mirror didn't work with by-hash. Using our "normal" error handling allows falling back to other mirrors which hopefully do support by-hash and/or are not broken on a filesystem level making us more resistent to the occasional bad apple in a wild mirror forest.
| * Never consider file/copy/cdrom sources bad sites due to errorsDavid Kalnischkies2024-11-223-3/+14
| | | | | | | | | | | | | | | | | | | | | | As the hostname for them is likely empty (for absolute or . or .. for relative paths) considering one a bad site causes all of the sites from this scheme to be considered bad. In a perfect world, we would figure out a good site-path to use for these sources, but as they are local sources which means we don't have to perform costly requests to the internet we just let them happen instead for now.
| * Clear previous ignored errors before processing acquire item againDavid Kalnischkies2024-11-221-0/+4
| | | | | | | | | | | | | | | | If an item failed previously, but that failure is ignored and a retry is triggered we might end up in a situation in which it is failing yet again for perhaps totally different reasons, but the error message displayed is the one we have shown (or, because it was ignored likely hidden) before which can be rather confusing in debugging.
| * Do not change UsedMirror on local copy of InReleaseDavid Kalnischkies2024-11-221-1/+1
| | | | | | | | | | | | | | | | While we pick most files from the file:/ mirror directly, we explicitly bring in the InRelease file with a copy one if-clause later, which causes this code to be run again and ends up considering the copy:/ the mirror the file was picked up from, which works, but isn't what we had signed up for.
| * Do not assume mirror-URIs end in a filename causing a hangDavid Kalnischkies2024-11-222-3/+23
| | | | | | | | | | | | | | In practice most of them will, but making the acquire process hang in response if they don't seems like an overly excessive response if we can just support it gracefully to reply with an error (or actually succeed in the unlikely event this URI is actually correct).
| * Allow AutoDetectProxy to work with more than just http(s)David Kalnischkies2024-11-225-39/+93
| | | | | | | | | | | | | | | | It is a bit unfair for third-party methods wrapping e.g. http to not have access to such tools, same for our old ftp and e.g. our tor even if in practice this isn't used much even for http. At least that is one less FIXME in the code.
| * Rewrite hidden config option checker for more patternsDavid Kalnischkies2024-11-223-54/+112
| | | | | | | | | | | | | | | | | | | | | | | | | | The previous version allowed only foo::*::bar as a pattern, now you can have as many starred parts as you want allowing for a finer control instead of hardcoding specific common star-values or giving up with **. It also adds a new INVALID type for an option to mark options which should not be used, but are covered by a pattern which would normally allow it. Our tests are more or less the only users of this hidden feature making this change not very risky even through it has no dedicated test itself.
| * Alternatively calculate alternative file hashes in file methodDavid Kalnischkies2024-11-222-1/+4
| | | | | | | | | | | | | | If we do not have the requested file we haven't calculated the hashes for it either and it is likely that the alternative file will be used, so to save the main thread from being busy with calculating hashes we do the calculation here in the method.
| * Show hashes calculated by main thread in error messagesDavid Kalnischkies2024-11-223-1/+26
| | | | | | | | | | | | | | | | | | | | | | The main thread calculates the hash sums if the method hasn't to be able to verify the files, but while a failure is detected and reported, the error messages did not include the hash sums which looks at least puzzling. So to have a complete error message we ensure that the hashes we calculated are appended to the message from the method so that it looks for the higher level code as if the method had calculated the hashes.
| * Support uncompressed indexes from partial file:/ mirrorsDavid Kalnischkies2024-11-224-38/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a file:/ mirror does not contain the requested compressed file, but the uncompressed variant of that file the method will sent us a fitting Done message with data only about the Alt-Filename, but the central hash verification code in the acquire system does not expect that looking for the non-existent compressed file to compare its hashes with the expected result – which fails (if we are verifying at least). That file:/ as well as the item code deals with this suggests that this feature was broken by 448c38bdcd72b52f11ec5f326f822cf57653f81c, but I couldn't (easily) compile that to verify, so this is only a guess. Supporting this allows using index files from /var/lib/apt/lists – which are likely uncompressed or can at least made so easily – to construct a file:/ mirror.
* | Merge branch 'test-gpgv-sq' into 'main'Julian Andres Klode2024-11-224-8/+44
|\ \ | | | | | | | | | | | | Test gpgv-sq See merge request apt-team/apt!403
| * | test: Ensure we test both gpgv-g10code and gpgv-sqJulian Andres Klode2024-11-223-6/+17
| | |
| * | test: Accept empty output from gpgv/apt-keyJulian Andres Klode2024-11-221-0/+3
| | | | | | | | | | | | | | | Our internal code always generates ENODATA for empty output and this is checked by the previous commit, so this is entirely safe.
| * | test-releasefile-verification: Test empty/invalid signaturesJulian Andres Klode2024-11-221-0/+22
| | |
| * | Accept BADSIG instead of EXPSIG for expired signaturesJulian Andres Klode2024-11-221-1/+1
| | | | | | | | | | | | Sequioa emits KEYEXPIRED + BADSIG instead of EXPSIG + VALIDSIG
| * | test-releasefile-verification: Fix signature timeJulian Andres Klode2024-11-221-1/+1
| | | | | | | | | | | | | | | | | | The key was only generated in 2010 and only bound in 2016, so we cannot have signatures from before then, so update the key accordingly, and change the default expiry to 2017.
* | | Merge branch 'fix/warnings' into 'main'Julian Andres Klode2024-11-2216-25/+30
|\ \ \ | |_|/ |/| | | | | | | | Fix a small bunch of compile-time warnings See merge request apt-team/apt!404
| * | Tag hidden acquire classes as final for devirtualizationDavid Kalnischkies2024-11-229-17/+17
| | | | | | | | | | | | | | | | | | | | | gcc reports `warning: Declaring type ‘class X’ final would enable devirtualization of Y calls [-Wsuggest-final-types]`. Not that this would massively improve performance, but it shouldn't hurt telling the compiler what is obvious for a human.
| * | Avoid unnormalized paths for CMake install destinationsDavid Kalnischkies2024-11-223-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CMake 3.31 is very noisy about our manpage (and to a lesser extend documentation in general) building as we used "//" and "/../" there. `cmake --help-policy CMP0177` documents the warning, so we could just decide on a value and deal with it, but given our usage is not really needed and rather trivial to change lets not pick a value and instead use a normalized path so we don't use different code paths in CMake depending on which CMake version we happen to be build with.
| * | Add a virtual destructor to private CacheSet MatcherDavid Kalnischkies2024-11-221-0/+1
| | | | | | | | | | | | | | | This silences the warning: | warning: ‘class Matcher’ has virtual functions and accessible non-virtual destructor [-Wnon-virtual-dtor]
| * | Fix expired std::string usage in APT::StringView testcaseDavid Kalnischkies2024-11-221-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | The anonymous std::string we were using here to create the view expires after the construction of the view and so the gcc warning | warning: ‘<anonymous>’ may be used uninitialized [-Wmaybe-uninitialized] is correct in that the memory we are pointing to is potentially overridden/reused already, if a bit roundabout in saying so.
| * | Acknowledge non-erase usage of remove_if for volatile sourcesDavid Kalnischkies2024-11-222-2/+2
|/ / | | | | | | | | | | | | | | | | | | | | | | | | gcc produces a length warning starting with: | warning: ignoring return value of ‘_FIter std::remove_if(_FIter, _FIter, _Predicate) [with …]’, declared with attribute ‘nodiscard’ [-Wunused-result] which is usually correct, but in the usage here we don't want to call an erase as we operate on a c-style new[] array here which contains pointers into argv and has an explicit sentinel (nullptr) – FileSize() is based on this sentinel. Casting to void silences the warning as is standard practice and hopefully indicates better that this is intended than ignoring the warning for the casual on-looker.
* | Italian program translation updateLuca Vercelli2024-11-221-312/+301
| | | | | | | | Closes: #1081992
* | Russian program translation updateАлексей Шилин2024-11-221-74/+64
| | | | | | | | Closes: #1087804
* | Dutch manpages translation updateFrans Spiesschaert2024-11-221-23/+10
| | | | | | | | Closes: #1087461
* | Dutch program translation updateFrans Spiesschaert2024-11-221-4/+4
|/ | | | Closes: #1087458
* test: Do not fail if valgrind is not availableJulian Andres Klode2024-11-211-2/+1
| | | | | | | | | | | | The command -v check inadvertently exited 0 because it confusingly also considers functions and aliases. Check if we are able to run a valgrind binary instead by querying its version. Then, the msgdebug in the alternative code path was included in the output that we are asserting so that was causing failures too and needs removing. Everything is horrible :D
* Release 2.9.132.9.13Julian Andres Klode2024-11-2015-16/+165
|
* tests: Temporarily disable valgrind-if-available on armhf and ppc64elJulian Andres Klode2024-11-201-1/+1
|
* Merge branch 'ftparchive-buffers' into 'main'Julian Andres Klode2024-11-204-28/+97
|\ | | | | | | | | ftparchive: Remove fixed-size buffers See merge request apt-team/apt!388
| * ftparchive: writer: Remove line length limit on file listsJulian Andres Klode2024-11-202-21/+36
| | | | | | | | LP: #2087848
| * ftparchive: cachedb: Remove 600-byte key limitJulian Andres Klode2024-11-202-3/+36
| | | | | | | | | | | | | | | | | | | | Instead of snprintf() into a fixed-size buffer, construct an std::string using assign() and append(). A previous version used strprintf() which would have been hilariously inefficient. LP: #2087848
| * ftparchive: override: Remove fixed-size line buffersJulian Andres Klode2024-11-182-5/+26
| | | | | | | | LP: #2087848
* | Merge branch 'history-comment' into 'main'Julian Andres Klode2024-11-185-0/+50
|\ \ | | | | | | | | | | | | Add a --comment option to record Comment: in history See merge request apt-team/apt!401
| * | Add a --comment option to record Comment: in historyJulian Andres Klode2024-11-185-0/+50
| |/ | | | | | | Suggested-by: mhoye on IRC
* | Merge branch 'glibcxx-assertions' into 'main'Julian Andres Klode2024-11-182-1/+18
|\ \ | |/ |/| | | | | patterns: Safely peek ahead one byte, and enable -Wp,-D_GLIBCXX_ASSERTIONS See merge request apt-team/apt!400
| * Add -Wp,-D_GLIBCXX_ASSERTIONS to CXXFLAGSJulian Andres Klode2024-11-181-0/+1
| | | | | | | | | | | | This ensures some more hardening of all our container accesses, preventing out of bounds. Especially now that we moved a lot of C-style arrays to std::array.
| * patterns: Safely peek ahead one byteJulian Andres Klode2024-11-181-1/+17
|/ | | | | | | | | | | | | | | The pattern tree parser was written using string_views in a no-copy fashion. However, at various places we assume that we can index one byte after the end of the string_view and get a NUL-byte there. This works fine in practice because we only ever pass it zero-terminated string_views, but it fails if you compile with -Wp,-D_GLIBCXX_ASSERTIONS because it will do bounds checks. So instead, do our own bounds checks here and always return '\0' if requesting one byte too many. Reported-by: Christian Heusel <christian@heusel.eu>
* Release 2.9.122.9.12Julian Andres Klode2024-11-185-7/+23
|
* Handle Retry-After by defaultJulian Andres Klode2024-11-181-1/+1
| | | | | | | | | | | | | | Let's get some feedback here. This is capped to 30s per retry, so a maximum of 90s per item that we delay the failure case. In the worst case, we hence have 90n seconds delay for n items. In a more realistic worse case however, the server likely does not collapse per-item, and we end up trying all items in a short burst so it seems more realistic we hit a 90s total delay. The bursty behavior may be suboptimal; we don't quite know, but it seems the specification of Retry-After is per-entity rather than per-host, so technically speaking it seems correct.
* Merge branch 'fm/apt-rate-limit' into 'main'Julian Andres Klode2024-11-188-6/+74
|\ | | | | | | | | http: use Retry-After HTTP header to optimize retries See merge request apt-team/apt!383
| * http: add test for Retry-After handlingFelix Moessbauer2024-11-182-0/+15
| | | | | | | | | | | | | | | | | | | | When enabling Acquire::Retries::HandleRetryAfter, the value of the Retry-After http header is honored and the retry is not executed earlier than this point in time (except if that is longer than the max delay time). The added test checks if the request is delayed at least by the requested time. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
| * http: use Retry-After HTTP header to optimize retriesFelix Moessbauer2024-11-186-6/+59
|/ | | | | | | | | | | | | | | | | | | Some mirrors like snapshot.debian.org apply strict http rate limits. While apt already has an exponential backoff mechanism implemented, this is not sufficient due to the following reasons: 1. all retries happen roughly at the same time 2. the retry-after information from the server is not used We fix this by improving the algorithm: First, if present, the timestamp or duration of the Retry-After header is added to the exponential backoff, optimizing for success-on-second-try. Second, a random delay is added (delay += [0, delay]) to distribute the retries to not immediately run into the limit again. To avoid super-long delays, this is capped by the configured maximum delay. This logic only becomes active if the option Acquire::Retries::HandleRetryAfter is enabled. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
* string_view: Include missing headersJulian Andres Klode2024-11-171-0/+3
| | | | | | | | | Specifically the lack of the `string_view` include is causing compilation failures for rust-rust-apt. Running iwyu also adds `algorithm` for our use of `std::min`, and `iosfwd` for our use of `std::ostream` none of which seem to be causing build failures so far.
* Merge branch 'reefer' into 'main'Julian Andres Klode2024-11-147-61/+50
|\ | | | | | | | | DEFER See merge request apt-team/apt!392
| * fdopen() failing does not close the fdнаб2024-11-141-0/+2
| |
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-10-14 02:04:22 +0000