apt/apt-pkg/acquire-worker.cc, branch 2.9.13

Handle Retry-After by default

2024-11-18T13:54:19Z

Let's get some feedback here. This is capped to 30s per retry, so a maximum of 90s per item that we delay the failure case. In the worst case, we hence have 90n seconds delay for n items. In a more realistic worse case however, the server likely does not collapse per-item, and we end up trying all items in a short burst so it seems more realistic we hit a 90s total delay. The bursty behavior may be suboptimal; we don't quite know, but it seems the specification of Retry-After is per-entity rather than per-host, so technically speaking it seems correct.

http: use Retry-After HTTP header to optimize retries

2024-11-18T12:44:49Z

Some mirrors like snapshot.debian.org apply strict http rate limits. While apt already has an exponential backoff mechanism implemented, this is not sufficient due to the following reasons: 1. all retries happen roughly at the same time 2. the retry-after information from the server is not used We fix this by improving the algorithm: First, if present, the timestamp or duration of the Retry-After header is added to the exponential backoff, optimizing for success-on-second-try. Second, a random delay is added (delay += [0, delay]) to distribute the retries to not immediately run into the limit again. To avoid super-long delays, this is capped by the configured maximum delay. This logic only becomes active if the option Acquire::Retries::HandleRetryAfter is enabled. Signed-off-by: Felix Moessbauer

Remove ftp, rsh, ssh methods (disabled since 1.8)

2024-11-07T13:04:30Z

These methods have been unsupported and disabled since 1.8, remove them for the 3.0 cleanup. Please migrate to http instead. If you need ad-hoc access to a remote repository, you can run `python3 -m http.server` on that machine and use SSH port forwarding to run http over ssh.

worker: Add an audit level to log audit messages

2024-07-15T17:16:27Z

Modernize standard library includes

2024-02-20T12:49:04Z

This was automated with sed and git-clang-format, and then I had to fix up the top of policy.cc by hand as git-clang-format accidentally indented it by two spaces.

Add support for a maximum delay and testing of delay

2021-07-28T11:08:35Z

This is very basic support on the testing side, we just test the debug output but not how long it actually took. Would be nice to check time really.

Convert to monotonic clock

2021-07-28T11:08:35Z

This yields more accurate delays and avoids issues with clock skew.

Implement exponential delay between retries

2021-07-28T11:04:10Z

Add a new Item field called FetchAfter, which determines the earliest time the item should be fetched at. Adjust insertion into queue to take it into account alongside priority, and only fill pipelines with items that are ready.

Keep URIs encoded in the acquire system

2020-12-18T18:31:19Z

We do not deal a lot with URIs which need encoding, but then we do it is a pain that we store it decoded in the acquire system as it means we have to decode and reencode URIs eventually which is potentially giving us slightly different URIs. We see that in our own testing framework while setting up redirects as the config options are effectively double-encoded and decoded to pass them around successfully as otherwise %2f and / in an URI are treated the same. This commit adds the infrastructure for methods to opt into getting URIs send in encoded form (and returning them to us in encoded form, too) so that we eventually do not have to touch the URIs which is how it should be. This means though that we have to deal with methods who do not support this yet (aka: all at the moment) for which we decode and encode while communicating with them.

Drop pkgAcquire::Item::ModifyRetries() ABI hack

2020-02-26T13:10:47Z