Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=2.3.0 2021-03-07T01:55:07Z 2021-03-07T01:55:07Z David Kalnischkies david@kalnischkies.de 2021-03-06T15:11:34Z urn:sha1:2a81f98b124d8fe551b160df55db1d3bf79a77c1 The mirror method can distribute requests for files based on various metadata bits, but some – the main index files – weren't actually passing those on to the methods as advertised in the manpage. This is hidden both by mirror usually falling back to other sources which will eventually hit the right one and that if the repository does not support by-hash apt will automatically stick to the mirror which was used for the Release file. 2021-03-07T01:55:07Z David Kalnischkies david@kalnischkies.de 2021-03-06T23:47:26Z urn:sha1:59933938f51105066161a6eb88253006826336a2 Especially in small sections of an archive it can happen that an index returns to a previous state (e.g. if a package was first added and then removed with no other changes happening in between). The result is that we have multiple patches which start from the same hash which if we perform clientside merging is no problem although not ideal as we perform needless work. For serverside merging it would not matter, but due to rred previously refusing to merge zero-size patches but dak ignoring failure letting it carry these size-zero patches until they naturally expire we run into a problem as these broken patches won't do and force us to fall back to downloading the entire index. By always starting from the last patch instead of the first with the starter hash we can avoid this problem and behave optimally in clientside merge cases, too. 2021-03-07T01:55:07Z David Kalnischkies david@kalnischkies.de 2021-03-06T18:55:09Z urn:sha1:246f66561e23911b9615bd337b3b6f6f25b6cd31 The rred method expects the patches to have a certain name, which we have to rename the file to before calling the method, but by delaying the rename we ensure that if the download of one of them fails and a successful fallback occurs they are all properly cleaned up as no longer useful while in the error case the next apt run can potentially pick them up as already downloaded. Our test-pdiff-usage test was encountering this every other run, but did not fail as the check for unaccounted files in partial/ was wrapped in a subshell so that the failure produced failing output, but did not change the exit code. 2021-02-04T10:00:00Z David Kalnischkies david@kalnischkies.de 2021-02-03T20:47:00Z urn:sha1:c31ba27ecf7e35e03af34c3d74d3c6c93976f89c APT tries to detect if applying patches is more costly than just downloading the complete index by combining the size of the patches. That is correct for client-side merging, but for server-side merging we actually don't know if we will jump directly from old to current or have still intermediate steps in between. With this commit we assume it will be a jump from old to current through as that is what dak implements and it seems reasonable if you go to the trouble of server side merging that the server does the entire merging in one file instead of leaving additional work for the client to do. Note that this just changes the heuristic to prevent apt from discarding patches as uneconomic in the now more common one merged-patch style, it still supports multiple merged patches as before. To resolve this cleanly we would need another field in the index file declaring which hash we will arrive at if a patch is applied (or a field differentiating between these merged-patch styles at least), but that seems like overkill for now. 2020-12-18T19:45:35Z David Kalnischkies david@kalnischkies.de 2020-07-10T18:19:31Z urn:sha1:a5859bafdaa6bcf12934d0fb1715a5940965e13a This commit potentially breaks code feeding apt an encoded URI using a method which does not get URIs send encoded. The webserverconfig requests in our tests are an example for this – but they only worked before if the server was expecting a double encoding as that was what was happening to an encoded URI: so unlikely to work as expected in practice. Now with the new methods we can drop this double encoding and rely on the URI being passed properly (and without modification) between the layers so that passing in encoded URIs should now work correctly. 2020-12-18T18:31:19Z David Kalnischkies david@kalnischkies.de 2020-07-09T14:38:49Z urn:sha1:e6c55283d235aa9404395d30f2db891f36995c49 We do not deal a lot with URIs which need encoding, but then we do it is a pain that we store it decoded in the acquire system as it means we have to decode and reencode URIs eventually which is potentially giving us slightly different URIs. We see that in our own testing framework while setting up redirects as the config options are effectively double-encoded and decoded to pass them around successfully as otherwise %2f and / in an URI are treated the same. This commit adds the infrastructure for methods to opt into getting URIs send in encoded form (and returning them to us in encoded form, too) so that we eventually do not have to touch the URIs which is how it should be. This means though that we have to deal with methods who do not support this yet (aka: all at the moment) for which we decode and encode while communicating with them. 2020-08-10T13:39:33Z Julian Andres Klode julian.klode@canonical.com 2020-08-10T13:39:33Z urn:sha1:64b45e294f0c6931a9b57ae6cc99ecded8f6a2d3 Closes: #931566 2020-02-26T13:10:47Z Julian Andres Klode julian.klode@canonical.com 2020-02-26T13:04:51Z urn:sha1:da18eb10188a22fc1698a9b8466272f2826447db 2020-02-26T12:55:38Z Julian Andres Klode julian.klode@canonical.com 2020-02-26T12:55:38Z urn:sha1:e4e9af454db78c287ca062ac7d75bdd63bd7f744 2020-02-16T11:46:09Z Julian Andres Klode julian.klode@canonical.com 2020-02-16T10:45:05Z urn:sha1:9cc0e2cab7c83ede99e21c70f248d884b8930983 This experiment did not turn out sensibly, as some servers do not accept credentials when none are expected and fail, so you cannot mirror such a repository. This reverts commit c2b9b0489538fed4770515bd8853a960b13a2618.