Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.1.exp13 2015-09-15T08:16:09Z 2015-09-15T08:16:09Z David Kalnischkies david@kalnischkies.de 2015-09-14T22:33:12Z urn:sha1:6c0765c096ffb4df14169236c865bbb2b10974ae This allows running tests in parallel. Git-Dch: Ignore 2015-06-09T10:57:36Z David Kalnischkies david@kalnischkies.de 2015-06-08T13:22:01Z urn:sha1:8d041b4f4f353079268039dcbfd8b5e575196b66 If we have a file on disk and the hashes are the same in the new Release file and the old one we have on disk we know that if we ask the server for the file, we will at best get an IMS hit – at worse the server doesn't support this and sends us the (unchanged) file and we have to run all our checks on it again for nothing. So, we can save ourselves (and the servers) some unneeded requests if we figure this out on our own. 2015-06-09T10:57:35Z David Kalnischkies david@kalnischkies.de 2015-06-06T10:28:00Z urn:sha1:448c38bdcd72b52f11ec5f326f822cf57653f81c Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed(). 2015-05-13T14:09:12Z David Kalnischkies david@kalnischkies.de 2015-05-13T14:09:12Z urn:sha1:8eafc759544298211cd0bfaa3919afc0fadd47d1 Not all servers we are talking to support If-Modified-Since and some are not even sending Last-Modified for us, so in an effort to detect such hits we run a hashsum check on the 'old' compared to the 'new' file, we got the hashes for the 'new' already for "free" from the methods anyway and hence just need to calculated the old ones. This allows us to detect hits even with unsupported servers, which in turn means we benefit from all the new hit behavior also here. 2015-05-11T15:22:32Z David Kalnischkies david@kalnischkies.de 2015-04-27T08:59:27Z urn:sha1:146f7715a9f36d246b461255b3c683b479296915 Especially pdiff-enhanced downloads have the tendency to fail for various reasons from which we can recover and even a successful download used to leave the old unpatched index in partial/. By adding a new method responsible for making the transaction of an individual file happen we can at specialisations especially for abort cases to deal with the cleanup. This also helps in keeping the compressed indexes around if another index failed instead of keeping the decompressed files, which we wouldn't pick up in the next call. 2015-04-18T23:13:10Z David Kalnischkies david@kalnischkies.de 2015-04-12T15:08:46Z urn:sha1:ba6b79bd0090077724fa1272ea4d3a31706fcd5a If we get a IMSHit for the Transaction-Manager (= the InRelease file or as its still supported fallback Release + Release.gpg combo) we can assume that every file we would queue based on this manager, but already have locally is current and hence would get an IMSHit, too. We therefore save us and the server the trouble and skip the queuing in this case. Beside speeding up repetative executions of 'apt-get update' this way we also avoid hitting hashsum errors if the indexes are in fact already updated, but the Release file isn't yet as it is the case on well behaving mirrors as Release files is updated last. The implementation is a bit harder than the theory makes it sound as we still have to keep reverifying the Release files (e.g. to detect now expired once to avoid an attacker being able to silently stale us) and have to handle cases in which the Release file hits, but some indexes aren't present (e.g. user added a new foreign architecture). 2015-04-18T23:13:08Z David Kalnischkies david@kalnischkies.de 2015-03-27T14:53:43Z urn:sha1:27925d82dd0cbae74d48040363fe6f6c2bae5215 The worker expects that the methods tell him when they start or finish downloading a file. Various information pieces are passed along in this report including the (expected) filesize. https was using a "global" struct for reporting which made it 'reuse' incorrect values in some cases like a non-existent InRelease fallbacking to Release{,.gpg} resulting in a size-mismatch warning. Reducing the scope and redesigning the setting of the values we can fix this and related issues. Closes: 777565, 781509 Thanks: Robert Edmonds and Anders Kaseorg for initial patchs 2015-03-16T17:01:54Z David Kalnischkies david@kalnischkies.de 2015-03-09T23:59:44Z urn:sha1:25b86db159fbc3c043628e285c0c1ef24dec2c6e We use test{success,failure} now all over the place in the framework, so its only consequencial to do this in the situations in which we test for a specific output as well. Git-Dch: Ignore 2014-10-20T08:37:46Z David Kalnischkies david@kalnischkies.de 2014-10-20T08:23:41Z urn:sha1:4fa34122cbe347d21b3a162ff2fa75dd2e73c3a8 Adds a new testwarning which tests for zero exit and the presents of a warning in the output, failing if either is not the case or if an error is found, too. This allows us to change testsuccess to accept only totally successful executions (= without warnings) which should help finding regressions. Git-Dch: Ignore 2014-10-20T08:37:46Z David Kalnischkies david@kalnischkies.de 2014-10-19T12:14:37Z urn:sha1:1df24acfdb8ba1cd8bbbaa166f170dda480ce41e These functions check the exit code of the command, but for apt commands we can go further and require an error message for non-zero exits and none for zero exits. Git-Dch: Ignore