Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=main 2024-03-30T15:59:57Z 2024-03-30T15:59:57Z David Kalnischkies david@kalnischkies.de 2024-03-20T12:30:19Z urn:sha1:e62162d010fc7d6374067964ced3ac227b0440b2 We don't store .diff_Index files anymore and so libapt cares even less about these purposefully leftover files from the testcases than it did previously. On a successful apt run they would just be deleted, but as we are testing a failed run they are not touched at all. Testing the file access bits then means we check with whatever umask they were created which might very well be different to what apt decides these files to have if it had touched them, so for this test we just delete them. For the other case we set it completely wrong just in case, but they will (hopefully) be non-existent anyhow as tested first. References: afcdbcf895284efd76903b2b3ba5cc849059ce50 2024-01-03T16:59:11Z David Kalnischkies david@kalnischkies.de 2024-01-03T16:25:35Z urn:sha1:afcdbcf895284efd76903b2b3ba5cc849059ce50 Nowadays we only download the index file if we have a non-current file on disk which we want to patch. If that is the case, any index file for patches we could have stored is by definition outdated, so storing those files just takes up disk space. At least, that is the case if we have a Release file – if we don't this commit introduces a needless redownload for such repositories but such repositories are an error by default and if they can't be bothered to provide a Release file its very unlikely they actually ship diffs, so adding detection code for this seems pointless at best. 2021-03-11T13:05:41Z David Kalnischkies david@kalnischkies.de 2021-03-11T12:38:53Z urn:sha1:0d25ce3d466ecddea02d171981f011f7dbf95e08 If a transaction is doomed we want to gracefully shutdown our zoo of worker processes. As explained in the referenced commit we do this by stopping the main process from handing out new work and ignoring the replies it gets from the workers, so that they eventually run out of work. We tested this previously by checking if a rred worker was given work items at all, but depending on how lucky the stars of the machine working on this are the worker would have already gotten work before the transaction was aborted – so we tried this 25 times a row (f35601e5d2). No machine can be this lucky, right? Turns out the autopkgtest armhf machine is very lucky. I feel a bit sorry for feeding grep such a long "line" to work with, but it seems to work out. Porterbox amdahl (who is considerably less lucky; had to turn down to 1 try to get it to fail sometimes) is now happily running the test in an endless loop. Of course, I could have broken the test now, but its still a rather generic grep (in some ways more generic even) and the main part of the testcase – the update process finishes and fails – is untouched. References: 38f8704e419ed93f433129e20df5611df6652620 Closes: #984966 2021-03-07T01:55:07Z David Kalnischkies david@kalnischkies.de 2021-03-06T23:47:26Z urn:sha1:59933938f51105066161a6eb88253006826336a2 Especially in small sections of an archive it can happen that an index returns to a previous state (e.g. if a package was first added and then removed with no other changes happening in between). The result is that we have multiple patches which start from the same hash which if we perform clientside merging is no problem although not ideal as we perform needless work. For serverside merging it would not matter, but due to rred previously refusing to merge zero-size patches but dak ignoring failure letting it carry these size-zero patches until they naturally expire we run into a problem as these broken patches won't do and force us to fall back to downloading the entire index. By always starting from the last patch instead of the first with the starter hash we can avoid this problem and behave optimally in clientside merge cases, too. 2021-03-07T01:55:07Z David Kalnischkies david@kalnischkies.de 2021-03-06T18:55:09Z urn:sha1:246f66561e23911b9615bd337b3b6f6f25b6cd31 The rred method expects the patches to have a certain name, which we have to rename the file to before calling the method, but by delaying the rename we ensure that if the download of one of them fails and a successful fallback occurs they are all properly cleaned up as no longer useful while in the error case the next apt run can potentially pick them up as already downloaded. Our test-pdiff-usage test was encountering this every other run, but did not fail as the check for unaccounted files in partial/ was wrapped in a subshell so that the failure produced failing output, but did not change the exit code. 2018-12-03T16:21:39Z Julian Andres Klode julian.klode@canonical.com 2018-12-03T08:16:20Z urn:sha1:f35601e5d2b9fe8b99c6178cb9b160f1a42f432f Try 10 times in a row 2018-05-11T16:28:29Z David Kalnischkies david@kalnischkies.de 2018-01-15T11:04:45Z urn:sha1:ed0e9eadeb3003f4f150da3c463b28cfa5e54b6f Individual items shouldn't concern themselves with these alternative locations, we can deal with this more efficiently within the infrastructure created for other alternative URIs now avoiding the need to implement this in each item. 2018-05-11T16:28:19Z David Kalnischkies david@kalnischkies.de 2018-01-13T23:07:20Z urn:sha1:2d6c6c8809c7b4c1a009df48387ba15066fda7e2 If we got a file but it produced a hash error, mismatched size or similar we shouldn't fallback to alternative URIs as they likely result in the same error. If we can we should instead use another mirror. We used to be a lot stricter by stopping all trys for this file if we got a non-404 (or a hash-based) failure, but that is too hard as we really want to try other mirrors (if we have them) in the hope that they have the expected and correct files. 2018-01-03T17:55:41Z David Kalnischkies david@kalnischkies.de 2017-08-09T21:26:19Z urn:sha1:02567e3084d2faec92e8bf248e89fda6452e634b The format isn't too hard to get right, but it gets funny with multiline fields (which we don't really have yet) and its just easier to deal with it once and for all which can be reused for more messages later. 2017-07-26T17:07:56Z David Kalnischkies david@kalnischkies.de 2017-07-26T16:35:42Z urn:sha1:f2f8e89f08cdf01c83a0b8ab053c65329d85ca90 Failing on too much data is good, but we can do better by checking for exact filesizes as we know with hashsums how large a file should be, so if we get a file which has a size we do not expect we can drop it directly, regardless of if the file is larger or smaller than what we expect which should catch most cases which would end up as hashsum errors later now a lot sooner.