Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=2.2.1 2020-12-18T18:31:19Z 2020-12-18T18:31:19Z David Kalnischkies david@kalnischkies.de 2020-07-09T22:02:25Z urn:sha1:97d6c3b2d05fe0d965657197adf56cc78f9edf81 Every method opts in to getting the encoded URI passed along while keeping compat in case we are operated by an older acquire system. Effectively this is just a change for the http-based methods as the others just decode the URI as they work with files directly. 2020-08-11T11:42:41Z Julian Andres Klode julian.klode@canonical.com 2020-08-11T11:09:14Z urn:sha1:8b35e2a3dd7b863639a8909fa2361ed4fd217bc3 The old code was fairly confusing, and contradictory. Notably, the second `if` also only applied to the Data state, whereas we already terminated the Data state earlier. This was bad. The else fallback applied in three cases: (1) We reached our limit (2) We are Persistent (3) We are headers Now, it always failed as a transient error if it had nothing left in the buffer. BUT: Nothing left in the buffer is the correct thing to happen if we were fetching content. Checking all combinations for the flags, we can compare the results of Die() between 2.1.7 - the last "known-acceptable-ish" version and this version: 2.1.7 this Data !Persist !Space !Limit OK (A) OK Data !Persist !Space Limit OK (A) OK Data !Persist Space !Limit OK (C) OK Data !Persist Space Limit OK OK Data Persist !Space !Limit ERR ERR * Data Persist !Space Limit OK (B) OK Data Persist Space !Limit ERR ERR Data Persist Space Limit OK OK => Data connections are OK if they have not reached their limit, or are persistent (in which case they'll probably be chunked) Header !Persist !Space !Limit ERR ERR Header !Persist !Space Limit ERR ERR Header !Persist Space !Limit OK OK Header !Persist Space Limit OK OK Header Persist !Space !Limit ERR ERR Header Persist !Space Limit ERR ERR Header Persist Space !Limit OK OK Header Persist Space Limit OK OK => Common scheme here is that header connections are fine if they have read something into the input buffer (Space). The rest does not matter. (A) Non-persistent connections with !space always enter the else clause, hence success (B) no Space means we enter the if/else, we go with else because IsLimit(), and we succeed because we don't have space (C) Having space we do enter the while (WriteSpace()) loop, but we never reach IsLimit(), hence we fall through. Given that our connection is not persistent, we fall through to the else case, and there we win because we have data left to write. 2020-08-11T11:09:04Z Julian Andres Klode julian.klode@canonical.com 2020-08-11T08:55:09Z urn:sha1:73780d7f664a4ea1da55527d726b4c9c7753f1fb We do not want to end up in a code path while reading content from the server where we have local data left to write, which can happen if a previous read included both headers and content. Restructure Flush() to accept a new argument to allow incomplete flushs (which do not match our limit), so that it can flush as far as possible, and modify Go() and use that before and after reading from the server. 2020-08-11T11:09:04Z Julian Andres Klode julian.klode@canonical.com 2020-08-11T09:42:15Z urn:sha1:13ab2317451931f055855f1aeaec6c8b28b14ce2 This causes some more issues, really. 2020-08-11T11:09:04Z Julian Andres Klode julian.klode@canonical.com 2020-08-11T09:40:14Z urn:sha1:af7ab7c0002ef2cdfb1a4c0a468c5dbbda3d5dd0 We have successfully finished reading data if our buffer is empty, so we don't need to do any further checks. 2020-08-04T09:46:39Z Julian Andres Klode julian.klode@canonical.com 2020-08-04T09:37:45Z urn:sha1:27d36318b98f3a070fb24557ce691718ef4eec34 We only add the file to the select() call if we have data to write to it prior to the select() call. This is problematic: Assuming we enter Go() with no data to write to the file, but we read some from the server as well as an EOF, we end up not writing it to the file because we did not add the file to the select. We can't always add the file to the select(), because it's basically always ready and we don't want to wake up if we don't have anything to read or write. So for a solution, let's just always write data to the file if there's data to write to it. If some gets leftover, or if some was already present when we started Go(), it will still be added to the select() call and unblock it. Closes: #959518 2020-07-24T14:30:43Z Julian Andres Klode julian.klode@canonical.com 2020-06-29T12:03:21Z urn:sha1:08f05aa8beb58fa32485e2087eb21a9f3cb267bb Instead of reading the data early, disable the timeout for the select() call and read the data later. Also, change Read() to call only once to drain the buffer in such instances. We could optimize this to call read() multiple times if there is also pending stuff on the socket, but that it slightly more complex and should not provide any benefits. 2020-07-24T14:30:43Z Julian Andres Klode julian.klode@canonical.com 2020-06-29T10:31:55Z urn:sha1:c2cb8abbf5d8a49b25071ffffca93a083fe725fc The error handling in Die() that's supposed to add useful error messages is not super useful here. 2020-07-24T14:30:43Z Julian Andres Klode julian.klode@canonical.com 2020-06-29T10:23:02Z urn:sha1:9742032dcdc0e72c117ae0c589fbb59452d6d33c This avoids a case where we read data, then write to the server and only then realize the connection was closed. It is somewhat slower, though. 2020-07-24T14:30:43Z Julian Andres Klode julian.klode@canonical.com 2020-06-29T09:48:28Z urn:sha1:24d308455a5f8751f57219f211a5672af340099e By changing the buffer implementation to return true if it read or wrote something, even on EOF, we should not have a need to flush the buffer in Die() anymore - we should only be calling Die() if the buffer is empty now.