Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.3.1 2016-08-16T17:20:28Z 2016-08-16T17:20:28Z David Kalnischkies david@kalnischkies.de 2016-08-12T09:05:58Z urn:sha1:9714d522056e5256f5a2de587d88eba7cb3291c2 If a server closes a connection after sending us a file that tends to mean that its a type of server who always closes the connection – it is therefore relatively pointless to try pipelining with it even if it isn't a problem by itself: apt is just restarting the pipeline each time after it got served one file and the connection is closed. The problem starts if one or more proxies are between the server and apt and they disagree about how the connection should be as in the bugreporters case where the responses apt gets contain both Keep-Alive and Proxy-Connection headers (which apt both ignores) indicating a proxy is trying to keep a connection open while the response also contains "Connection: close" indicating the opposite which apt understands and respects as it is required to do. We avoid stepping into this abyss by not performing pipelining anymore if we got a respond with the indication to close connection if the response was otherwise a success – error messages are sent by some servers via this method as their pages tend to be created dynamically and hence their size isn't known a priori to them. Closes: #832113 2016-08-16T16:49:37Z David Kalnischkies david@kalnischkies.de 2016-08-11T16:24:35Z urn:sha1:d94b1d80d8326334d17f6a43061368e783b8e0aa If the server told us in a previous request that it isn't supporting Ranges with bytes via an Accept-Ranges header missing bytes, we don't try to formulate requests using Ranges. 2016-08-16T16:49:37Z David Kalnischkies david@kalnischkies.de 2016-08-11T14:59:13Z urn:sha1:ebdb6f1810a20ac240b5b2192dc2e6532ff149d2 We keep various information bits about the server around, some only effecting the currently handled file (like sizes) while others should be persistent (like pipeline detections). http used to reset all file-related manually, which is a bit silly if we already have a Reset() method – which does reset all through –, so extending it with a parameter for reuse and calling it from https too (as this was previously resetting by just creating a new state struct – it uses no value of the persistent state-keeping yet as it supports no pipelining). Gbp-Dch: Ignore 2016-08-13T06:49:35Z David Kalnischkies david@kalnischkies.de 2016-08-12T20:13:09Z urn:sha1:148c049150cc39f2e40894c1684dc2aefea1117e It seems completely pointless from a server-POV to sent empty header fields, so most of them don't do it (simply proven by this limitation existing since day one) – but it is technically allowed by the RFC as the surounding whitespaces are optional and Github seems to like sending "X-Geo-Block-List:\r\n" since recently (bug reports in other http clients indicate July) at least sometimes as the reporter claims to have seen it on https only even through it can happen with both. Closes: 834048 2016-08-10T23:34:39Z David Kalnischkies david@kalnischkies.de 2016-08-06T20:54:31Z urn:sha1:0568d325ad8660a9966d552634aa17c90ed22516 With apts http transport supporting socks5h proxies and all the work in terms of configuration of methods based on the name it is called with it becomes surprisingly easy to implement Tor support equally (and perhaps even a bit exceeding) what is available currently in apt-transport-tor. How this will turn out to be handled packaging wise we will see in https://lists.debian.org/deity/2016/08/msg00012.html , but until this is resolved we can add the needed support without actively enabling it for now, so that this can be tested better. 2016-08-10T21:19:44Z David Kalnischkies david@kalnischkies.de 2016-07-31T16:05:56Z urn:sha1:30060442025824c491f58887ca7369f3c572fa57 The https method implemented for a long while now a hardcoded fallback to the same options in http, which, while it works, is rather inflexible if we want to allow the methods to use another name to change their behavior slightly, like apt-transport-tor does to https – most of the diff being s#https#tor#g which then fails to do the full circle fallthrough tor -> https -> http for https sources. With this config infrastructure this could be implemented now. 2016-08-10T21:19:44Z David Kalnischkies david@kalnischkies.de 2016-08-02T12:49:58Z urn:sha1:4bba5a88d0f6afde4414b586b64c48a4851d5324 cURL which backs our https implementation can handle redirects on its own, but by dealing with them on our own we gain finer control over which redirections will be performed (we don't like https → http) and by whom so that redirections to other hosts correctly spawn a new https method dealing with these instead of letting the current one deal with it. 2016-08-10T21:19:44Z David Kalnischkies david@kalnischkies.de 2016-08-02T20:44:50Z urn:sha1:57401c48fadc0c78733a67294f9cc20a57e527c9 Having the detection handled in specific (http) workers means that a redirection loop over different hostnames isn't detected. Its also not a good idea have this implement in each method independently even if it would work 2016-07-27T11:25:18Z David Kalnischkies david@kalnischkies.de 2016-07-27T11:25:18Z urn:sha1:353b7bab08704cd2f7e2b6951c9dcd7cf3023e3a The rewrite in 742f67eaede80d2f9b3631d8697ebd63b8f95427 is based on the assumption that the pipeline will always be at least one item short each time it is called, but the logs in #832113 suggest that this isn't always the case. I fail to see how at the moment, but the old implementation had this behavior, so restoring it can't really hurt, can it? 2016-07-27T07:23:18Z David Kalnischkies david@kalnischkies.de 2016-07-27T06:08:50Z urn:sha1:b9c20219dc17db1d29eaf297263a4b008bd1b90b Also fixes message itself to mention the correct option name as noticed in #832113.