Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.7.0_alpha3 2018-08-20T16:29:16Z 2018-08-20T16:29:16Z David Kalnischkies david@kalnischkies.de 2018-08-18T15:32:04Z urn:sha1:29658a3a74af49e2a24e17bdebb20e1612aac3ec APT in 1.6 saw me rewriting the mirror:// transport method, which works comparable to the decommissioned httpredir.d.o "just" that apt requests a mirror list and performs all the redirections internally with all the bells like parallel download and automatic fallback (more details in the apt-transport-mirror manpage included in the 1.6 release). The automatic fallback is the problem here: The intend is that if a file fails to be downloaded (e.g. because the mirror is offline, broken, out-of-sync, …) instead of erroring out the next mirror in the list is contacted for a retry of the download. Internally the acquire process of an InRelease file (works with the Release/Release.gpg pair, too) happens in steps: 1) download file and 2) verify file, both handled as URL requests passed around. Due to an oversight the fallbacks for the first step are still active for the second step, so that the successful download from another mirror stands in for the failed verification… *facepalm* Note that the attacker can not judge by the request arriving for the InRelease file if the user is using the mirror method or not. If entire traffic is observed Eve might be able to observe the request for a mirror list, but that might or might not be telling if following requests for InRelease files will be based on that list or for another sources.list entry not using mirror (Users have also the option to have the mirror list locally (via e.g. mirror+file://) instead of on a remote host). If the user isn't using mirror:// for this InRelease file apt will fail very visibly as intended. (The mirror list needs to include at least two mirrors and to work reliably the attacker needs to be able to MITM all mirrors in the list. For remotely accessed mirror lists this is no limitation as the attacker is in full control of the file in that case) Fixed by clearing the alternatives after a step completes (and moving a pimpl class further to the top to make that valid compilable code). mirror:// is at the moment the only method using this code infrastructure (for all others this set is already empty) and the only method-independent user so far is the download of deb files, but those are downloaded and verified in a single step; so there shouldn't be much opportunity for regression here even through a central code area is changed. Upgrade instructions: Given all apt-based frontends are affected, even additional restrictions like signed-by are bypassed and the attack in progress is hardly visible in the progress reporting of an update operation (the InRelease file is marked "Ign", but no fallback to "Release/Release.gpg" is happening) and leaves no trace (expect files downloaded from the attackers repository of course) the best course of action might be to change the sources.list to not use the mirror family of transports ({tor+,…}mirror{,+{http{,s},file,…}}) until a fixed version of the src:apt packages are installed. Regression-Of: 355e1aceac1dd05c4c7daf3420b09bd860fd169d, 57fa854e4cdb060e87ca265abd5a83364f9fa681 LP: #1787752 2018-08-19T15:30:31Z David Kalnischkies david@kalnischkies.de 2018-08-16T21:36:41Z urn:sha1:b934870c4f893be28eb1537910c0aadce6dd6e09 gpgs DETAILS documentation file declares that GOODSIG could report keyid or fingerprint since gpg2, but for the time being it is still keyid only. Who knows if that will ever change as that feels like an interface break with dangerous security implications, but lets be better safe than sorry especially as the code dealing with signed-by keyids is prepared for this already. This code is rewritten still to have them all use the same code for this type of problem. 2018-08-19T15:25:43Z David Kalnischkies david@kalnischkies.de 2018-08-14T22:23:37Z urn:sha1:6df0d31d60d6ad0b706b15d57bd433d036cce82f The 1.7 series rework of show started in bf53f39c9a0221b670ffff74053ed36fc502d5a0 resolved the issue already, but its always a good idea to at least bring the tests along so that we hopeful do not regress in the future with another rewrite. Tests: #905527 Gbp-Dch: Ignore 2018-06-27T13:09:45Z Julian Andres Klode julian.klode@canonical.com 2018-06-27T09:31:21Z urn:sha1:1d53cffad22c92645090e0e6ddde31fe4f7c3b05 JSON hooks might disappear and the common idiom to work around hooks disappearing is to check for the hook in the shell snippet that is in the apt.conf file and if it does not exist, do nothing. This caused APT to fail however, expecting it to acknowledge the handshake. Ignoring ECONNRESET on handshakes solves the problem. The error case, and the other error cases also did not stop execution of the hook, causing more errors to pile up. Fix this by directly going to the closing part of the code. LP: #1776218 2018-05-28T15:57:25Z David Kalnischkies david@kalnischkies.de 2018-05-28T15:02:17Z urn:sha1:484babb7d00f7550cbaa592b7cb0022d38217fad Commit d7c92411dc1f4c6be098d1425f9c1c075e0c2154 introduced a warning for non-existent files from components not mentioned in Components to hint users at a mispelling or the disappearance of a component. The debian-installer subcomponent isn't actively advertised in the Release file through, so if apt ends up in acquiring a file which doesn't exist for this component (like Translation files) apt would produce a warning: W: Skipping acquire of configured file 'main/debian-installer/i18n/Translation-en' as repository 'http://deb.debian.org/debian buster InRelease' doesn't have the component 'main/debian-installer' (component misspelt in sources.list?) We prevent this in the future by checking if any file exists from this component which results in the warning to be produced still for the intended cases and silence it on the d-i case. This could potentially cause the warning not to be produced in cases it should be if some marginal file remains, but as this message is just a hint and the setup a bit pathological lets ignore it for now. There is also the possibility of having no file present as they would all be 0-length files and being a "hidden" component, but that would be easy to workaround from the repository side and isn't really actively used at the moment in the wild. Closes: #879591 2018-05-28T15:57:20Z David Kalnischkies david@kalnischkies.de 2018-05-26T14:28:03Z urn:sha1:c94988e75ea923afe0daae5747f974927325b186 Hardcoding the IPv4 address 127.0.0.1 stops stunnel4 from also binding on IPv6 as well which not only binds on another port but confuses our crude port extraction by splitting on ':' with ::1. Gbp-Dch: Ignore 2018-05-24T13:04:22Z Julian Andres Klode jak@debian.org 2018-05-24T13:04:22Z urn:sha1:6eaeec549241677335813af78f394010e5b3eefb more volatile: build-dep foo.deb/release & show foo.deb See merge request apt-team/apt!14 2018-05-24T12:54:34Z Julian Andres Klode jak@debian.org 2018-05-24T12:54:34Z urn:sha1:b367f2d393fa95a386feb03a64d4bf5e32cc28a4 Don't force the same mirror for by-hash URIs See merge request apt-team/apt!15 2018-05-21T16:19:33Z Filipe Brandenburger filbranden@google.com 2018-05-16T05:24:29Z urn:sha1:5d263a39d6106f6d20b2b447ffc14ef0c096ea22 Run `apt-get autoremove -o Debug::pkgAutoRemove=yes` and confirm the logged reason for packages to be kept is correct. Only check for specific debug lines containing 'MarkPackage:' in order to prevent new debug logging to break the test case. 2018-05-19T19:39:08Z David Kalnischkies david@kalnischkies.de 2018-05-19T19:10:17Z urn:sha1:ea901a491bcfe406267c7c38c227c5caabf017a0 id: '': no such user ./test-bug-611729-mark-as-manual: 59: [: Illegal number: Regression-of: 68842e1741a5005b1e3f0a07deffd737c65e3294 Gbp-Dch: Ignore