apt/test, branch 1.7.0_rc1

Merge branch 'bugfix/statusfd' into 'master'

2018-09-18T12:05:20Z

Process all of --status-fd and don't expect duplicate status msg See merge request apt-team/apt!26

Show all architectures in 'apt list' output

2018-09-15T15:45:16Z

The uniqueness in std::set containers is ensured by the ordering operator we provide, but it was not considering that different versions can have the same description like the different architectures for a version of a package. Closes: #908218

Reorder progress report messages

2018-09-11T11:08:58Z

We are seeing 'processing' messages from dpkg first, so it makes sense to translate them to "Preparing" messages instead of using "Installing" and co to override these shortly after with the "Preparing" messages. The difference isn't all to visible as later messages tend to linger far longer in the display than the ealier ones, but at least in a listing it seems more logical.

Don't expect duplicated dpkg status-fd messages

2018-09-11T11:08:58Z

The progress reporting relies on parsing the status reports of dpkg which used to repeat being in the same state multiple times in the same run, but by fixing #365921 it will stop doing so. The problem is in theory just with 'config-files' in case we do purge as this (can) do remove + purge in one step, but we remove this also for the unpack + configure combination althrough we handle these currently in two independent dpkg calls.

Unset more environment variables in test framework

2018-09-11T10:59:01Z

It is an uphill battle to "reset" the environment to a clean state without making it needlessly hard to use 'good' environment variables, so we just try a little harder here without really trying for completeness. Gbp-Dch: Ignore

Don't use gpg directly in apt-key test

2018-09-10T19:57:51Z

Reported-By: Guillem Jover Gbp-Dch: Ignore

Fix typos reported by codespell & spellintian

2018-08-29T15:51:43Z

No user-visible change as it effects mostly code comments and not a single error message, manpage or similar. Reported-By: codespell & spellintian Gbp-Dch: Ignore

Don't use invalid iterator in Fallback-Of handling

2018-08-29T15:50:31Z

cppcheck reports: (error) Iterator 't' used after element has been erased. The loop is actually fashioned to deal with this (not in the most efficient way, but in simplest and speed isn't really a concern here) IF this codepath had a "break" at the end… so I added one. Note that the tests aren't failing before (and hopefully after) the change as the undefined behavior we encounter is too stable. Thanks: David Binderman for reporting

clear alternative URIs for mirror:// between steps (CVE-2018-0501)

2018-08-20T16:29:16Z

APT in 1.6 saw me rewriting the mirror:// transport method, which works comparable to the decommissioned httpredir.d.o "just" that apt requests a mirror list and performs all the redirections internally with all the bells like parallel download and automatic fallback (more details in the apt-transport-mirror manpage included in the 1.6 release). The automatic fallback is the problem here: The intend is that if a file fails to be downloaded (e.g. because the mirror is offline, broken, out-of-sync, …) instead of erroring out the next mirror in the list is contacted for a retry of the download. Internally the acquire process of an InRelease file (works with the Release/Release.gpg pair, too) happens in steps: 1) download file and 2) verify file, both handled as URL requests passed around. Due to an oversight the fallbacks for the first step are still active for the second step, so that the successful download from another mirror stands in for the failed verification… *facepalm* Note that the attacker can not judge by the request arriving for the InRelease file if the user is using the mirror method or not. If entire traffic is observed Eve might be able to observe the request for a mirror list, but that might or might not be telling if following requests for InRelease files will be based on that list or for another sources.list entry not using mirror (Users have also the option to have the mirror list locally (via e.g. mirror+file://) instead of on a remote host). If the user isn't using mirror:// for this InRelease file apt will fail very visibly as intended. (The mirror list needs to include at least two mirrors and to work reliably the attacker needs to be able to MITM all mirrors in the list. For remotely accessed mirror lists this is no limitation as the attacker is in full control of the file in that case) Fixed by clearing the alternatives after a step completes (and moving a pimpl class further to the top to make that valid compilable code). mirror:// is at the moment the only method using this code infrastructure (for all others this set is already empty) and the only method-independent user so far is the download of deb files, but those are downloaded and verified in a single step; so there shouldn't be much opportunity for regression here even through a central code area is changed. Upgrade instructions: Given all apt-based frontends are affected, even additional restrictions like signed-by are bypassed and the attack in progress is hardly visible in the progress reporting of an update operation (the InRelease file is marked "Ign", but no fallback to "Release/Release.gpg" is happening) and leaves no trace (expect files downloaded from the attackers repository of course) the best course of action might be to change the sources.list to not use the mirror family of transports ({tor+,…}mirror{,+{http{,s},file,…}}) until a fixed version of the src:apt packages are installed. Regression-Of: 355e1aceac1dd05c4c7daf3420b09bd860fd169d, 57fa854e4cdb060e87ca265abd5a83364f9fa681 LP: #1787752

Report (soon) worthless keys if gpg uses fpr for GOODSIG

2018-08-19T15:30:31Z

gpgs DETAILS documentation file declares that GOODSIG could report keyid or fingerprint since gpg2, but for the time being it is still keyid only. Who knows if that will ever change as that feels like an interface break with dangerous security implications, but lets be better safe than sorry especially as the code dealing with signed-by keyids is prepared for this already. This code is rewritten still to have them all use the same code for this type of problem.