apt/test/integration, branch 1.4_beta4

https: Quote path in URL before passing it to curl

2017-01-17T00:59:15Z

Curl requires URLs to be urlencoded. We are however giving it undecoded URLs. This causes it go completely nuts if there is a space in the URI, producing requests like: GET /a file HTTP/1.1 which the servers then interpret as a GET request for "/a" with HTTP version "file" or some other non-sense. This works around the issue by encoding the path component of the URL. I'm not sure if we should encode other parts of the URL as well, this one seems to do the trick for the actual issue at hand. A more correct fix is to avoid the dequoting and (re-)quoting of URLs when a redirect occurs / a new request is sent. That's been on the radar for probably a year or two now, but nobody bothered implementing that yet. LP: #1651923

Read dpkg tables to handle architecture wildcards

2017-01-17T00:43:50Z

Our implementation of wildcards was rudimentary. It worked for some common ones, but it was also broken: For example, armel matched any-armel, but should match any-arm. With this commit, we load the correct tables from dpkg. Supported are both triplets and quadruplet tables (the latter introduced in dpkg 1.18.11). There are some odd things we have to deal with in the cache filter for historical and API reasons: * The character "*" must be accepted as an alternative to any - in fact it may appear anywhere in the wildcard as we also allow fnmatch() style wildcard matching on the commandline. * The code might get passed an arch with a minus at the end, for example the cmdline "install apt:any-arm-" will first try to check if any-arm- is a valid architecture. We deal with this by rejecting any wildcard ending in a minus. * Triplets are actually implemented by extending them to faux quadruplets - by prepending a "base" component for the architecture tuple, and "any" if there is a wildcard component. Once we have constructed a wildcard, it is transformed into an fnmatch() expression for historical reasons. In the future, we should really get a tuple class and implement matching in a better, more explicit way. This does for now though - it passes all the test cases and accepts all things it should accept. Closes: #748936 Thanks: James Clarke for the initial patch

allow warning generation for non-whitelisted options

2016-12-31T17:24:12Z

The idea is simple: Each¹ Find*( call starts with a call check if the given option (with the requested type) exists in the whitelist. The whitelist is specified via our configure-index file so that we have a better chance at keeping it current. the whitelist is loaded via a special (undocumented for now) configuration stanza and if none is loaded the empty whitelist will make it so that no warnings are shown. Much needs to be done still, but that is as good a time as any to take a snapshot of the current state and release it into the wild given that it found some bugs already and has no practical effect on users. ¹ not all in this iteration, but many

expand -f to --fix-broken in error messages

2016-12-31T01:29:20Z

Users end up believing that this is a --force mode as -f is common for that, but apt doesn't have such a mode and --fix-broken is really not about forcing something but actually trying to fix the breakage which tends to be the result of a user forcing something on its system via low-level forced dpkg calls. Example: The "common" pattern of "dpkg -i ./foo.deb; apt install -f" is nowadays far better dealt with via "apt install ./foo.deb". And while at it the two places handing out this suggestion are changed to use the same strings to avoid needless translation work in the future and the suggestion uses 'apt' instead of 'apt-get' as this will be run interactively by a user, so its a good opportunity to showcase what we can do and will allow us to be more helpful to the user. Closes: #709092 Thanks: Kristian Glass for initial patch!

allow default build-essentials to be overridden

2016-12-31T01:29:20Z

The config list APT::Build-Essential gets a similar treatment to other lists now by having the value of the option itself be an override for the list allowing to disable build-essentials entirely as well as adding/overriding as usual by now in other lists. Reported-By: Johannes 'josch' Schauer on IRC

add --indep-only for build-dep command

2016-12-31T01:29:20Z

The implementation is quite different compared to --arch-only due to ABI reasons but functionality wise they are similar and usually both available for symmetry at least. Closes: #845775

ensure generation of valid EDSP error stanzas

2016-12-31T01:29:20Z

The crude way of preparing a message to be a multiline value failed at generation valid deb822 in case the error message ended with a new line like the resolving errors from apt do. apt itself can parse these, but other tools like grep-dctrl choke on it, so be nice and print valid. Reported-By: Johannes 'josch' Schauer on IRC

warn if clearsigned file has ignored content parts

2016-12-31T01:29:19Z

Clearsigned files like InRelease, .dsc, .changes and co can potentially include unsigned or additional messages blocks ignored by gpg in verification, but a potential source of trouble in our own parsing attempts – and an unneeded risk as the usecases for the clearsigned files we deal with do not reasonably include unsigned parts (like emails or some such). This commit changes the silent ignoring to warnings for now to get an impression on how widespread unintended unsigned parts are, but eventually we want to turn these into hard errors.

tests: cache the apt-key homedir used for Release signing

2016-12-21T18:36:10Z

Importing a new secret key into gpg(2) can be increadibly slow which prolongs the test runs significantly – by caching the homedir we gain a significant speedbonus as reimporting already present keys seems like a far less costly operation. Git-Dch: Ignore

let {dsc,tar,diff}-only implicitly enable download-only

2016-12-16T12:50:00Z

That was the case already for tar-only and diff-only, but in a more confusing way and without a message while dsc "worked" before resulting in a dpkg-source error shortly after as tar/diff files aren't available…