summaryrefslogtreecommitdiff
path: root/apt-pkg
Commit message (Collapse)AuthorAgeFilesLines
* init: Add Dir::Bin::planners default entryJulian Andres Klode2016-08-291-0/+1
| | | | | | Apparently we had no default defined for this. Reported-By: David Kalnischkies
* init: Fix path to external solversJulian Andres Klode2016-08-291-1/+1
| | | | | | | This accidentally had two apt in it. This fixes a regression from commit 8757a0f. Gbp-Dch: ignore
* don't loop on pinning pkgs from absolute debs by regexDavid Kalnischkies2016-08-291-5/+4
| | | | | | | | | | | | | | | | | | | | | An absolute filename for a *.deb file starts with a /. A package with the name of the file is inserted in the cache which is provided by the "real" package for internal reasons. The pinning code detects a regex based wildcard by having the regex start with /. That is no problem as a / can not be included in a package name… expect that our virtual filename package can and does. We fix this two ways actually: First, a regex is only being considered a regex if it also ends with / (we don't support flags). That stops our problem with the virtual filename packages already, but to be sure we also do not enter the loop if matcher and package name are equal. It has to be noted that the creation of pins for virtual packages like the here effected filename packages is pointless as only versions can be pinned, but checking that a package is really purely virtual is too costly compared to just creating an unused pin. Closes: 835818
* randomize acquire order for same type index filesDavid Kalnischkies2016-08-291-0/+18
| | | | | | | | | | | | | | | Without randomizing the order in which we download the index files we leak needlessly information to the mirrors of which architecture is native or foreign on this system. More importantly, we leak the order in which description translations will be used which in most cases will e.g. have the native tongue first. Note that the leak effect in practice is limited as apt detects if a file it wants to download is already available in the latest version from a previous download and does not query the server in such cases. Combined with the fact that Translation files are usually updated infrequently and not all at the same time, so a mirror can never be sure if it got asked about all files the user wants.
* Merge branch 'portability/freebsd'Julian Andres Klode2016-08-2713-30/+42
|\
| * CMake: Add missing iconv dependencyJulian Andres Klode2016-08-261-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | FreeBSD has two iconv systems: It ships an iconv.h itself, and symbols for that in the libc. But there's also the port of GNU libiconv, which unfortunately for us, Doxygen depends on. This changes things to prefer a separate libiconv library over the system one; that is, the port on FreeBSD. Gbp-Dch: ignore
| * Make root group configurable via ROOT_GROUPJulian Andres Klode2016-08-264-8/+8
| | | | | | | | | | This is needed on BSD where root's default group is wheel, not root.
| * changelog: Respect Dir setting for local changelog gettingJulian Andres Klode2016-08-261-1/+2
| | | | | | | | | | This fixes issues with chroots, but the goal here was to get the test suite working on systems without dpkg.
| * Make directory paths configurableJulian Andres Klode2016-08-264-12/+12
| | | | | | | | | | | | | | This allows other vendors to use different paths, or to build your own APT in /opt for testing. Note that this uses + 1 in some places, as the paths we receive are absolute, but we need to strip of the initial /.
| * Use C locale instead of C.UTF-8 for protocol stringsJulian Andres Klode2016-08-264-6/+6
| | | | | | | | | | | | The C.UTF-8 locale is not portable, so we need to use C, otherwise we crash on other systems. We can use std::locale::classic() for that, which might also be a bit cheaper than using locale("C").
| * CMake: Do not use -lresolv if res_init exists in libcJulian Andres Klode2016-08-261-1/+1
| | | | | | | | Gbp-Dch: ignore
| * CMake: Do not hardcode -ldlJulian Andres Klode2016-08-261-1/+1
| | | | | | | | | | | | Does not exist on FreeBSD Gbp-Dch: ignore
| * Add missing includes and external definitionsJulian Andres Klode2016-08-264-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several modules use std::array without including the array header. Bad modules. Some modules use STDOUT_FILENO and friends, or close() without including unistd.h, where they are defined. One module also uses WIFEXITED() without including sys/wait.h. Finally, environ is not specified to be defined in unistd.h. We are required to define it ourselves according to POSIX, so let's do that.
* | treat .ddeb files like .deb, especially for dpkgDavid Kalnischkies2016-08-252-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ubuntu uses *.ddeb files for their debug packages, but the interface we are using since f495992428a396e0f98886c9a761a804aa161c68 to talk to dpkg isn't supporting *.ddeb files. This used to work previously as apt itself isn't caring about the filenames at all and if they are explicitly mentioned dpkg will accept all, too. It might or might not be a good idea to patch dpkg, too, but regardless of it happening, we don't want to couple us to closely to dpkg for this minor feature but testing for this at runtime as it would delay shipping the fix for the too long commandlines further. It is also questionable if it is really a good idea to allow any file extension to be used here (like .foobar in the testcase), but we used to and we tend to avoid breaking existing usecases if we can help it. As a bonus, this also allows the installation of ddeb files directly from the commandline as you can with deb files already. We continue to ignore udeb through as the user-mistake to useful ratio is too high. LP: #1616909
* | do not restore selections for already purged packagesDavid Kalnischkies2016-08-241-6/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In most cases apt was already skipping the (re)setting of packages as to be removed/purged if dpkg had told us that it already did, but we haven't dealt with it in the most obvious of the cases: Selections set for packages we touched in this operation which either restores selections even dpkg would have overridden or e.g. tries to restore a purge selection for a package which was just purged – does not happen with apt itself as it isn't using selections in this way, but higher frontends like aptitude do. The result in the later case is a warning printed by dpkg that we try to set selections for an unknown package, which is harmless per se, but can be confusing for users and we really shouldn't cause warnings in dpkg if we can help it. Reported-By: Guillem Jover on IRC
* | improve code & doc for aquire weak/loop failingDavid Kalnischkies2016-08-241-6/+6
| | | | | | | | | | | | Improve-Upon: 2e2865ae53a65c00dd55a892d5b48458f3110366 Reported-By: Julian Andres Klode Gbp-Dch: Ignore
* | do fail on weakhash/loop earlier in acquireDavid Kalnischkies2016-08-243-46/+58
|/ | | | | | | | | | | | | | | | | The bugreport shows a segfault caused by the code not doing the correct magical dance to remove an item from inside a queue in all cases. We could try hard to fix this, but it is actually better and also easier to perform these checks (which cause instant failure) earlier so that they haven't entered queue(s) yet, which in return makes cleanup trivial. The result is that we actually end up failing "too early" as if we wouldn't be careful download errors would be logged before that process was even started. Not a problem for the acquire system, but likely to confuse users and programs alike if they see the download process producing errors before apt was technically allowed to do an acquire (it didn't, so no violation, but it looks like it to the untrained eye). Closes: 835195
* install-progress: Call the real ::fork() in our fork() methodJulian Andres Klode2016-08-231-1/+1
| | | | | | We basically called ourselves before, creating an endless loop. Reported-By: clang
* prevent C++ locale number formatting in text APIs (try 3)David Kalnischkies2016-08-232-36/+44
| | | | | | | | | | | This time it is the formatting of floating numbers in progress reporting with a radix charater potentially not being dot. Followup of 7303e11ff28f920a6277c159aa46f80c007350bb. Regression of b58e2c7c56b1416a343e81f9f80cb1f02c128e25 in so far as it exchanging very effected with slightly less effected code. LP: 1611010
* do dpkg --configure before --remove/--purge --pendingDavid Kalnischkies2016-08-231-3/+7
| | | | | | | | | | | | | | | | | | | Commit 7ec343309b7bc6001b465c870609b3c570026149 got us most of the way, but the last mile was botched by having the pending calls in the wrong order as this way we potentially 'force' dpkg to remove/purge a package it doesn't want to as another package still depends on it and the replacement isn't fully installed yet. So what we do now is a configure before remove and purge (all with --no-triggers) and finishing off with another configure pending call to take care of the triggers. Note that in the bugreport example our current planner is forcing dpkg to remove the package earlier via --force-depends which we could do for the pending calls as well and could be used as a workaround, but we want to do less forcing eventually. Closes: 835094
* CMake: Fix uninitialized variablesJulian Andres Klode2016-08-201-1/+1
| | | | | | This fixes some actual bugs for PROJECT and BZIP2_INCLUDE_DIR. Gbp-Dch: ignore
* Ignore SIGINT and SIGQUIT for Pre-Install hooksJulian Andres Klode2016-08-191-7/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of erroring out when receiving a SIGINT, let the child deal with it - we'll error out anyway if the child exits with an error or due to the signal. Also ignore SIGQUIT, as system() ignores it. This basically fixes Bug #832593, but: we are running the hooks via sh -c. Some shells exit with a signal error even if the command they are executing catches the signal and exits successfully. So far, this has been noticed on dash, which unfortunately, is our default shell. Example: $ cat trap.sh trap 'echo int' INT; sleep 10; exit 0 $ if dash -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint FAIL: 130 $ if mksh -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint OK: 0 $ if bash -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint OK: 0
* don't count each Type as an individual deb822-sources stanzaDavid Kalnischkies2016-08-171-5/+3
| | | | Reported-By: Mattia Rizzolo <mattia@debian.org> in #834629
* add --with-source option and Packages/Sources supportDavid Kalnischkies2016-08-173-3/+45
| | | | | | | | | | | | | We support "./foobar.deb" as a way to install a deb file directly. Recently .changes files were added. This highlights a problem as you can't add the changes file without also trying to install all of them. Now, it could also be handy to add entire Packages/Sources files to perhaps get a bunch of packages in without installing them all implicitly. This commit introduces --with-source which allows to add *.deb, *.changes, *.dsc, source-dirs, Packages & Sources files (the later can also be compressed) without also installing them.
* default to Dir=/ in dpkg/status file finding magicDavid Kalnischkies2016-08-171-12/+10
| | | | | | | | | | | | | Seen in cme #833656 if Dir isn't set (yet) we end up later absoluting a path which was supposed to be absolute already, so if Dir is empty we assume it to be '/' instead. In practice this is a bug in the software using libapt, but for maxium compatibility lets explicitly set the default value here to be safe. Reported-By: Paul Wise <pabs@debian.org> Inspired-By: Brendan O'Dea <bod@debian.org> Fixes-Regression: 475f75506db48a7fa90711fce4ed129f6a14cc9a Shadows-Bug: #833656
* support compression and by-hash for .diff/Index filesDavid Kalnischkies2016-08-172-114/+121
| | | | | | | | | | | In af81ab9030229b4ce6cbe28f0f0831d4896fda01 by-hash got implemented as a special compression type for our usual index files like Packages. Missing in this scheme was the special .diff/Index index file containing the info about individual patches for this index file. Deriving from the index file class directly we inherent the compression handling infrastructure and in this way also by-hash nearly for free. Closes: #824926
* support getting only-uncompressed files via by-hashDavid Kalnischkies2016-08-171-0/+2
| | | | | | The URI we later want to modify to get the file via by-hash was unset in case a file was only available uncompressed (which is usually not the case) causing an acquire error.
* set the correct item FileSize in by-hash caseDavid Kalnischkies2016-08-171-4/+3
| | | | | | | | | | In af81ab9030229b4ce6cbe28f0f0831d4896fda01 we implement by-hash as a special compression type, which breaks this filesize setting as the code is looking for a foobar.by-hash file then. Dealing this slightly gets us the intended value. Note that this has no direct effect as this value will be set in other ways, too, and could only effect progress reporting. Gbp-Dch: Ignore
* retry without same redirection mirror on 404 errorsDavid Kalnischkies2016-08-171-1/+21
| | | | | | | | If 9b8034a9fd40b4d05075fda719e61f6eb4c45678 serves the Release files from a partial mirror we will end up getting 404 for some of the indexes. Instead of giving up, we will instead ignore our same redirection mirror constrain and ask the redirection service as a potential hashsum mismatch is better than keeping the certain 404 error.
* check internal redirections for loops, tooDavid Kalnischkies2016-08-172-0/+22
| | | | | | | | | | Now that we have the redirections loopchecker centrally in our items we can use it also to prevent internal redirections to loop caused by bugs as in a few instances we get into the business of rewriting the URI we will query by ourself as we predict we would see such a redirect anyway. Our code has no bugs of course, hence no practical difference. ;) Gbp-Dch: Ignore
* log with the failed item description, not with next tryDavid Kalnischkies2016-08-161-3/+4
| | | | | | | | | | The failure handling frequently changes URI & Description of the failed item to try a slightly different combination which might work, but the logging of the failure happens only afterwards as the same failure handling decides if this is a critical error or not so we need a backup here instead of potentially new content. A purely cosmetic issue, but can still be confusing for humans.
* drop incorrect const attribute from DirectoryExistsDavid Kalnischkies2016-08-121-1/+1
| | | | | | | | | | | | | | | | | Since its existence in 2010 DirectoryExists was always marked with this attribute, but for no real reason. Arguably a check for the existence of the file is not modifying global state, so theoretically this shouldn't be a problem. It is wrong from a logical point of view through as between two calls the directory could be created so the promise we made to the compiler that it could remove the second call would be wrong, so API wise it is wrong. It's a bit mysterious that this is only observeable on ppc64el and can be fixed by reordering code ever so slightly, but in the end its more our fault for adding this attribute than the compilers fault for doing something silly based on the attribute. LP: 1473674
* fileutl: empty file support: Avoid fstat() on -1 fd and check resultJulian Andres Klode2016-08-121-2/+3
| | | | | When checking if a file is empty, we forget to check that fstat() actually worked.
* ensure a good clock() value for usage and testsDavid Kalnischkies2016-08-121-1/+1
| | | | | | | | | | | We use clock() as a very cheap way of getting a "random" value, but the manpage warns that this could return -1, so we should be dealing with this. Additionally, e.g. on hurd-i386 the value increases only slowly – to slow for our fast running tests for randomness hence producing the same range in both samples, so we introduce a simple busy-wait loop (as clock is counting processor time used by the program) in the test which delays the second sample just enough making our randomness a bit more predictable.
* don't perform int<float in progress bar drawingDavid Kalnischkies2016-08-121-12/+8
| | | | | | | | Comparing floating numbers is always fun and in this instance a 9 < 9.0 is "somehow" true on hurd-i386 letting the tests fail by reporting that too much progress achieved. A bit mysterious, but with some rework we can use code which avoids dealing with the floats in this way entirely and make our testcases happy.
* Merge branch 'feature/apt-dpkg-comm'David Kalnischkies2016-08-117-89/+533
|\
| * disable explicit configuration of all packages at the endDavid Kalnischkies2016-08-102-12/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With b4450f1dd6bca537e60406b2383ab154a3e1485f we dropped what we calculated here later on and now that we don't need it in the meantime either we can just skip the busy work by default and expect dpkg to do the right thing dropping also our little "last explicit configures" removal trick introduced in b4450f1dd6bca537e60406b2383ab154a3e1485f. This enables the last of a bunch of previously experimental options, some of them existing still, but are very special and hence not really worth documenting anymore (especially as it would need to be rewritten now entirely) which is why the documentation is nearly completely dropped. The order of configuration stanzas in the simulation code changes slightly as it isn't concerning itself with finding the 'right' order, but any order is valid anyhow as long as the entire set happens in the same call.
| * simulate all package manager actions explicitlyDavid Kalnischkies2016-08-106-31/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a planner lets actions to be figured out by dpkg in pending calls these actions aren't mentioned in a simulation. While that might be a good thing for debugging, it would be a change in behavior and especially if a planner avoids explicit removals could be confusing for users. As such we perform the same 'trick' as in the dpkg implementation by performing explicitly what would be done by the pending calls. To save us some work and avoid desyncs we perform a layer violation by using deb/ code in the generic simulation – and further we perform ugly dynamic_cast to avoid breaking the ABI for nothing; aptitude is the only other user of the simulation class according to codesearch.d.n and for that our little trick works. It just isn't working if you happen to extend pkgSimulate or otherwise manage to call the protected Go methods directly – which isn't very realistic/practical.
| * try to avoid removal of crossgraded packagesDavid Kalnischkies2016-08-101-26/+140
| | | | | | | | | | | | | | | | The user has to approve the removal of a crossgraded package as it might be needed to remove it (temporarily) in the process, but in most cases we can happily avoid it and let dpkg unpack over it skipping the remove. This has some effects on progress reporting and how deal with selections through which makes this a tiny bit complicated.
| * ensure all removes are reported to hook scriptsDavid Kalnischkies2016-08-101-0/+13
| | | | | | | | Same reason and implementation as for configure.
| * ensure all configures are reported to hook scriptsDavid Kalnischkies2016-08-101-0/+17
| | | | | | | | | | | | | | A planner might not explicitly configure all packages, but we need to know all packages which will be configured for progress reporting and to tell the hook scripts about them as they rely on this for their own functionality.
| * don't purge directly, but remove and do purge at the endDavid Kalnischkies2016-08-101-61/+86
| | | | | | | | | | | | | | | | | | | | If we want a package to be purged from the system tell dpkg in the ordering (if it has to touch it explicitly) to remove it and cover the purging of the config files at the end with a --purge --pending call. That should help packages move conffiles around between packages correctly even if the user is purging packages directly in big actions like dist-upgrades involving many packages.
| * call dpkg with --no-triggers by defaultDavid Kalnischkies2016-08-101-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | Implemented a long while ago now with relatively good progress reporting involving triggers is a good time to try delaying the execution of triggers across dpkg invocations finally by default. Note: The bugreport talks also about 'smarter' configuration which is a much bigger part and approached from multiple directions, but doesn't really involve triggers per-se so considering it decoupled should help in getting it done… Closes: #626599
| * select remove/purge packages early on for dpkgDavid Kalnischkies2016-08-102-14/+84
| | | | | | | | | | | | | | Telling dpkg early on that we are going to remove these packages later helps it with auto-deconfiguration decisions and its another area where a planner can ignore the nitty gritty details and let dpkg decide the course of action if there are no special requirements.
| * save and restore selection states before/after calling dpkgDavid Kalnischkies2016-08-102-7/+38
| | | | | | | | | | | | | | | | | | | | | | | | dpkg decides certain things on its own based on selections and especially if we want to call --pending on purge/remove actions, we need to ensure a clean slate or otherwise we surprise the user by removing packages we weren't allowed to remove by the user in this run (the selection might be an overarching plan for the not-yet "future"). Ideally dpkg would have some kind of temporal selection interface for this case, but it hasn't, so we make it temporal with the risk of loosing state if we don't manage to restore them.
| * use dpkg --unpack --recursive to avoid long cmdlinesDavid Kalnischkies2016-08-101-7/+95
| | | | | | | | | | | | | | | | | | | | Having long commandlines split into two is a huge problem if it happens and additionally if we want to introduce planners which perform less micromanagment its a good idea to leave the details for dpkg to decide. In practice this doesn't work yet unconditionally as a bug is hiding in the ordering code of dpkg, but it works if apt imposes its ordering so this commit allows for now at least to solve the first problem.
| * pass --force-remove-essential to dpkg only if neededDavid Kalnischkies2016-08-101-2/+13
| | | | | | | | | | | | APT (usually) knows which package is essential or not, so we can avoid passing this force flag to dpkg unconditionally if the user hasn't chosen a non-default essential handling obscuring the information.
* | Merge branch 'feature/methods'David Kalnischkies2016-08-115-29/+124
|\ \
| * | allow methods to be disabled and redirected via configDavid Kalnischkies2016-08-101-7/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To prevent accidents like adding http-sources while using tor+http it can make sense to allow disabling methods. It might even make sense to allow "redirections" and adding "symlinked" methods via configuration. This could e.g. allow using different options for certain sources by adding and configuring a "virtual" new method which picks up the config based on the name it was called with like e.g. http does if called as tor+http.
| * | implement socks5h proxy support for http methodDavid Kalnischkies2016-08-102-0/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Socks support is a requested feature in sofar that the internet is actually believing Acquire::socks::Proxy would exist. It doesn't and this commit isn't adding it as that isn't how our configuration works, but it allows Acquire::http::Proxy="socks5h://…". The HTTPS method was changed already to support socks proxies (all versions) via curl. This commit implements only SOCKS5 (RFC1928) with no auth or pass&user auth (RFC1929), but not GSSAPI which is required by the RFC. The 'h' in the protocol name further indicates that DNS resolution is delegated to the socks proxy rather than performed locally. The implementation works and was tested with Tor as socks proxy for which implementing socks5h only can actually be considered a feature. Closes: 744934