apt/apt-pkg/cachefile.cc, branch 1.4.3

fail instead of segfault on unreadable config files

2016-05-20T07:37:24Z

The report mentions "apt list --upgradable", but there are others which have inconsistent behavior ranging from segfaulting to doing something with the partial (and hence incomplete) data. We had a recent report about sources.list (#818628), this one mentions prefences, the obvious next step is conf files… so the testcase is adapted to check for all three in file and directory versions and run a bunch of commands each time which should all have more or less the same behavior in such a case (aka error out). Closes: 824503

cachefile: Only set members that were initialized successfully

2016-03-19T06:19:24Z

Otherwise, things will just start failing later down the stack, because (a) the lazy getters do not check if building was successful and (b) any further getter call would return the invalid object anyway. Also initialize VS in pkgCache to nullptr by default. Closes: #818628

get dpkg lock in build-dep if cache was invalid again

2016-02-10T12:03:00Z

Regression introduced in a249b3e6fd798935a02b769149c9791a6fa6ef16, which in the case of an invalid cache would build the first part unlocked and later pick up the (still unlocked) cache for further processing, so the system got never locked and apt would end up complaining about being unable to release the lock at shutdown. The far more common case of having a valid cache worked as expected and hence covered up the problem – especially as tests who would have noticed it are simulations only, which do not lock. Closes: 814139 Reported-By: Balint Reczey Reported-By: Helmut Grohne on IRC

deal better with (very) small apt::cache-start values

2016-01-26T23:15:12Z

It is a bit academic to support values which aren't big enough to fit even the hashtables without resizing, but cleaning up ensures that we do the right thing (aka not segfaulting) even if something goes wrong in these deep layers. You still can't have very very small values through… Git-Dch: Ignore

reimplement build-dep via apts normal resolver

2016-01-25T17:15:44Z

build-dep was implemented by parsing the build-dependencies of a package and figuring out which packages to install/remove based on this. That means that for the first level of dependencies build-dep was implementing its very own resolver with all the benefits (aka: bugs) this gives us for not using the existing resolver for all levels. Making this work involves generating a dummy binary package with fitting Depends and Conflicts and as we can't create them out of thin air the cache generation needs to be involved so we end up writing a Packages file which we want to parse – after we have parsed the other Packages files already. With .dsc/.deb files we could add them before we started parsing anything. With a bit of care we can avoid generating too much data we have to throw away again (as many parts assume that e.g. the count of packages doesn't change midair), so that on a speed front there shouldn't be much of a difference, but output can be slightly confusing as if we have a completely valid cache on disk the "Reading package lists... Done" is printed two times – but apt is pretty quick about it in that case. Closes: #137560, #444930, #489911, #583914, #728317, #812173

pkgCacheGenerator: Allow passing down an already created cache

2015-12-29T15:29:05Z

If we already have opened a cache, there is no point in having to open it again.

do not segfault in cache generation on mmap failure

2015-11-19T23:54:07Z

Out of memory and similar circumstanzas could cause MMap::Map to fail and especially the mmap/malloc calls in it. With some additional checking we can avoid segfaults and similar in such situations – at least in theory as if this is a real out of memory everything we do to handle the error could just as well run into a memory problem as well… But at least in theory (if MMap::Map is made to fail always) we can deal with it so good that a user actually never sees a failure (as the cache it tries to load with it fails and is discarded, so that DynamicMMap takes over and a new one is build) instead of segfaulting. Closes: 803417

wrap every unlink call to check for != /dev/null

2015-11-04T17:42:28Z

Unlinking /dev/null is bad, we shouldn't do that. Also, we should print at least a warning if we tried to unlink a file but didn't manage to pull it of (ignoring the case were the file is /dev/null or doesn't exist in the first place). This got triggered by a relatively unlikely to cause problem in pkgAcquire::Worker::PrepareFiles which would while temporary uncompressed files (which are set to keep compressed) figure out that to files are the same and prepare for sharing by deleting them. Bad move. That also shows why not printing a warning is a bad idea as this hide the error for in non-root test runs. Git-Dch: Ignore

avoid using global PendingError to avoid failing too often too soon

2015-09-14T13:22:18Z

Our error reporting is historically grown into some kind of mess. A while ago I implemented stacking for the global error which is used in this commit now to wrap calls to functions which do not report (all) errors via return, so that only failures in those calls cause a failure to propergate down the chain rather than failing if anything (potentially totally unrelated) has failed at some point in the past. This way we can avoid stopping the entire acquire process just because a single source produced an error for example. It also means that after the acquire process the cache is generated – even if the acquire process had failures – as we still have the old good data around we can and should generate a cache for (again). There are probably more instances of this hiding, but all these looked like the easiest to work with and fix with reasonable (aka net-positive) effects.

ignore AllowMem parameter in cache generation

2015-08-27T09:27:44Z

The parameter name suggests that it should forbid the building of the entire cache in memory, but this isn't how it was previously and as AllowMem is false by default it actually prevents previous usecases from working like being root and configuring apt to build no caches at all. This should be fixed at some point to actually work, but that is hard to pull off as it means switching the default and some callers (including apt itself) actually did call it explicitly with false in certain cases for no apparent reason (at least now where it is common to have enough memory to throw at every problem and even if not is a slow apt usally better than an apt erroring out). Closes: 796459