apt/test/integration/test-pdiff-usage, branch 1.1.exp1Debians commandline package managerhttps://git.kalnischkies.de/apt/atom?h=1.1.exp12014-05-09T11:06:27Zuse HashStringList in the acquire system2014-05-09T11:06:27ZDavid Kalnischkieskalnischkies@gmail.com2013-08-18T22:00:23Zurn:sha1:b3501edb7091ca3aa6c2d6d96dc667b8161dd2b9
It is not very extensible to have the supported Hashes hardcoded
everywhere and especially if it is part of virtual method names.
It is also possible that a method does not support the 'best' hash
(yet), so we might end up not being able to verify a file even though we
have a common subset of supported hashes. And those are just two of the
cases in which it is handy to have a more dynamic selection.
The downside is that this is a MAJOR API break, but the HashStringList
has a string constructor for compatibility, so with a bit of luck the
few frontends playing with the acquire system directly are okay.
always cleanup patchfiles at the end of rred call2014-02-10T17:07:50ZDavid Kalnischkiesdavid@kalnischkies.de2014-02-10T16:55:13Zurn:sha1:34d6ece7566ea4fcda2286478b31641378aefc93
With APT::Get::List-Cleanup disabled the ed-style patch files are
lingering in the lists/ directory otherwise. That was kinda okay in the
old none-client-merge as the filename was always the same so it was
constantly overridden, but now with different names for client-merge
quiet a few could pill up on the system and are used by the next call
as it picks them up based on the filename.
integrate Anthonys rred with POC for client-side merge2014-01-15T21:57:47ZDavid Kalnischkiesdavid@kalnischkies.de2014-01-15T16:23:05Zurn:sha1:50bd6fd3794dd1f61185302129dc6cd218d20b98
Providing the benefits of both without the downsides :)
(ABI breaks or external dependencies)
For this Anthonys rred is equipped with:
- magic-filename-pickup of patches rather than explicit messages
- use of FileFd instead of FILE* to get on-the-fly uncompress
of the gzip compressed pdiff patches
The acquire code in turn stops checking for apt-file's helper
as our own rred is now clever enough for our needs.
implement POC client-side merging of pdiffs via apt-file2013-12-13T10:59:49ZDavid Kalnischkiesdavid@kalnischkies.de2013-12-06T11:17:48Zurn:sha1:47d2bc78adb49f3182f9a3d7a4baea363e772d64
The idea of pdiffs is to avoid downloading the hole file by patching the
existing index. This works very well, but becomes slow if a lot of
patches needs to be applied to reconstruct an up-to-date index and in
recent years more and more dinstall (or similar) runs are executed
creating more and more pdiffs in the same amount of time, so pdiffs
became less useful.
The solution is simple: Reduce the amount of patches (which are very
small) which need to be applied on top of the index we have available
(which is usually pretty big).
This can be done in two ways: Either merge the patches on the
server-side so that the client has to download only one patch or the
patches are all downloaded and merged on the client-side.
The first needs a client who is doing one step at a time who can also
skip patches if it needs (APT supports this for a long time now).
The later is implemented by this commit, but depends on the server NOT
merging the patches and the patches being in a strict order in which no
patch is skipped.
This is traditionally the case for dak, but other repository creators
support merging – e.g. reprepro (which helpfully adds a flag indicating
that the patches are merged). To support both or even mixes a client
needs more information which isn't available for now.
This POC uses the external diffindex-rred included in apt-file to
do the heavy lifting of merging & applying all patches in one pass,
hence to test this feature apt-file needs to be installed.
configurable compression for testcases2013-08-27T22:24:32ZDavid Kalnischkieskalnischkies@gmail.com2013-08-27T22:24:32Zurn:sha1:276e51dd701590d187ca2999722329518af96121
Compressing files in 4 different styles eats test-time for no practical
gain if we don't test them explicitly, so default to just building 'gz'
compressed files as it is the simplest compression algorithm supported
Git-Dch: Ignore
add chronic-like testsuccess/testfailure helpers2013-08-12T16:01:37ZDavid Kalnischkieskalnischkies@gmail.com2013-08-10T08:01:06Zurn:sha1:0440d936d70b1e331d49bbc32735f1b749632604
For many commands the output isn't stable (like then dpkg is called) but
the exitcode is, so this helper enhances the common && msgpass ||
msgfail by generating automatically a msgtest and showing the output of
the command in case of failure instead of discarding it unconditionally,
the later being chronic-like behaviour
Git-Dch: Ignore
simple fork and pidfile aptwebserver2013-06-24T15:01:33ZDavid Kalnischkieskalnischkies@gmail.com2013-06-24T14:34:38Zurn:sha1:e3c62328abbd548bb0da42fdbad954b3ce4f7102
Forking only after being ready to accept clients avoids running races
with the tests which sometimes failed on the first 'apt-get update'
(or similar) with the previous background-start and hope for the best…
The commit fixes also some oversight output-order changes in regards to
Description-md5 and (I-M-S) race conditions in various tests.
Git-Dch: Ignore
fix the remaining lzma calls with xz --format=lzma in the testcases2012-04-11T10:36:32ZDavid Kalnischkieskalnischkies@gmail.com2012-04-11T10:36:32Zurn:sha1:3b4d8136e78b9f2a70431ece4c850861d90e8bb1do not add Index file by hand now that ftparchive does it by itself2011-01-24T14:41:57ZDavid Kalnischkieskalnischkies@gmail.com2011-01-24T14:41:57Zurn:sha1:321dfa5e570389c489b0f7f0f34ab983cebe7463* methods/rred.cc:2011-01-15T19:28:03ZDavid Kalnischkieskalnischkies@gmail.com2011-01-15T19:28:03Zurn:sha1:caffd4807f612e931ecca8f7d9b0c0c10de27ce6
- operate optional on gzip compressed pdiffs
* apt-pkg/acquire-item.cc:
- don't uncompress downloaded pdiff files before feeding it to rred