apt/methods/rred.cc, branch 1.0.9.5Debians commandline package managerhttps://git.kalnischkies.de/apt/atom?h=1.0.9.52014-03-13T12:58:45Zcleanup headers and especially #includes everywhere2014-03-13T12:58:45ZDavid Kalnischkiesdavid@kalnischkies.de2014-03-05T21:11:25Zurn:sha1:453b82a388013e522b3a1b9fcd6ed0810dab1f4f
Beside being a bit cleaner it hopefully also resolves oddball problems
I have with high levels of parallel jobs.
Git-Dch: Ignore
Reported-By: iwyu (include-what-you-use)
fix -Wformat= warnings about size_t != %lu on e.g. armel2014-03-13T12:58:45ZDavid Kalnischkiesdavid@kalnischkies.de2014-02-25T21:30:39Zurn:sha1:6298ff8b6492e2071a8f2ca7669a3aeef0fb29c7
Git-Dch: Ignore
Reported-By: gcc
use utimes instead of utimensat/futimens2014-02-11T01:28:27ZDavid Kalnischkiesdavid@kalnischkies.de2014-02-10T20:52:38Zurn:sha1:246bbb611d4cd5e2a13ecffb6cbe0e76390eae6f
cppcheck complains about the obsolete utime as it was removed in
POSIX1.2008 and recommends usage of utimensat/futimens instead
as those are in POSIX and so commit 9ce3cfc9 switched to them.
It is just that they aren't as portable as the standard suggests:
At least our kFreeBSD and Hurd ports stumble over it at runtime.
So to make both, the ports and cppcheck happy, we use utimes instead.
Closes: 738567
fix various style/performance warnings in rred2014-01-29T23:16:49ZDavid Kalnischkiesdavid@kalnischkies.de2014-01-29T23:11:05Zurn:sha1:25d99f3b42ada24679ddc1d911530425acc8e475
Reported-By: cppcheck
Git-Dch: Ignore
methods/rred: minor robustness improvements2014-01-20T19:00:12ZAnthony Townsaj@erisian.com.au2014-01-20T19:00:12Zurn:sha1:2fd754cfe9a92fe85f4dd56447c70112b64b003e
Use retry_fwrite to better handle partial fwrite successes, and to keep
the Hashes in sync with what's actually written.
integrate Anthonys rred with POC for client-side merge2014-01-15T21:57:47ZDavid Kalnischkiesdavid@kalnischkies.de2014-01-15T16:23:05Zurn:sha1:50bd6fd3794dd1f61185302129dc6cd218d20b98
Providing the benefits of both without the downsides :)
(ABI breaks or external dependencies)
For this Anthonys rred is equipped with:
- magic-filename-pickup of patches rather than explicit messages
- use of FileFd instead of FILE* to get on-the-fly uncompress
of the gzip compressed pdiff patches
The acquire code in turn stops checking for apt-file's helper
as our own rred is now clever enough for our needs.
reimplement rred to allow applying all the diffs in a single pass2014-01-15T21:43:14ZAnthony Townsaj@erisian.com.au2014-01-15T15:33:36Zurn:sha1:dbd5418b895c0ae3d6585ad38bd40920a456855e
Based on the idea presented in:
https://lists.debian.org/deity/2009/08/msg00169.html and
https://lists.debian.org/debian-devel/2014/01/msg00081.html
It reads all patches one by one and merges them in-memory before
applying the merged changes to the index.
Beware: This commit by David Kalnischkies rips out the rred binary
rewrite unchanged (expect minor format issue corrections) from the
proposed changes, so this commit alone BREAKS pdiff completely.
The integration into the acquire system as it was prepared in the
previous POC will be done in the next commit to have proper 'blame'.
implement POC client-side merging of pdiffs via apt-file2013-12-13T10:59:49ZDavid Kalnischkiesdavid@kalnischkies.de2013-12-06T11:17:48Zurn:sha1:47d2bc78adb49f3182f9a3d7a4baea363e772d64
The idea of pdiffs is to avoid downloading the hole file by patching the
existing index. This works very well, but becomes slow if a lot of
patches needs to be applied to reconstruct an up-to-date index and in
recent years more and more dinstall (or similar) runs are executed
creating more and more pdiffs in the same amount of time, so pdiffs
became less useful.
The solution is simple: Reduce the amount of patches (which are very
small) which need to be applied on top of the index we have available
(which is usually pretty big).
This can be done in two ways: Either merge the patches on the
server-side so that the client has to download only one patch or the
patches are all downloaded and merged on the client-side.
The first needs a client who is doing one step at a time who can also
skip patches if it needs (APT supports this for a long time now).
The later is implemented by this commit, but depends on the server NOT
merging the patches and the patches being in a strict order in which no
patch is skipped.
This is traditionally the case for dak, but other repository creators
support merging – e.g. reprepro (which helpfully adds a flag indicating
that the patches are merged). To support both or even mixes a client
needs more information which isn't available for now.
This POC uses the external diffindex-rred included in apt-file to
do the heavy lifting of merging & applying all patches in one pass,
hence to test this feature apt-file needs to be installed.
we don't need zlib (anymore) in rred so don't include it2012-05-09T22:15:52ZDavid Kalnischkieskalnischkies@gmail.com2012-05-09T22:15:52Zurn:sha1:4ac4a98e16e1ecd34928f6d93389b7106c93c65emake these retry_write methods static so that they don't end up as symbols2012-03-22T21:18:05ZDavid Kalnischkieskalnischkies@gmail.com2012-03-22T21:18:05Zurn:sha1:136a6c13c8df7c403dd5284ff8bda20c8a84b614