Debians commandline package manager https://git.kalnischkies.de/apt/atom?h=1.1.8 2015-12-26T21:46:04Z 2015-12-26T21:46:04Z Julian Andres Klode jak@debian.org 2015-12-26T21:23:43Z urn:sha1:01152444ba96051fa0ca90b08dcbb8fec9d81745 There is not much point and this is more readable. Gbp-Dch: ignore 2015-12-26T21:46:04Z Julian Andres Klode jak@debian.org 2015-12-26T21:06:14Z urn:sha1:a9024b1be2e26c3c6b1f67093ddbf7602636ec9f This is mostly a documentation issue, as the size we want to read is always less than or equal to the size of the buffer, so the return value will be the same as the size argument. Nonetheless, people wondered about it, and it seems clearer to just always use the return value. 2015-12-26T16:38:40Z Julian Andres Klode jak@debian.org 2015-12-26T16:38:40Z urn:sha1:83e22e26f9f10472aed97f889967c86ee218d28d This further improves our performance, and rred on uncompressed files now spents 78% of its time in writing. Which means that we should really look at buffering those. 2015-12-26T13:49:00Z Julian Andres Klode jak@debian.org 2015-12-26T13:49:00Z urn:sha1:0b29c72bdfc1466d47567cc3191b9661f81d3d3f The code uses memmove() to move parts of the buffer to the front when the buffer is only partially read. By simply reading one page at a time, the maximum size of bytes that must be moved has a hard limit, and performance improves: In one test case, consisting of a 430 MB Contents file, and a 75K PDiff, applying the PDiff previously took about 48 seconds and now completes in 2 seconds. Further speed up can be achieved by buffering writes, they account for about 60% of the run-time now. 2015-12-24T16:14:15Z Julian Andres Klode jak@debian.org 2015-12-24T16:14:15Z urn:sha1:65ac6aad5f707849a127202f808d087d1fcaddc9 Gbp-Dch: ignore 2015-12-23T18:42:53Z David Kalnischkies david@kalnischkies.de 2015-12-23T18:42:53Z urn:sha1:117551478551958d62d9555c595f2990805137c0 And as we are at it lets fix the 'style' issue I introduced with the filefd changes as well. Reported-By: gcc -fsanitize's & cppcheck Git-Dch: Ignore 2015-12-23T16:16:13Z David Kalnischkies david@kalnischkies.de 2015-12-23T16:16:13Z urn:sha1:b3db9d81325769fb9d9c6518f9d917f90328bef1 We don't need the buffer that often - only for ReadLine - as it is only occasionally used, so it is actually more efficient to allocate it if needed instead of statically by default. It also allows the caller to influence the buffer size instead of hardcoding it. Git-Dch: Ignore 2015-12-23T15:42:12Z David Kalnischkies david@kalnischkies.de 2015-12-23T15:42:12Z urn:sha1:f63123c385738b059cd854b694a0f62558fb8cbe The default implementation of ReadLine was very naive by just reading each character one-by-one. That is kinda okay for libraries implementing compression as they have internal buffers (but still not great), but while working with files directly or via a pipe as there is no buffer there so all those reads are in fact system calls. This commit introduces an internal buffer in the FileFd implementation which is only used by ReadLine. The more low-level Read and all other actions remain unbuffered – they just changed to deal with potential "left-overs" in the buffer correctly. Closes: 808579 2015-12-22T15:35:06Z David Kalnischkies david@kalnischkies.de 2015-12-22T15:32:56Z urn:sha1:7a68effcb904b4424b54a30e448b6f2560cd1078 If we use the library to compress xz, still try to understand and pick up the arguments we would have used to call xz to figure out which level the user wants us to use instead of defaulting to level 6 (which is the default level of xz). 2015-12-22T14:56:47Z David Kalnischkies david@kalnischkies.de 2015-12-22T14:56:47Z urn:sha1:885a1ffd27e621d7cd2452b39e2053e2f1044253 dpkg switched from CRC32 to CRC64 in 777915108d9d36d022dc4fc4151a615fc95e5032 with the message: | This is the default CRC used by the xz command-line tool, align with | it and switch from CRC32 to CRC64. It should provide slightly better | detection against damaged data, at a negligible speed difference.