summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* FileFd: Add a buffered writing modeJulian Andres Klode2015-12-272-0/+153
| | | | | This is somewhat experimental right now, and might not work for everyone, so it is on an opt-in basis.
* FildFd: Introduce a Flush() function and call it from Close()Julian Andres Klode2015-12-272-0/+16
| | | | The flush function can be used for buffered writers.
* FileFdPrivate: Add getter and setter for fieldsJulian Andres Klode2015-12-271-9/+42
| | | | | We will soon implement a buffered writing decorator and we will need to forward attribute changes to those.
* fileutl: simple_buffer: Add write() and full() methodsJulian Andres Klode2015-12-271-0/+11
| | | | | | These can be used to implement write buffering Gbp-Dch: ignore
* fileutl: simple_buffer: Mark accessors as constJulian Andres Klode2015-12-271-2/+3
| | | | | | Suggested by David. Gbp-Dch: ignore
* FileFdPrivate: Extract SimpleBuffer and mark it as hiddenJulian Andres Klode2015-12-271-21/+24
| | | | Gbp-Dch: ignore
* rred: Only call pkgInitConfig() in test modeJulian Andres Klode2015-12-271-2/+2
| | | | | | | This accidentally slipped in in a previous commit, but it should be used only for testing mode. Reported-By: David Kalnischkies <david@kalnischkies.de>
* ParseDepends: Mark branches for build-dep parsing as unlikelyJulian Andres Klode2015-12-271-2/+2
| | | | | | We do not see those branches at all during normal mode of operation (that is, during cache generation), so tell the compiler about it.
* debListParser: Do not validate Description-md5 for correctness twiceJulian Andres Klode2015-12-271-2/+4
| | | | | The Set() method returns false if the input is no hex number, so simply use that.
* Hex2Digit: Do not use isxdigit()Niels Thykier2015-12-271-4/+9
| | | | | | | We directly check if we are a hex digit in HexDigit, so use that information. [jak@debian.org: Commit message wording]
* debListParser: ParseDepends: Only query native arch if neededJulian Andres Klode2015-12-271-1/+2
| | | | | | This makes the code parsing architecture lists slower, but on the other hand, improves the more generic case of reading dependencies from Packages files.
* pkgcachegen: Use std::unordered_map instead of std::mapJulian Andres Klode2015-12-272-7/+7
| | | | | std::unordered_map is faster than std::map in our use case, reducing cache generation time by about 10% in my benchmark.
* Convert most callers of isspace() to isspace_ascii()Julian Andres Klode2015-12-278-34/+37
| | | | | This converts all callers that read machine-generated data, callers that might work with user input are not converted.
* Introduce isspace_ascii() for use by parsersJulian Andres Klode2015-12-272-0/+19
| | | | This is like isspace(), but ignores the current locale.
* Release 1.1.81.1.8Julian Andres Klode2015-12-2613-13/+20
| | | | This release is made for Niels Thykier and apt-file.
* apt-helper: Check that we can open stdoutJulian Andres Klode2015-12-261-1/+2
| | | | Gbp-Dch: ignore
* Refactor InternalReadLine to not unroll Size == 0 caseJulian Andres Klode2015-12-261-5/+4
| | | | | | There is not much point and this is more readable. Gbp-Dch: ignore
* Change InternalReadLine to always use buffer.read() return valueJulian Andres Klode2015-12-261-12/+8
| | | | | | | | | This is mostly a documentation issue, as the size we want to read is always less than or equal to the size of the buffer, so the return value will be the same as the size argument. Nonetheless, people wondered about it, and it seems clearer to just always use the return value.
* apt-helper: Add a cat-file command for concatening filesJulian Andres Klode2015-12-261-0/+33
| | | | | This can automatically handle compressed files and is useful for stuff like apt-file.
* Release 1.1.71.1.7Julian Andres Klode2015-12-2613-13/+23
|
* Get rid of memmove() in our read bufferingJulian Andres Klode2015-12-261-76/+57
| | | | | | This further improves our performance, and rred on uncompressed files now spents 78% of its time in writing. Which means that we should really look at buffering those.
* rred: Allow passing files as arguments for compressor testingJulian Andres Klode2015-12-261-2/+14
| | | | | | | | This introduces a -t mode in which the first argument is input, the second is output and the remaining are diffs. This allows us to test patching compressed files, which are detected using their file extension.
* Use a hardcoded buffer size of 4096 to fix performanceJulian Andres Klode2015-12-261-4/+2
| | | | | | | | | | | | | | The code uses memmove() to move parts of the buffer to the front when the buffer is only partially read. By simply reading one page at a time, the maximum size of bytes that must be moved has a hard limit, and performance improves: In one test case, consisting of a 430 MB Contents file, and a 75K PDiff, applying the PDiff previously took about 48 seconds and now completes in 2 seconds. Further speed up can be achieved by buffering writes, they account for about 60% of the run-time now.
* Mark all FileFdPrivate classes as hidden1.1.6Julian Andres Klode2015-12-241-6/+6
| | | | Gbp-Dch: ignore
* 1.1.6 Christmas releaseJulian Andres Klode2015-12-2458-410/+432
|
* fix new[] vs delete mismatch introduced by b3db9d81David Kalnischkies2015-12-231-7/+7
| | | | | | | | And as we are at it lets fix the 'style' issue I introduced with the filefd changes as well. Reported-By: gcc -fsanitize's & cppcheck Git-Dch: Ignore
* ensure we got a lock in clean operationDavid Kalnischkies2015-12-232-4/+14
| | | | | | We try to acquired the locks, but we didn't stop if we failed to get it… Closes: 808561
* use a dynamic buffer for ReadLineDavid Kalnischkies2015-12-231-15/+22
| | | | | | | | | We don't need the buffer that often - only for ReadLine - as it is only occasionally used, so it is actually more efficient to allocate it if needed instead of statically by default. It also allows the caller to influence the buffer size instead of hardcoding it. Git-Dch: Ignore
* implement a buffer system for FileFd::ReadLineDavid Kalnischkies2015-12-231-27/+140
| | | | | | | | | | | | | | | The default implementation of ReadLine was very naive by just reading each character one-by-one. That is kinda okay for libraries implementing compression as they have internal buffers (but still not great), but while working with files directly or via a pipe as there is no buffer there so all those reads are in fact system calls. This commit introduces an internal buffer in the FileFd implementation which is only used by ReadLine. The more low-level Read and all other actions remain unbuffered – they just changed to deal with potential "left-overs" in the buffer correctly. Closes: 808579
* parse xz-compression level from configurationDavid Kalnischkies2015-12-221-2/+28
| | | | | | | If we use the library to compress xz, still try to understand and pick up the arguments we would have used to call xz to figure out which level the user wants us to use instead of defaulting to level 6 (which is the default level of xz).
* follow dpkg and xz and use CRC64 for xz compressionDavid Kalnischkies2015-12-221-1/+1
| | | | | | | | dpkg switched from CRC32 to CRC64 in 777915108d9d36d022dc4fc4151a615fc95e5032 with the message: | This is the default CRC used by the xz command-line tool, align with | it and switch from CRC32 to CRC64. It should provide slightly better | detection against damaged data, at a negligible speed difference.
* shuffle compressor-specific code into private subclassesDavid Kalnischkies2015-12-222-635/+692
| | | | | | | | | | | | | | This isn't implementing any new features, it is "just" moving code around from FileFd methods which decided on each call how to handle the request by including all logic for all possible compressor backends in the method body to a model in which backend-specifics are implemented in a FileFdPrivate subclass. This avoids a big chunk of #ifdef's and should make it a tiny bit more obvious which backend uses which code. The execution of the idea is slightly uglified by the need to preserve ABI and API which causes liberal befriending. Git-Dch: Ignore
* tests: support gpg2 properly in all testcasesDavid Kalnischkies2015-12-194-44/+48
| | | | | | | | | The output changes slightly between different versions, which we already dealt with in the main testcase for apt-key, but there are two more which do not test both versions explicitly and so still had gpg1 output to check against as this is the default at the moment. Git-Dch: Ignore
* avoid triggering gpg2 migration in apt-keyDavid Kalnischkies2015-12-191-15/+15
| | | | | | | | | | | | | | | | The presents (even of an empty) secring.gpg is indication enough for gpg2 to tigger the migration code which not only produces a bunch of output on each apt-key call, but also takes a while to complete as an agent needs to be started and all that. We workaround the first part by forcing the migration to happen always in a call we forced into silence, but that leaves us with an agent to start all the time – with a bit of reordering we can make it so that we do not explicitly create the secring, but let gpg create it if needed, which prevents the migration from being triggered and we have at least a bit less of a need for an agent. Changes - even to public only keyrings - still require one, but such actions are infrequent in comparison to verification calls, so that should be a net improvement.
* avoid evaluating shell in paths used in apt-keyDavid Kalnischkies2015-12-193-23/+45
| | | | | | | | | | | | apt-key creates internally a script (since ~1.1) which it will call to avoid dealing with an array of different options in the code itself, but while writing this script it wraps the values in "", which will cause the shell to evaluate its content upon execution. To make 'use' of this either set a absolute gpg command or TMPDIR to something as interesting as: "/tmp/This is fü\$\$ing cràzy, \$(man man | head -n1 | cut -d' ' -f1)\$!" If such paths can be encountered in reality is a different question…
* tests: support spaces in path and TMPDIRDavid Kalnischkies2015-12-19207-518/+495
| | | | | | | This doesn't allow all tests to run cleanly, but it at least allows to write tests which could run successfully in such environments. Git-Dch: Ignore
* buildsystem: deal with spaces in path to sourceDavid Kalnischkies2015-12-198-11/+11
| | | | Git-Dch: Ignore
* Do not try to read in FileFd::Read() if Size is 0Julian Andres Klode2015-12-191-3/+2
| | | | | | | There's no point trying to read 0 bytes, so let's just not do this and switch to a while loop like in Write(). Gbp-Dch: ignore
* Do nothing in FileFd::Write() if Size is 0Julian Andres Klode2015-12-191-7/+5
| | | | | | | | | | | Turn the do-while loop into while loops, so it simply does nothing if the Size is already 0. This reverts commit c0b271edc2f6d9e5dea5ac82fbc911f0e3adfa7a which introduced a fix for a specific instance of the issue in the CopyFile() function. Closes: #808381
* CopyFile: avoid failing on EOF on some systemsPino Toscano2015-12-191-1/+1
| | | | | | | | | On EOF, ToRead will be 0, which might trigger on some systems (e.g. on the Hurd) an error due to the invalid byte count passed to write(). The whole loop already checks for ToRead != 0, so perform the writing step only when there was actual data read. Closes: #808381
* CopyFile: fix BufSize to a sane valuePino Toscano2015-12-191-2/+2
| | | | | | | | | | | | | | Commit e977b8b9234ac5db32f2f0ad7e183139b988340d tries to make BufSize calculated based on the size of the buffer; the problem is that std::unique_ptr::size() returns a pointer to the data, so sizeof() equals to the size of a pointer (later divided by sizeof(char), which is 1). The result is that the CopyFile copies in chunks of 8 bytes, which is not exactly ideal... As solution, declare BufSize in advance, and use its value to allocate the Buf array. Closes: #808381
* Fix FileUtlTest.GetTempDir failure when run as rootPino Toscano2015-12-191-3/+7
| | | | | | | | | | | Testing /usr as TMPDIR assumes that GetTempDir() cannot use it because it cannot write to it; this is true for non-root users, but not so much for root. Since root can access everything, perform this particular test case only when not running as root. Closes: #808383
* Release 1.1.51.1.5Julian Andres Klode2015-12-1458-279/+924
|
* test-apt-download-progress: Run the https test multiple timesJulian Andres Klode2015-12-141-2/+7
| | | | | This filters out errors due to timing issues. Early exits if enough pulses occured.
* pkgcache: Make hash arch-independent using fixed size integerJulian Andres Klode2015-12-142-3/+3
| | | | | | | | This helps writing test cases. Also adapt the test case that expected 64-bit. Nothing changes performance wise, the distribution of the hash values remains intact.
* debian/apt.auto-removal.sh: Adjust -dbg exclusion for multi-archJulian Andres Klode2015-12-141-2/+1
| | | | Allow an optional colon followed by anything at the end.
* tagfile: Hardcode error message for out of range integer valuesJulian Andres Klode2015-12-142-5/+4
| | | | | | This makes the test suite work on 32 bit-long platforms. Gbp-Dch: ignore
* test framework: More noopchroot fixesJulian Andres Klode2015-12-141-6/+11
| | | | | | | | | | | Use asprintf() so we have easy error detection and do not depend on PATH_MAX. Do not add another separator to the generated path, in both cases the path inside the chroot is guaranteed to have a leading / already. Also pass -Wall to gcc.
* test framework: Unset no_proxy as wellJulian Andres Klode2015-12-141-1/+1
| | | | | | This caused test-bug-717891-abolute-uris-for-proxies to fail Gbp-Dch: ignore
* test framework: Unset http proxy variables in setupenvironmentJulian Andres Klode2015-12-141-0/+1
| | | | | | This breaks a lot of test cases Gbp-Dch: ignore
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2024-11-04 18:06:05 +0000