summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Harden test for no new acquires after transaction abortHEADmasterDavid Kalnischkies2021-03-111-9/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If a transaction is doomed we want to gracefully shutdown our zoo of worker processes. As explained in the referenced commit we do this by stopping the main process from handing out new work and ignoring the replies it gets from the workers, so that they eventually run out of work. We tested this previously by checking if a rred worker was given work items at all, but depending on how lucky the stars of the machine working on this are the worker would have already gotten work before the transaction was aborted – so we tried this 25 times a row (f35601e5d2). No machine can be this lucky, right? Turns out the autopkgtest armhf machine is very lucky. I feel a bit sorry for feeding grep such a long "line" to work with, but it seems to work out. Porterbox amdahl (who is considerably less lucky; had to turn down to 1 try to get it to fail sometimes) is now happily running the test in an endless loop. Of course, I could have broken the test now, but its still a rather generic grep (in some ways more generic even) and the main part of the testcase – the update process finishes and fails – is untouched. References: 38f8704e419ed93f433129e20df5611df6652620 Closes: #984966
* Release 2.3.02.3.0Julian Andres Klode2021-03-085-6/+17
|
* Merge branch 'fix/rredemptypatches' into 'master'Julian Andres Klode2021-03-086-39/+48
|\ | | | | | | | | Deal with rred shortcomings around empty patch files See merge request apt-team/apt!159
| * Ensure all index files sent custom tags to the methodsDavid Kalnischkies2021-03-072-5/+16
| | | | | | | | | | | | | | | | | | | | | | The mirror method can distribute requests for files based on various metadata bits, but some – the main index files – weren't actually passing those on to the methods as advertised in the manpage. This is hidden both by mirror usually falling back to other sources which will eventually hit the right one and that if the repository does not support by-hash apt will automatically stick to the mirror which was used for the Release file.
| * Start pdiff patching from the last possible starting pointDavid Kalnischkies2021-03-072-18/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Especially in small sections of an archive it can happen that an index returns to a previous state (e.g. if a package was first added and then removed with no other changes happening in between). The result is that we have multiple patches which start from the same hash which if we perform clientside merging is no problem although not ideal as we perform needless work. For serverside merging it would not matter, but due to rred previously refusing to merge zero-size patches but dak ignoring failure letting it carry these size-zero patches until they naturally expire we run into a problem as these broken patches won't do and force us to fall back to downloading the entire index. By always starting from the last patch instead of the first with the starter hash we can avoid this problem and behave optimally in clientside merge cases, too.
| * Rename pdiff merge patches only after they are all downloadedDavid Kalnischkies2021-03-073-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The rred method expects the patches to have a certain name, which we have to rename the file to before calling the method, but by delaying the rename we ensure that if the download of one of them fails and a successful fallback occurs they are all properly cleaned up as no longer useful while in the error case the next apt run can potentially pick them up as already downloaded. Our test-pdiff-usage test was encountering this every other run, but did not fail as the check for unaccounted files in partial/ was wrapped in a subshell so that the failure produced failing output, but did not change the exit code.
| * Allow merging with empty pdiff patchesDavid Kalnischkies2021-03-062-3/+8
|/ | | | | | | | There isn't a lot of sense in working on empty patches as they change nothing (quite literally), but they can be the result of merging multiple patches and so to not require our users to specifically detect and remove them, we can be nice and just ignore them instead of erroring out.
* Revert "Branch of as 2.2.y for bullseye"Julian Andres Klode2021-03-012-3/+3
| | | | | This reverts commit d96c9a0280bffcfb0f4a319e003e9af60c6cfaf1. It is not correct for master.
* Merge tag '2.2.1'Julian Andres Klode2021-03-0114-66/+132
|\ | | | | | | apt Debian release 2.2.1
| * Release 2.2.12.2.1Julian Andres Klode2021-03-015-6/+21
| |
| * regression fix: do require force-loopbreak for ConflictsJulian Andres Klode2021-03-012-16/+39
| | | | | | | | | | | | | | | | Conflicts do require removing the package temporarily, so they really should not be used. We need to improve that eventually such that we can deconfigure packages when we have to remove their dependencies due to conflicts.
| * RunScripts: Do not reset SIGQUIT and SIGINT to SIG_DFLJulian Andres Klode2021-03-011-4/+0
| | | | | | | | | | | | | | | | | | | | | | This caused python-apt to unset the Python signal handler when running update or install commands, breaking KeyboardInterrupt amongst possibly other things. We do not set those signal handlers in this functions, and the calling functions restore signal handlers to previous ones. LP: #1898026
| * Russian translation updateАлексей Шилин2021-02-241-10/+8
| | | | | | | | Closes: #983348
| * Remove .travis.yml, we are using GitLab CI these daysJulian Andres Klode2021-02-241-23/+0
| |
| * configuration: Add missing #include <array>Julian Andres Klode2021-02-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As user "DaOfficialRolex" on GitHub pointed out: This is needed to allow for APT on iOS to compile correctly. If not included the two following errors happen while compiling APT. ~/apt/apt-pkg/contrib/configuration.cc:900:44: error: constexpr variable cannot have non-literal type 'const std::array<APT::StringView, 3>' constexpr std::array<APT::StringView, 3> magicComments { "clear"_sv, "include"_sv, "x-apt-configure-index"_sv }; ^ ~/apt/apt-pkg/contrib/configuration.cc:900:44: error: implicit instantiation of undefined template 'std::__1::array<APT::StringView, 3>' /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tuple:219:64: note: template is declared here template <class _Tp, size_t _Size> struct _LIBCPP_TEMPLATE_VIS array; ^
| * Branch of as 2.2.y for bullseyeJulian Andres Klode2021-02-232-3/+3
| |
| * Do not require force-loopbreak on Protected packagesJulian Andres Klode2021-02-233-5/+61
|/ | | | | | | | | dpkg will be changed in 1.20.8 to not require --force-remove for deconfiguration anymore, but we want to decouple our changes from the dpkg ones, so let's always pass --force-remove-protected when installing packages such that we can deconfigure protected packages. Closes: #983014
* Release 2.2.02.2.0Julian Andres Klode2021-02-185-6/+15
|
* symbols: Remove spurios package line, add kernel autoremoval helperJulian Andres Klode2021-02-181-1/+1
|
* kernels: Avoid std::regex for escaping '.' and '+'Julian Andres Klode2021-02-121-6/+10
| | | | | | | | | std::regex pulls in about 50 weak symbols which is complete and utter madness, especially because we version all our symbols, so no other library could ever reuse them. Avoid using the regular expression here all together, loop using string::find_first_of() and insert backslashes with strng::insert().
* Do not make DefaultRootSetFunc2 public symbolJulian Andres Klode2021-02-121-1/+1
|
* Release 2.1.202.1.20Julian Andres Klode2021-02-105-6/+15
|
* Adjust loops to use size_t instead of intJulian Andres Klode2021-02-091-3/+3
| | | | Gbp-Dch: ignore
* Fix test suite regression from StrToNum fixesJulian Andres Klode2021-02-092-56/+44
| | | | | | | | | | | | We ignored the failure from strtoul() that those test cases had values out of range, hence they passed before, but now failed on 32-bit platforms because we use strtoull() and do the limit check ourselves. Move the tarball generator for test-github-111-invalid-armember to the createdeb helper, and fix the helper to set all the numbers for like uid and stuff to 0 instead of the maximum value the fields support (all 7s). Regression-Of: e0743a85c5f5f2f83d91c305450e8ba192194cd8
* CI: Run test as user on i386Julian Andres Klode2021-02-091-0/+1
|
* Release 2.1.192.1.19Julian Andres Klode2021-02-0948-270/+249
|
* Merge branch 'pu/fuzzerpatches' into 'master'Julian Andres Klode2021-02-0930-426/+524
|\ | | | | | | | | Various patches uplifted from unfinished fuzzer branches See merge request apt-team/apt!158
| * Prevent temporary directory from triggering failure greppingDavid Kalnischkies2021-02-041-0/+1
| | | | | | | | | | | | | | | | The grep for case-insensitive GPG finds also e.g. "/tmp/tmp.Kc5kKgPg0D" which is not the intention, so we simply eliminate the variation of the /tmp directory here from the output to prevent these false positives. Gbp-Dch: Ignore
| * Remove spurious periods on progress strings in po/de.poDavid Kalnischkies2021-02-041-2/+2
| | | | | | | | | | | | They result in what looks odd: Abhängigkeitsbaum wird aufgebaut.… Fertig Statusinformationen werden eingelesen.… Fertig
| * Use size of the old cache as APT::Cache-Start defaultDavid Kalnischkies2021-02-042-5/+29
| | | | | | | | | | | | | | Depending on your configured source 25 MB is hardly enough, so the mmap housing the cache while it is build has to grow. Repeatedly. We can cut down on the repeats of this by keeping a record of the size of the old cache assuming the sizes will remain roughly in the same ballpark.
| * Limit on first patch size only for server-merged patchesDavid Kalnischkies2021-02-041-17/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | APT tries to detect if applying patches is more costly than just downloading the complete index by combining the size of the patches. That is correct for client-side merging, but for server-side merging we actually don't know if we will jump directly from old to current or have still intermediate steps in between. With this commit we assume it will be a jump from old to current through as that is what dak implements and it seems reasonable if you go to the trouble of server side merging that the server does the entire merging in one file instead of leaving additional work for the client to do. Note that this just changes the heuristic to prevent apt from discarding patches as uneconomic in the now more common one merged-patch style, it still supports multiple merged patches as before. To resolve this cleanly we would need another field in the index file declaring which hash we will arrive at if a patch is applied (or a field differentiating between these merged-patch styles at least), but that seems like overkill for now.
| * Ensure HTTP status code text has sensible contentDavid Kalnischkies2021-02-042-1/+4
| | | | | | | | | | | | | | We use the code in error messages, so at least for that edgecase we should ensure that it has sensible content. Note that the acquire system aborts on non-sensible message content in SendMessage, so you can't really exploit this.
| * Replace PrintStatus with SendMessage usageDavid Kalnischkies2021-02-048-43/+73
| | | | | | | | | | | | | | varg API is a nightmare as the symbols seems different on ever other arch, but more importantly SendMessage does a few checks on the content of the message and it is all outputted via C++ iostreams and not mixed in FILE* which is handy for overriding the streams.
| * Use error reporting instead of assert in rred patchingDavid Kalnischkies2021-02-041-68/+88
| | | | | | | | | | | | | | | | | | | | | | The rest of our code uses return-code errors and it isn't that nice to crash the rred method on bad patches anyway, so we move to properly reporting errors in our usual way which incidently also helps writing a fuzzer for it. This is not really security relevant though as the patches passed hash verification while they were downloaded, so an attacker has to overtake a trusted repository first which gives plenty better options of attack.
| * Avoid undefined pointer arithmetic while growing mmapDavid Kalnischkies2021-02-044-46/+29
| | | | | | | | | | | | | | | | | | The undefined behaviour sanitizer complains with: runtime error: addition of unsigned offset to 0x… overflowed to 0x… Compilers and runtime do the right thing in any case and it is a codepath that can (and ideally should) be avoided for speed reasons alone, but fixing it can't hurt (too much).
| * Avoid overstepping bounds in config file parsingDavid Kalnischkies2021-02-033-60/+88
| | | | | | | | | | | | | | Our configuration files are not security relevant, but having a parser which avoids crashing on them even if they are seriously messed up is not a bad idea anyway. It is also a good opportunity to brush up the code a bit avoiding a few small string copies with our string_view.
| * Forbid negative values in unsigned StrToNum explicitlyDavid Kalnischkies2021-02-032-25/+72
| | | | | | | | | | | | | | strtoul(l) surprises us with parsing negative values which should not exist in the places we use to parse them, so we can just downright refuse them rather than trying to work with them by having them promoted to huge positive values.
| * Remove Word size limit from ParseQuote and CWordDavid Kalnischkies2021-02-031-26/+17
| | | | | | | | | | | | It isn't super likely that we will encounter such big words in the real world, but we can return arbitrary length, so lets just do that as that also means we don't have to work with a second buffer.
| * Don't parse \x and \0 past the end in DeEscapeStringDavid Kalnischkies2021-02-032-4/+14
| | | | | | | | | | | | This has no attack surface though as the loop is to end very soon anyhow and the method only used while reading CD-ROM mountpoints which seems like a very unlikely attack vector…
| * Fix incorrect base64 encoding due to int promotionDavid Kalnischkies2021-02-032-1/+5
| | | | | | | | | | | | | | For \xff and friends with the highest bit set and hence being a negative value on signed char systems the wrong encoding is produced as we run into undefined behaviour accessing negative array indexes. We can avoid this problem simply by using an unsigned data type.
| * Retire and deprecate _strtabexpandDavid Kalnischkies2021-02-033-40/+15
| | | | | | | | | | | | | | | | If the Configuration code calling this was any indication, it is hard to use – and even that monster still caused heap-buffer-overflow errors, so instead of trying to fix it, lets just use methods which are far easier to use. The question why this is done at all remains, but is left for another day as an exercise for the reader.
| * Add a simple test for APT::String::DisplayLengthDavid Kalnischkies2021-02-031-0/+17
| | | | | | | | | | References: 2497198e9599a6a8d4d0ad08627bcfc7ea49c644 Gbp-Dch: Ignore
| * Fail ConfigDir reading if directory listing failedDavid Kalnischkies2021-02-031-1/+4
| | | | | | | | | | | | We were printing an error and hence have non-zero exit code either way, but API wise it makes sense to have this properly reported back to the caller to propagate it down the chain e.g. while parsing #include stanzas.
| * Free XXH3 state to avoid leak in cache hashingDavid Kalnischkies2021-02-031-1/+3
| | | | | | | | | | We do this once (usually), so the leak is tremendously big, but it is detected as a leak by the fuzzer and trips it up.
| * Remove unused config check for pre-XXH3 cache hashingDavid Kalnischkies2021-02-033-45/+0
| | | | | | | | | | References: 1460eebf2abe913df964e031eff081a57f043697 Gbp-Dch: Ignore
| * Guess compressor only if no AR nember with exact name existsDavid Kalnischkies2021-02-022-26/+35
| | | | | | | | | | | | | | | | | | | | | | Explicitly opening a tar member is a bit harder than it needs to be as you have to remove the compressor extension so that it can be guessed here gain potentially choosing the wrong member. Doesn't really matter for deb packages of course as the member count is pretty low and strongly defined, but testing is easier this way. It also finally fixes an incorrectly formatted error message.
| * Use 500 MB memory limit for xz/lzma decodingDavid Kalnischkies2021-02-021-15/+6
| | | | | | | | | | | | | | | | | | | | | | | | The buffers we feed in and read out are usually a couple kilobytes big so allowing lzma to use an unlimited amount of memory is easy & okay, but not needed and confuses memory checkers as it will cause lzma to malloc a huge chunk of memory (which it will never use). So lets just use a "big enough" value instead. In exchange we simplify the decoder calling as we were already using the auto-variant for xz, so we can just avoid the if-else and let liblzma decide what it decodes.
* | Merge branch 'bugfix/deptreedone' into 'master'Julian Andres Klode2021-02-091-0/+3
|\ \ | | | | | | | | | | | | Show 'Done' always for 'Building dependency tree' See merge request apt-team/apt!157
| * | Show 'Done' always for 'Building dependency tree'David Kalnischkies2021-02-041-0/+3
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For years I subconsciously thought this is wrong but ignored it: $ LANG=C apt install -s Reading package lists... Done Building dependency tree Reading state information... Done Then I noticed: $ LANG=C apt install -s -o dir::state::extended_states=/dev/null Reading package lists... Done Building dependency tree... Done That can't be! Then it really should be: $ LANG=C apt install -s Reading package lists... Done Building dependency tree... Done Reading state information... Done This oddity seems to be in since the introduction of the auto bit in 2005 which makes this rather hard to solve in theory, but in practice no front end seems to call the readStateFile method directly, so we might actually be lucky. The alternative would be to call Done in the calling method and again at the end of readStateFile while dropping it from the current place, but as that is more shuffling around it could be more upsetting for other front ends. Not sure, but now that I have seen it, I want to have it fixed one way or another… aptitude at least seems not to explode. References: afb1e2e3bb580077c6c917e6ea98baad8f3c39b3
* | Dutch manpages translation updateFrans Spiesschaert2021-02-041-82/+63
| | | | | | | | Closes: #981883
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2024-04-27 01:19:05 +0000