summaryrefslogtreecommitdiff
path: root/test/integration
Commit message (Collapse)AuthorAgeFilesLines
* t/i/test-method-gpgv-legacy-keyring: Switch to http, extendJulian Andres Klode2025-01-141-28/+41
| | | | | Add some more test cases where only one of the sources is bad; and switch to htp to make the order more reliable.
* Drop SourceListWarnings::NonFreeFirmware messagesDavid Kalnischkies2025-01-051-36/+0
| | | | | | | | | They were implemented to help a tiny bit in notifying users even if they didn't read the Release notes (or didn't realize it would apply to them) that they likely have to change their sources.list entries, but that transition is now over for a while, so everyone who should have seen it has hopefully reacted to it and so all this code now does is waste space, time and idly waiting for false positives.
* Resolve autopkgtest failures caused by dpkg >= 1.22.12David Kalnischkies2025-01-053-8/+12
| | | | | | | | | | | | | | | | | | | | Streamlining the rules around the Section field and especially what happens if its isn't present are a good idea in general, but for testing proposes we want a package without a section ~ something dpkg and co do not want us to have for good reasons (outside of tests). While a real nosection package would interact with more of our code, for this test we don't really need a real one and so we can adopt to the new reality of encountering such packages only if they are already installed instead of trying to build a sectionless package by hand. The other failure is a simple and trivial string change around the formatting of the version number in dpkg error message about file overrides, which we can resolve by just not looking at the version to sidestep dealing with the formatting difference. References: https://lists.debian.org/debian-dpkg/2024/12/msg00010.html Closes: #1092090
* test: Check "${METHODSDIR}/sqv" not "/usr/bin/sqv"Julian Andres Klode2025-01-013-4/+4
| | | | | This is the correct check to find out whether we have built with sqv support.
* sqv: Avoid propagating errors between filesJulian Andres Klode2025-01-011-4/+19
| | | | | | | | | | | We inadvertently did not discard pending errors when the fallback to Dir::Etc::trusted succeeded, do so. Also add some quick safety checks to make sure we enter the verification in a good state, and that if somehow errors are pending after a verification we return false. Closes: #1091679
* methods: Add new sqv methodJulian Andres Klode2024-12-226-44/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new sqv method uses sequoia's sqv tool to verify files. The tool's interface is quite simple: It returns 0 on success, and prints one line per good signer with the fingerprint. sqv has a configurable crypto policy. We have defined apt-specific override mechanisms for sequoia's standard policy, allowing both users, distributions, and apt package to provide overrides for Sequoia's default policy in meaningful ways. The sqv method will be built and be the standard method for verifying OpenPGP signatures provided that `sqv` is in the PATH during building. It is not built if there was no sqv at build time, so you need a rebuild to enable sqv later on. On the flip side, the gpgv method is always built, but it does need to be always built: If APT::Key::GPGVCommand is set, we need to fallback to it - this is important to support existing users of that interface such as mmdebstrap. Also we want to fall back to it when /usr/bin/sqv disappears - for example in our CI :D A couple of concessions have been made for test suite purposes: - Failure to split a clearsigned file only shows the summary, as the gpgv method also only showed it, and no details why it failed. - We write "Got GOODSIG <fingerprint>" in debug mode to mimic the gpgv code to keep the test suite happy. - In various places in the test suite we assert minimal output from sqv, but sqv's human output is not intended to be stable. This will incur additional work when it breaks. However we do not _parse_ the output, so actual operation of apt is unaffected. A couple of things are suboptimal here: - We are still doing clearsigned splitting ourselves. sqv only has support for detached signatures right now, whereas sqopv has support for clearsigned files as well (but sqopv does not provide any reasons for why signature verification fails or means to set a policy). - Deprecation of algorithms happens on a timebomb basis in sequoia. We have no means to give users warnings ahead of time if their configuration is outdated. We have implemented various bits that probably should be going away: - Fallback to trusted.gpg This is just annoying... - Support for fingerprints in Signed-By (no subkey matching though) The extra work here is arguably less compared to gpgv. - Check the keyring for correctness. With Signed-By everywhere, we should just error out rather than skip broken keyrings. Moo
* Use sq in the test suite, remove apt-keyJulian Andres Klode2024-12-198-116/+100
| | | | | | | | | | | | | Remove the test case for MD5 and expired signatures, as we can't create them (can't set signing digest, and can't set signature expiry). Tests for them have been added to test-method-gpgv instead. We override sq in a function with cert-store and key-store set to none. This supports both sq 0.40 and sq 1.0.
* Remove legacy apt-key testsJulian Andres Klode2024-12-192-469/+0
|
* apt show: Add pinning information for --fullJulian Andres Klode2024-12-191-0/+3
| | | | | Add new fields `APT-Pin`, `APT-Candidate`, and `APT-Release` to the output, same as in EDSP (limited to one release for now).
* Be more strict about which PAGER to execvp() directlyJulian Andres Klode2024-12-191-0/+11
| | | | | | | | | | | There could be | or ; or other stuff in PAGER that needs special handling, let's encode a list of known-good characters instead. This covers all known pagers. You could also try to execvp() PAGER in its entirety and if it fails with ENOENT fall back to the shell but then you don't get correct error handling if the pager really doesn't exist (in which case we should abort trying to page).
* Introduce automatic pager for read commandsJulian Andres Klode2024-12-182-2/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Automatically show the output of `show`, `policy`, `list`, `search`, `showsrc` in a pager. The pager setup is inspired by git's pager setup. Notably, the pager is found using APT_PAGER and PAGER variables. We wait for the pager to be setup somewhat correctly by using a notify pipe to figure out whether execvp() was succesful - then the pipe will read EOF as the other end got closed by CLOEXEC during exec - or not, then the pipe will contain an errno. We set up the correct handlers for signals and exit to close the fds and wait for the pager. Notably inside the signal handler we cannot flush our streams, only close them, so there is some duplication. We call the InitOutputPager() function from inside the various Do...() functions rather than setting it up generally in InitOutput(). Doing so allows us to first render the progress without a pager, and then setup the pager for the content only which improves user experience. When we setup a pager we also take care to disable standard input, as we should not be prompting users while a pager is running (the pager will be reading from the tty directly). We do this by dup2-ing() a /dev/null over it; if we just close()d the fd, another open() might reuse the fd number and problems could occur.
* Fix out-of-bounds read in `apt show :`Julian Andres Klode2024-12-161-0/+4
| | | | | | | | We tried to access str.size() - 1 on a 0-sized string, which does not work, and now that we build with assertions triggers it. Closes: #1090068
* methods/gpgv: Remove apt-key mentionsJulian Andres Klode2024-12-071-2/+2
|
* Stop installing apt-key, make it a test suite helperJulian Andres Klode2024-12-072-44/+1
|
* gpgv: Merge all keyrings into a single one againJulian Andres Klode2024-12-041-2/+3
| | | | | | | | | | | | | | | gpgv is quite picky, it takes at most 40 keyrings but it also doesn't even take keyrings with : in the name, generating an ERROR message instead. So instead let's always merge the configured keyrings for the source into a single one. While we are at it, since we are merging now, let's also add some rudimentary error checking to what we write into the merged keyring: If copying into it fails, we should abort. Reported-by: SuShY on IRC
* solver3: Reject all non-candidates outright for strict pinningJulian Andres Klode2024-12-032-3/+6
| | | | | | | | Reimplement strict pinning by rejecting the non-candidates when translating the problem from the depcache to the solver. This is substantially better than restricting the list of alternatives for an or group to only include allowed ones for debugging purposes, albeit a bit slower.
* solver3: Fix intransitivity of version comparison on upgradeJulian Andres Klode2024-12-021-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | We only compared candidate to installed version, but candidate should dominate all versions, otherwise we end up in the fancy problem of elpa-notmuch in upgrade-noble-t64-remove-desktop-2024-03-29.edsp Where we had three versions: not-installable 0.38.3-1ubuntu1 candidate 0.38.2-1.1ubuntu2 installed 0.38.2-1ubuntu2 And received an ordering: installed > non-installable > candidate despite candidate > installed This is only visible with no-strict-pinning right now, as we are otherwise filtering out invalid choices (and hence we only have candidate and installed otherwise).
* Directly call gpgv instead of apt-keyJulian Andres Klode2024-11-282-19/+12
|
* test-apt-cdrom: Hande assert-pubkey-algo like other gpgv messagesJulian Andres Klode2024-11-241-18/+2
| | | | | | Not all gpgv implementations support that flag and we no longer default to gpgv; so we may not have support for it, so let's just handle it like other gpgv messages.
* Collect unprinted Ign errors for display in Err outputDavid Kalnischkies2024-11-222-2/+42
| | | | | | | | | | | | | | | | Instead of deciding if the first or the last error in a retry chain is the best one to display if the entire chain fails in the end we teach our progress reporting to collect the ignored errors (if it hasn't shown them) and print them all together in the Err line. We de-duplicate the collection which makes it harder to match the Ign line with the error messages, but for these cases its probably better to enable direct display of the text anyhow – and repeating the same message multiple times is boring and probably not that helpful for a user wanting to know why all the downloads failed even they asked apt to retry 429 times… References: 841243df877b39dfcb907a7fab9c4e8b4f65d715
* Merge branch 'fix/partialfilemirror' into 'main'Julian Andres Klode2024-11-225-55/+159
|\ | | | | | | | | Support uncompressed indexes from partial file:/ mirrors See merge request apt-team/apt!235
| * Look at non by-hash paths in copy and file methodsDavid Kalnischkies2024-11-224-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ideally copy and file mirrors would support by-hash as well, but its harder to setup and maintain especially if you want to cache an online mirror who has by-hash enabled. We can avoid unneeded roundtrips (in the best case) and entire cache misses (in the usual worst case) by "just" telling methods that the URI we passed it has the requested file perhaps also in other paths. This is done in pseudo-relative paths as we would otherwise need to teach redirection code to rewrite those URIs as well. A method like http can easily ignore this value and await explicit instructions to look at that file, but inspecting the path in local sources via file or copy is (comparatively) free, so we just do it immediately. If that ends up being the wrong version of the file as by-hash would have protected us from we are in this feature branch now falling back to other mirrors which are like the ones online and in support of by-hash.
| * Allow HashSum mismatches to fallback to other mirrorsDavid Kalnischkies2024-11-221-0/+30
| | | | | | | | | | | | | | | | | | | | | | Perhaps a common error in the past nowadays thanks to by-hash we rarely get mismatches due to our calling behaviour and more because the mirror is genuinely bad especially if this mirror didn't work with by-hash. Using our "normal" error handling allows falling back to other mirrors which hopefully do support by-hash and/or are not broken on a filesystem level making us more resistent to the occasional bad apple in a wild mirror forest.
| * Do not assume mirror-URIs end in a filename causing a hangDavid Kalnischkies2024-11-221-1/+14
| | | | | | | | | | | | | | In practice most of them will, but making the acquire process hang in response if they don't seems like an overly excessive response if we can just support it gracefully to reply with an error (or actually succeed in the unlikely event this URI is actually correct).
| * Allow AutoDetectProxy to work with more than just http(s)David Kalnischkies2024-11-221-12/+36
| | | | | | | | | | | | | | | | It is a bit unfair for third-party methods wrapping e.g. http to not have access to such tools, same for our old ftp and e.g. our tor even if in practice this isn't used much even for http. At least that is one less FIXME in the code.
| * Alternatively calculate alternative file hashes in file methodDavid Kalnischkies2024-11-221-1/+2
| | | | | | | | | | | | | | If we do not have the requested file we haven't calculated the hashes for it either and it is likely that the alternative file will be used, so to save the main thread from being busy with calculating hashes we do the calculation here in the method.
| * Show hashes calculated by main thread in error messagesDavid Kalnischkies2024-11-221-0/+1
| | | | | | | | | | | | | | | | | | | | | | The main thread calculates the hash sums if the method hasn't to be able to verify the files, but while a failure is detected and reported, the error messages did not include the hash sums which looks at least puzzling. So to have a complete error message we ensure that the hashes we calculated are appended to the message from the method so that it looks for the higher level code as if the method had calculated the hashes.
| * Support uncompressed indexes from partial file:/ mirrorsDavid Kalnischkies2024-11-222-31/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a file:/ mirror does not contain the requested compressed file, but the uncompressed variant of that file the method will sent us a fitting Done message with data only about the Alt-Filename, but the central hash verification code in the acquire system does not expect that looking for the non-existent compressed file to compare its hashes with the expected result – which fails (if we are verifying at least). That file:/ as well as the item code deals with this suggests that this feature was broken by 448c38bdcd72b52f11ec5f326f822cf57653f81c, but I couldn't (easily) compile that to verify, so this is only a guess. Supporting this allows using index files from /var/lib/apt/lists – which are likely uncompressed or can at least made so easily – to construct a file:/ mirror.
* | test: Ensure we test both gpgv-g10code and gpgv-sqJulian Andres Klode2024-11-222-6/+16
| |
* | test: Accept empty output from gpgv/apt-keyJulian Andres Klode2024-11-221-0/+3
| | | | | | | | | | Our internal code always generates ENODATA for empty output and this is checked by the previous commit, so this is entirely safe.
* | test-releasefile-verification: Test empty/invalid signaturesJulian Andres Klode2024-11-221-0/+22
| |
* | Accept BADSIG instead of EXPSIG for expired signaturesJulian Andres Klode2024-11-221-1/+1
| | | | | | | | Sequioa emits KEYEXPIRED + BADSIG instead of EXPSIG + VALIDSIG
* | test-releasefile-verification: Fix signature timeJulian Andres Klode2024-11-221-1/+1
|/ | | | | | The key was only generated in 2010 and only bound in 2016, so we cannot have signatures from before then, so update the key accordingly, and change the default expiry to 2017.
* test: Do not fail if valgrind is not availableJulian Andres Klode2024-11-211-2/+1
| | | | | | | | | | | | The command -v check inadvertently exited 0 because it confusingly also considers functions and aliases. Check if we are able to run a valgrind binary instead by querying its version. Then, the msgdebug in the alternative code path was included in the output that we are asserting so that was causing failures too and needs removing. Everything is horrible :D
* Merge branch 'ftparchive-buffers' into 'main'Julian Andres Klode2024-11-201-1/+72
|\ | | | | | | | | ftparchive: Remove fixed-size buffers See merge request apt-team/apt!388
| * ftparchive: writer: Remove line length limit on file listsJulian Andres Klode2024-11-201-1/+23
| | | | | | | | LP: #2087848
| * ftparchive: cachedb: Remove 600-byte key limitJulian Andres Klode2024-11-201-0/+32
| | | | | | | | | | | | | | | | | | | | Instead of snprintf() into a fixed-size buffer, construct an std::string using assign() and append(). A previous version used strprintf() which would have been hilariously inefficient. LP: #2087848
| * ftparchive: override: Remove fixed-size line buffersJulian Andres Klode2024-11-181-1/+18
| | | | | | | | LP: #2087848
* | Add a --comment option to record Comment: in historyJulian Andres Klode2024-11-181-0/+40
|/ | | | Suggested-by: mhoye on IRC
* http: add test for Retry-After handlingFelix Moessbauer2024-11-181-0/+9
| | | | | | | | | | When enabling Acquire::Retries::HandleRetryAfter, the value of the Retry-After http header is honored and the retry is not executed earlier than this point in time (except if that is longer than the max delay time). The added test checks if the request is delayed at least by the requested time. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
* Merge branch 'json-config' into 'main'Julian Andres Klode2024-11-141-42/+1088
|\ | | | | | | | | JSON: Pass options to hooks in hello message See merge request apt-team/apt!389
| * json: Add "options" array to JSON hooksJulian Andres Klode2024-11-121-0/+120
| | | | | | | | | | See the attached documentation. Note that defaults are missing, so using this is not particularly easy so far.
| * test: Prettify JSON tests using jqJulian Andres Klode2024-11-121-42/+968
| | | | | | | | | | | | | | | | This also ensures that our JSON is actually correct JSON. echo may fail now with EPIPE, given that it is multiline, so wrap it in 2>/dev/null
* | Only run valgrind in tests if valgrind available on hostнаб2024-11-112-7/+7
| |
* | Merge branch 'keep-more' into 'main'Julian Andres Klode2024-11-112-0/+48
|\ \ | | | | | | | | | | | | Fix keeping back removals of obsolete packages See merge request apt-team/apt!373
| * | Return an error if ResolveByKeep() is unsuccessfulJulian Andres Klode2024-11-111-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | In case of held packages or packages that otherwise can't be kept back, we should return an error and not require the client to check the broken_count after calling us. LP: #2078720
| * | Fix keeping back removals of obsolete packagesJulian Andres Klode2024-11-112-0/+30
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | ResolveByKeepInternal() inadvertently considered packages that were marked for removals as not being possible to keep back, but that's not true, all they need is to have a currently installed version. The test case is a bit awkward to construct because our upgrade code doesn't get into a place where this happens that we know off. Or maybe it does with phased upgrades, but here we simplify this by using a helper binary that dumps state, marks for removal, and then calls ResolveByKeep and checks the removal is undone. LP: #2078720
* / solver3: Introduce new Assume() and Enqueue() helpers and use themJulian Andres Klode2024-11-024-0/+14
|/ | | | | | | | | | | | | These are taken roughly from the MiniSAT paper. We still have a bit to go in actually encoding all clauses so the reasons are still variables, and Assume() isn't fully working yet. Adjust the existing Install()/Reject() code to use these functions, we already see additional lines in the log that we failed to log before, and this ensures more consistency. This is sort of still the wrong direction: Install()/Reject() do the propagation too; but that is tbd.
* test: Check that apt-ftparchive handles deep pathsJulian Andres Klode2024-10-151-0/+49
| | | | | | | | | This checks for the previous boundaries for path sizes; but we also added a check for many files in a directory, limited to 4096. Sadly the directory check did not fail with the old code either, it's hard to actually get it to fail correctly.
* test framework: Improve valgrind supportJulian Andres Klode2024-10-151-1/+4
| | | | | | Add support for testing apt-ftparchive, set valgrind to quiet such that we can check program output, and pass --error-exitcode=7 to get an (should be unused) magic error code if it detects errors.
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-10-14 11:27:28 +0000