summaryrefslogtreecommitdiff
path: root/test
Commit message (Collapse)AuthorAgeFilesLines
* solver3: Replace PropagateInstall() with Discover()Julian Andres Klode2025-02-071-0/+1
| | | | | | | | | | | | | | | Just propagate the stored clauses after we have discovered them; this is quite straightforward. We now more reliably discover common dependencies at the package level, adjust the test case accordingly. The next step is to make discovery recursive, or iterative, to build an entire recursive tree from all roots, and then we can reject reverse dependencies based on it. A bunch of refactorings are needed in the process. We remove the useless Hint enumeration and insert a flags struct into the State, such that we can record whether a package/version has been discovered, to avoid spending double time on discovery.
* solver3: Extract TranslateOrGroup() out of EnqueueOrGroup()Julian Andres Klode2025-02-071-1/+1
| | | | | | | This is not *purely* a refactoring, we accidentally used the version of the dependency when enqueuing conflicts rather than the reason, so the conflict string in the test case is different; the logging had the same issue.
* Also support debian-ports modernizationSamuel Thibault2025-02-021-0/+12
| | | | That just needs using '-' to replace spaces in Origin to get the keyring file name.
* solver3: Remove fixed test cases from broken listJulian Andres Klode2025-01-301-2/+0
|
* solver3: Use a propagation queueJulian Andres Klode2025-01-3012-33/+21
| | | | | | | | | | | | | | | | | | | Instead of directly propagating in a recursive fashion, queue propagations in a queue and work on them in a loop per the miniSAT paper. We call Propagate() only at the end of the FromDepCache() function and then in the Solve loop. Delaying the initial propagation means that we get a stronger reasoning: Assume you have x->a->b->c, y->c and you install x,y: - Previously we traversed: x, y, x->a, a->b, b->c, (y->c) - but now we traverse: x, y, x->a, y->c, a->b, (b->c) Notably c now has the implication y->c instead of x->a->b->c. Inside the solver we need to call Propagate in a loop: Propagating facts can fail and we then backtrack. If backtracking is succesful, we have gained a new fact to propagate.
* solver3: Remove NewUnsatRecommends groupJulian Andres Klode2025-01-301-0/+46
| | | | | | This is more or less unused; but it particularly has the bad problem of forcing new unsat recommends to be solved *before* dependencies. Which is awkward.
* solver3: Soft unit propagation for optional itemsJulian Andres Klode2025-01-301-1/+7
| | | | | If both items are optional, unit items should be processed first.
* Make notices about .list sources without signed-by less obnoxiousJulian Andres Klode2025-01-281-8/+12
|
* apt: Introduce the modernize-sources commandJulian Andres Klode2025-01-281-0/+85
| | | | | | | | | | | | | | | | | | | | | | This handles rewriting .list entries to .sources ones; the only options handled are - signed-by => Signed-By - trusted => Trusted The options to specify architectures, check-valid-until, etc. are not supported as they are not exposed in the code. We probably should refactor the parser at some point so we can actually read the parsed entry rather than the abstract objects we create from it. This can be improved further: - Keyrings from /etc/apt/trusted.gpg.d should be moved to /etc/apt/keyrings. - automatic detection by trying to verify using each keyring in trusted.gpg.d until we find the right one and then using that for Signed-By rather than leave some sources empty.
* apt-secure manual page is in manual section 8Andreas Rönnquist2025-01-281-2/+2
|
* sqv, gpgv: Do not fail if Dir::Etc::Trusted is set and trusted.gpg.d is missingJulian Andres Klode2025-01-221-0/+48
| | | | | | | | | | This should remain a warning case for now. So push the errors to a stack and see if we can find any file. Explictly warn again if Dir::Etc::trusted is set, as it is like _super_ deprecated. Reported-By: Cyril Brulebois <kibi@debian.org> in Bug#1032131
* Remove superseded warning about trusted.gpg fallbackJulian Andres Klode2025-01-211-65/+0
| | | | | | | | | | This warning has been superseded by the missing sources.list notice (which will also become a warning shortly) Adjust the sqv exit to avoid introducing a new spurious "No good signature" that we did not reach before. moo
* Warn about missing Signed-By in .list formatJulian Andres Klode2025-01-212-2/+13
| | | | | | Warn about the missing field there and suggest the transition to deb822 .sources files if we found any non-deb822 source without signed-by.
* Grep for more evil in test as a signature can contain evilDavid Kalnischkies2025-01-171-2/+2
| | | | | | | | The test was failing for me as the PGP signature contained "GeviljGB" which is clearly not the evil we are looking for. Sadly, reproducing this is hard, but random failures of the test suite are annoying, so lets grep for a bit longer strings that can't happen in the signature block to avoid this particular problem.
* Write Pre-Depends with a dash again in EDSPDavid Kalnischkies2025-01-171-5/+24
| | | | | | | | | | | | | | For some reason ~ that I don't know and can't come up with on the spot ~ we output the Pre-Depends field as PreDepends in the translated user-visible output and I accidentally copied that over to the untranslated variant that replaced the embedded copy (with a dash) in the EDSP generator. Code-copies and implementation details littered all over the place are apparently sometimes a good bug-free thing, too… Regression-of: 6828ae2c2f9268c8187f0fa91b3c464ed84a8476 Closes: #1093254
* t/i/test-method-gpgv-legacy-keyring: Switch to http, extendJulian Andres Klode2025-01-141-28/+41
| | | | | Add some more test cases where only one of the sources is bad; and switch to htp to make the order more reliable.
* Drop SourceListWarnings::NonFreeFirmware messagesDavid Kalnischkies2025-01-051-36/+0
| | | | | | | | | They were implemented to help a tiny bit in notifying users even if they didn't read the Release notes (or didn't realize it would apply to them) that they likely have to change their sources.list entries, but that transition is now over for a while, so everyone who should have seen it has hopefully reacted to it and so all this code now does is waste space, time and idly waiting for false positives.
* Drop usage of macro APT_OVERRIDE for simple overrideDavid Kalnischkies2025-01-055-14/+14
| | | | | | | | | We were rather inconsistent in using it and as our public headers contain deduction guides (a c++17 feature) it seems silly to try to hide a c++11 feature in a macro, so lets stop this charade and drop the macro and while we are changing all these lines lets apply [[nodiscard]] (another c++17 feature) and other suggestions from clang-tidy and formatting for a little more consistency.
* Resolve autopkgtest failures caused by dpkg >= 1.22.12David Kalnischkies2025-01-053-8/+12
| | | | | | | | | | | | | | | | | | | | Streamlining the rules around the Section field and especially what happens if its isn't present are a good idea in general, but for testing proposes we want a package without a section ~ something dpkg and co do not want us to have for good reasons (outside of tests). While a real nosection package would interact with more of our code, for this test we don't really need a real one and so we can adopt to the new reality of encountering such packages only if they are already installed instead of trying to build a sectionless package by hand. The other failure is a simple and trivial string change around the formatting of the version number in dpkg error message about file overrides, which we can resolve by just not looking at the version to sidestep dealing with the formatting difference. References: https://lists.debian.org/debian-dpkg/2024/12/msg00010.html Closes: #1092090
* Ensure we can build with APT_PKG_ABI > 601Julian Andres Klode2025-01-051-1/+4
| | | | | There were some references to APT::StringView that don't make much sense anymore.
* test: Check "${METHODSDIR}/sqv" not "/usr/bin/sqv"Julian Andres Klode2025-01-013-4/+4
| | | | | This is the correct check to find out whether we have built with sqv support.
* sqv: Avoid propagating errors between filesJulian Andres Klode2025-01-011-4/+19
| | | | | | | | | | | We inadvertently did not discard pending errors when the fallback to Dir::Etc::trusted succeeded, do so. Also add some quick safety checks to make sure we enter the verification in a good state, and that if somehow errors are pending after a verification we return false. Closes: #1091679
* methods: Add new sqv methodJulian Andres Klode2024-12-226-44/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new sqv method uses sequoia's sqv tool to verify files. The tool's interface is quite simple: It returns 0 on success, and prints one line per good signer with the fingerprint. sqv has a configurable crypto policy. We have defined apt-specific override mechanisms for sequoia's standard policy, allowing both users, distributions, and apt package to provide overrides for Sequoia's default policy in meaningful ways. The sqv method will be built and be the standard method for verifying OpenPGP signatures provided that `sqv` is in the PATH during building. It is not built if there was no sqv at build time, so you need a rebuild to enable sqv later on. On the flip side, the gpgv method is always built, but it does need to be always built: If APT::Key::GPGVCommand is set, we need to fallback to it - this is important to support existing users of that interface such as mmdebstrap. Also we want to fall back to it when /usr/bin/sqv disappears - for example in our CI :D A couple of concessions have been made for test suite purposes: - Failure to split a clearsigned file only shows the summary, as the gpgv method also only showed it, and no details why it failed. - We write "Got GOODSIG <fingerprint>" in debug mode to mimic the gpgv code to keep the test suite happy. - In various places in the test suite we assert minimal output from sqv, but sqv's human output is not intended to be stable. This will incur additional work when it breaks. However we do not _parse_ the output, so actual operation of apt is unaffected. A couple of things are suboptimal here: - We are still doing clearsigned splitting ourselves. sqv only has support for detached signatures right now, whereas sqopv has support for clearsigned files as well (but sqopv does not provide any reasons for why signature verification fails or means to set a policy). - Deprecation of algorithms happens on a timebomb basis in sequoia. We have no means to give users warnings ahead of time if their configuration is outdated. We have implemented various bits that probably should be going away: - Fallback to trusted.gpg This is just annoying... - Support for fingerprints in Signed-By (no subkey matching though) The extra work here is arguably less compared to gpgv. - Check the keyring for correctness. With Signed-By everywhere, we should just error out rather than skip broken keyrings. Moo
* Use sq in the test suite, remove apt-keyJulian Andres Klode2024-12-1910-964/+100
| | | | | | | | | | | | | Remove the test case for MD5 and expired signatures, as we can't create them (can't set signing digest, and can't set signature expiry). Tests for them have been added to test-method-gpgv instead. We override sq in a function with cert-store and key-store set to none. This supports both sq 0.40 and sq 1.0.
* Remove legacy apt-key testsJulian Andres Klode2024-12-192-469/+0
|
* apt show: Add pinning information for --fullJulian Andres Klode2024-12-191-0/+3
| | | | | Add new fields `APT-Pin`, `APT-Candidate`, and `APT-Release` to the output, same as in EDSP (limited to one release for now).
* Be more strict about which PAGER to execvp() directlyJulian Andres Klode2024-12-191-0/+11
| | | | | | | | | | | There could be | or ; or other stuff in PAGER that needs special handling, let's encode a list of known-good characters instead. This covers all known pagers. You could also try to execvp() PAGER in its entirety and if it fails with ENOENT fall back to the shell but then you don't get correct error handling if the pager really doesn't exist (in which case we should abort trying to page).
* Introduce automatic pager for read commandsJulian Andres Klode2024-12-182-2/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Automatically show the output of `show`, `policy`, `list`, `search`, `showsrc` in a pager. The pager setup is inspired by git's pager setup. Notably, the pager is found using APT_PAGER and PAGER variables. We wait for the pager to be setup somewhat correctly by using a notify pipe to figure out whether execvp() was succesful - then the pipe will read EOF as the other end got closed by CLOEXEC during exec - or not, then the pipe will contain an errno. We set up the correct handlers for signals and exit to close the fds and wait for the pager. Notably inside the signal handler we cannot flush our streams, only close them, so there is some duplication. We call the InitOutputPager() function from inside the various Do...() functions rather than setting it up generally in InitOutput(). Doing so allows us to first render the progress without a pager, and then setup the pager for the content only which improves user experience. When we setup a pager we also take care to disable standard input, as we should not be prompting users while a pager is running (the pager will be reading from the tty directly). We do this by dup2-ing() a /dev/null over it; if we just close()d the fd, another open() might reuse the fd number and problems could occur.
* Fix out-of-bounds read in `apt show :`Julian Andres Klode2024-12-161-0/+4
| | | | | | | | We tried to access str.size() - 1 on a 0-sized string, which does not work, and now that we build with assertions triggers it. Closes: #1090068
* methods/gpgv: Remove apt-key mentionsJulian Andres Klode2024-12-071-2/+2
|
* Stop installing apt-key, make it a test suite helperJulian Andres Klode2024-12-074-44/+849
|
* gpgv: Merge all keyrings into a single one againJulian Andres Klode2024-12-041-2/+3
| | | | | | | | | | | | | | | gpgv is quite picky, it takes at most 40 keyrings but it also doesn't even take keyrings with : in the name, generating an ERROR message instead. So instead let's always merge the configured keyrings for the source into a single one. While we are at it, since we are merging now, let's also add some rudimentary error checking to what we write into the merged keyring: If copying into it fails, we should abort. Reported-by: SuShY on IRC
* solver3: Reject all non-candidates outright for strict pinningJulian Andres Klode2024-12-032-3/+6
| | | | | | | | Reimplement strict pinning by rejecting the non-candidates when translating the problem from the depcache to the solver. This is substantially better than restricting the list of alternatives for an or group to only include allowed ones for debugging purposes, albeit a bit slower.
* solver3: Fix intransitivity of version comparison on upgradeJulian Andres Klode2024-12-021-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | We only compared candidate to installed version, but candidate should dominate all versions, otherwise we end up in the fancy problem of elpa-notmuch in upgrade-noble-t64-remove-desktop-2024-03-29.edsp Where we had three versions: not-installable 0.38.3-1ubuntu1 candidate 0.38.2-1.1ubuntu2 installed 0.38.2-1ubuntu2 And received an ordering: installed > non-installable > candidate despite candidate > installed This is only visible with no-strict-pinning right now, as we are otherwise filtering out invalid choices (and hence we only have candidate and installed otherwise).
* fileutl: Use execvp() in Popen() instead of execv()Julian Andres Klode2024-11-281-1/+1
|
* Directly call gpgv instead of apt-keyJulian Andres Klode2024-11-282-19/+12
|
* strutl: Add Base64DecodeJulian Andres Klode2024-11-281-1/+20
| | | | | | | | Lifted from the dpkg mtree code that mjg59 submitted in https://lists.debian.org/debian-dpkg/2018/05/msg00005.html Which as dpkg is GPL-2.0+ also is GPL-2.0+
* test-apt-cdrom: Hande assert-pubkey-algo like other gpgv messagesJulian Andres Klode2024-11-241-18/+2
| | | | | | Not all gpgv implementations support that flag and we no longer default to gpgv; so we may not have support for it, so let's just handle it like other gpgv messages.
* Collect unprinted Ign errors for display in Err outputDavid Kalnischkies2024-11-222-2/+42
| | | | | | | | | | | | | | | | Instead of deciding if the first or the last error in a retry chain is the best one to display if the entire chain fails in the end we teach our progress reporting to collect the ignored errors (if it hasn't shown them) and print them all together in the Err line. We de-duplicate the collection which makes it harder to match the Ign line with the error messages, but for these cases its probably better to enable direct display of the text anyhow – and repeating the same message multiple times is boring and probably not that helpful for a user wanting to know why all the downloads failed even they asked apt to retry 429 times… References: 841243df877b39dfcb907a7fab9c4e8b4f65d715
* Merge branch 'fix/partialfilemirror' into 'main'Julian Andres Klode2024-11-225-55/+159
|\ | | | | | | | | Support uncompressed indexes from partial file:/ mirrors See merge request apt-team/apt!235
| * Look at non by-hash paths in copy and file methodsDavid Kalnischkies2024-11-224-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ideally copy and file mirrors would support by-hash as well, but its harder to setup and maintain especially if you want to cache an online mirror who has by-hash enabled. We can avoid unneeded roundtrips (in the best case) and entire cache misses (in the usual worst case) by "just" telling methods that the URI we passed it has the requested file perhaps also in other paths. This is done in pseudo-relative paths as we would otherwise need to teach redirection code to rewrite those URIs as well. A method like http can easily ignore this value and await explicit instructions to look at that file, but inspecting the path in local sources via file or copy is (comparatively) free, so we just do it immediately. If that ends up being the wrong version of the file as by-hash would have protected us from we are in this feature branch now falling back to other mirrors which are like the ones online and in support of by-hash.
| * Allow HashSum mismatches to fallback to other mirrorsDavid Kalnischkies2024-11-221-0/+30
| | | | | | | | | | | | | | | | | | | | | | Perhaps a common error in the past nowadays thanks to by-hash we rarely get mismatches due to our calling behaviour and more because the mirror is genuinely bad especially if this mirror didn't work with by-hash. Using our "normal" error handling allows falling back to other mirrors which hopefully do support by-hash and/or are not broken on a filesystem level making us more resistent to the occasional bad apple in a wild mirror forest.
| * Do not assume mirror-URIs end in a filename causing a hangDavid Kalnischkies2024-11-221-1/+14
| | | | | | | | | | | | | | In practice most of them will, but making the acquire process hang in response if they don't seems like an overly excessive response if we can just support it gracefully to reply with an error (or actually succeed in the unlikely event this URI is actually correct).
| * Allow AutoDetectProxy to work with more than just http(s)David Kalnischkies2024-11-221-12/+36
| | | | | | | | | | | | | | | | It is a bit unfair for third-party methods wrapping e.g. http to not have access to such tools, same for our old ftp and e.g. our tor even if in practice this isn't used much even for http. At least that is one less FIXME in the code.
| * Alternatively calculate alternative file hashes in file methodDavid Kalnischkies2024-11-221-1/+2
| | | | | | | | | | | | | | If we do not have the requested file we haven't calculated the hashes for it either and it is likely that the alternative file will be used, so to save the main thread from being busy with calculating hashes we do the calculation here in the method.
| * Show hashes calculated by main thread in error messagesDavid Kalnischkies2024-11-221-0/+1
| | | | | | | | | | | | | | | | | | | | | | The main thread calculates the hash sums if the method hasn't to be able to verify the files, but while a failure is detected and reported, the error messages did not include the hash sums which looks at least puzzling. So to have a complete error message we ensure that the hashes we calculated are appended to the message from the method so that it looks for the higher level code as if the method had calculated the hashes.
| * Support uncompressed indexes from partial file:/ mirrorsDavid Kalnischkies2024-11-222-31/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a file:/ mirror does not contain the requested compressed file, but the uncompressed variant of that file the method will sent us a fitting Done message with data only about the Alt-Filename, but the central hash verification code in the acquire system does not expect that looking for the non-existent compressed file to compare its hashes with the expected result – which fails (if we are verifying at least). That file:/ as well as the item code deals with this suggests that this feature was broken by 448c38bdcd72b52f11ec5f326f822cf57653f81c, but I couldn't (easily) compile that to verify, so this is only a guess. Supporting this allows using index files from /var/lib/apt/lists – which are likely uncompressed or can at least made so easily – to construct a file:/ mirror.
* | Merge branch 'test-gpgv-sq' into 'main'Julian Andres Klode2024-11-223-8/+43
|\ \ | | | | | | | | | | | | Test gpgv-sq See merge request apt-team/apt!403
| * | test: Ensure we test both gpgv-g10code and gpgv-sqJulian Andres Klode2024-11-222-6/+16
| | |
| * | test: Accept empty output from gpgv/apt-keyJulian Andres Klode2024-11-221-0/+3
| | | | | | | | | | | | | | | Our internal code always generates ENODATA for empty output and this is checked by the previous commit, so this is entirely safe.
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-10-14 08:40:36 +0000