summaryrefslogtreecommitdiff
path: root/test/integration
Commit message (Collapse)AuthorAgeFilesLines
* test: Improve output cleaning for solver progressJulian Andres Klode2025-02-181-12/+5
| | | | | | | | | | | We don't really care about solver progress messages in testing; hide them when comparing output, and hide them better than before. Previously testsuccessequal ... --solver 3.0 behaved differently than tests run with globally forced on 3.0 solver. Stop cleaning up autoremovals, but do clean up "Calculating upgrade" as well.
* solver3: Fix test-apt-move-and-forget-manual-sectionsJulian Andres Klode2025-02-181-1/+0
| | | | | Implement the moving of the auto bit. The whole auto-bit management is not entirely optimal yet, but this works.
* solver3: Only enqueue shared dependencies at the package levelJulian Andres Klode2025-02-1421-65/+64
| | | | | | | | | | | | | | | | | | | | | Dependencies shared by all versions are enqueued at the package level, so avoid enqueuing duplicates at the version level. This presumably has no meaningful impact on performance, potentially a negative performance impact on some workloads as we now need to find the duplicates again; it can become useful when there is a lot of backtracking. More importantly though this improves error messages, because now we can say that "all versions of foo depend on X", rather than saying "foo=1 depends on X" and you are left wondering why we did not select "foo=2". In this commit though, improved error messages are not implemented, they depend on redesigning the reason tracking to use clauses. Also the rationale tracking includes a lot more dependencies of the form "pkg:arch=version -> pkg:arch" which are annoying. Improved error messages should fold them into one node.
* solver3: Remove work rescoring in favor of unit propagationJulian Andres Klode2025-02-144-4/+2
| | | | | | | | | | | | | Instead of expensive rescoring of all outstanding items, use unit propagation to find new units after conflicts. We still count the items when adding them; but unless they are 0 or 1, which they should not be, they don't have any effect: The size field is now effectively static. If the size of an optional clause changed to 1, it is inserted a second time, and then moves up to the top of the optional items per the Work::operator< rules.
* test: Remove long obsolete FIXME for solver3Julian Andres Klode2025-02-141-2/+1
|
* test: Annotate the broken test casesJulian Andres Klode2025-02-141-4/+4
|
* test-ubuntu-bug-614993: Avoid implicit upgrade-by-source requirementJulian Andres Klode2025-02-142-11/+8
| | | | | | | This allows it to work with solver3 before that feature is implemented. This essentially reverts the change to that test made in 119a8d0ed19a4e946591b9aef7d662e14ca7ece5
* ResolveByKeep: Do not pass UPGRADE_ALL to EDSP solverJulian Andres Klode2025-02-141-1/+0
| | | | | | | | | | When we call an EDSP solver from ResolveByKeep(), we are not in an upgrade context: In an upgrade context, we call the solver directly from upgrade.cc instead of doing any Resolve() or ResolveByKeep() calls. This fixes the test-bug-602412-dequote-redirect test case, where ResolveByKeep() was called as part of --fix-missing
* test-dont-forget-conflicts-via-unknown-architectures: Use install, not upgradeJulian Andres Klode2025-02-142-2/+2
| | | | | The 3.0 solver does not support upgrade arguments to override the policy.
* Fix test-phased-updates-upgradeJulian Andres Klode2025-02-142-3/+2
| | | | | | | Our upgrade policy does not support specifying packagenames to upgrade commands that are not subject to the policy. It's possible this can be addressed at a later time, but let's not block the rest of the test from running.
* solver3: Fix ordering of or groupsJulian Andres Klode2025-02-142-10/+33
| | | | | | | | | | | | | | | | | | | | | | We inadvertently performed a global ordering of all possible solutions for the or group using CompareProviders3. This however is not correct, as we lose the ordering of the dependency group *too* much. Mostly this has no effect, but you can see for example in test-explore-or-groups-in-markinstall various instances of it. Adjust said test case to work with the 3.0 solver to the extent possible under the current design. The 3.0 solver does Recommends after processing any manually installed packages; as such the various Recommends test cases do not work: A `Recommends: okay|upgrade` will not upgrade `upgrade` if it visited `upgrade` first. This may change at a future time, but the correct semantics for Recommends are not entirely clear. Notably, the existing solver is not always consistent. You can see here where they matter, but recently I added test-solver-recommends-depends in which the Recommends do not influence the choice of other Depends.
* solver3: Implement phasing as part of strict pinning rulesJulian Andres Klode2025-02-141-4/+1
| | | | | | | | Phased updates are ignored when strict pinning is on; such that only the installed version will be available. By design of SAT solvers, this means that the version selection clause is unit, and hence the version can be directly propagated, i.e. that choice is safe.
* solver3: Correctly call MarkKeep() for kept back packagesJulian Andres Klode2025-02-141-11/+1
| | | | | | This fixes the difference in test-unpack-different-version-unpacked, but more importantly this is needed for phasing to be displayed correctly once that is implemented.
* test: Fix 7 test cases for 3.0 solver, annotate othersJulian Andres Klode2025-02-1410-36/+112
| | | | | Fix the test cases we can fix directly; add annotations where we know what work is needed, next we'll look at the other bugs.
* test: Support comments in --skip/--only filesJulian Andres Klode2025-02-141-2/+2
| | | | Strip everything after "#" and spaces before it.
* solver3: Install Essential on dist-upgradeJulian Andres Klode2025-02-142-3/+14
| | | | | | | | This is a bit gnarly, but dist-upgrade is mapped to is an upgrade + removals allowed + installs allowed :D
* solver3: Defer 3.0 'deep' autoremoval to 3.1, fix autoremoveJulian Andres Klode2025-02-142-7/+2
| | | | | | Restore the depcache's MarkRequired logic for 3.0 solver; and change the MarkInstall() call to pass a more correct value for FromUser, to not override an existing automatic status.
* solver3: Reject reverse conflicts on installJulian Andres Klode2025-02-143-9/+6
| | | | | | We forgot this in the previous iteration. This makes the reasoning in the test cases much nicer and apt-test's mantic-to-noble-jak.edsp now finishes rather than running into the timeout (potentially forever).
* solver3: Reject reverse dependencies nativelyJulian Andres Klode2025-02-145-18/+9
| | | | | | | | | | | | | | | | | | Instead of utilizing the reverse depends functionality of the cache and marking all possible reverse dependencies for removal, mark them ourselves by keeping track of reverse-implication-clauses. Notably, this improves the reverse dependency rejection substantially: The previous RejectReverseDependencies() function did not handle Provides. For this to work correctly right now, we need to discover optional clauses too when queuing them. This is somewhat suboptimal as we technically we don't care if they become unsat, we just waste time tracking them. The tests get a bit awkward, but oh well, we use what we can use.
* solver3: Discover recursive dependenciesJulian Andres Klode2025-02-141-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | When we have discovered all clauses for a version, discover each possible solution for the clauses. This means that when Discover(foo) is called _anything_ that could lead to foo becoming uninstallable is translated; so we can extend this next by keeping a list of reverse dependencies for each package and rejecting those. We limit the discovery to those variables that we did not already enqueue as a negative fact at the root level, as those can never become true. We are utilizing a queue here which is not the most performant solution possible, but where it excels is in producing usable stack traces when debugging. Traversing the entire dependency tree using recursion can easily produce thousand levels of recursion. The queue means that we discover packages in a breadth-first manner compatible with the order in which we propagate dependencies, which is helpful for consistency. The queue did not appear as a bottleneck in benchmarking. If it did, we could switch to a grow-only ring buffer (std::queue's underlying deque also shrinks automatically which is suboptimal).
* solver3: Defer version selection where possibleJulian Andres Klode2025-02-149-31/+37
| | | | | | | | | | | | | If a dependency can be satisfied by all versions of a package, add the package to the clause instead of the version object. This works only if there are no providers for the package: Providers are quite hard to enumerate over and make sure that all versions of a package satisfy the provider dependency. Implement arbitrary selection between packages and versions for the CompareProviders class: We pick the best version for each package and then pit them against each other.
* test: Fix skipping test casesArnaud Rebillout2025-02-121-2/+2
| | | | | | | | | | | The `--skip` option was introduced in commit 1fbdc04be7547b8c4bb84f400329415ea0bc60b5, and it was broken straight from the start for the case where `TESTTORUN` is not set (see last hunk in this commit, the mistake is obvious). It went unnoticed as the GitLab CI runs tests with the `-j` option, so `TESTTORUN` is set. While ci.debian.net runs without -j, but doesn't skip any test, so it doesn't trigger the bug either.
* solver3: Replace PropagateInstall() with Discover()Julian Andres Klode2025-02-071-0/+1
| | | | | | | | | | | | | | | Just propagate the stored clauses after we have discovered them; this is quite straightforward. We now more reliably discover common dependencies at the package level, adjust the test case accordingly. The next step is to make discovery recursive, or iterative, to build an entire recursive tree from all roots, and then we can reject reverse dependencies based on it. A bunch of refactorings are needed in the process. We remove the useless Hint enumeration and insert a flags struct into the State, such that we can record whether a package/version has been discovered, to avoid spending double time on discovery.
* solver3: Extract TranslateOrGroup() out of EnqueueOrGroup()Julian Andres Klode2025-02-071-1/+1
| | | | | | | This is not *purely* a refactoring, we accidentally used the version of the dependency when enqueuing conflicts rather than the reason, so the conflict string in the test case is different; the logging had the same issue.
* Also support debian-ports modernizationSamuel Thibault2025-02-021-0/+12
| | | | That just needs using '-' to replace spaces in Origin to get the keyring file name.
* solver3: Remove fixed test cases from broken listJulian Andres Klode2025-01-301-2/+0
|
* solver3: Use a propagation queueJulian Andres Klode2025-01-3012-33/+21
| | | | | | | | | | | | | | | | | | | Instead of directly propagating in a recursive fashion, queue propagations in a queue and work on them in a loop per the miniSAT paper. We call Propagate() only at the end of the FromDepCache() function and then in the Solve loop. Delaying the initial propagation means that we get a stronger reasoning: Assume you have x->a->b->c, y->c and you install x,y: - Previously we traversed: x, y, x->a, a->b, b->c, (y->c) - but now we traverse: x, y, x->a, y->c, a->b, (b->c) Notably c now has the implication y->c instead of x->a->b->c. Inside the solver we need to call Propagate in a loop: Propagating facts can fail and we then backtrack. If backtracking is succesful, we have gained a new fact to propagate.
* solver3: Remove NewUnsatRecommends groupJulian Andres Klode2025-01-301-0/+46
| | | | | | This is more or less unused; but it particularly has the bad problem of forcing new unsat recommends to be solved *before* dependencies. Which is awkward.
* solver3: Soft unit propagation for optional itemsJulian Andres Klode2025-01-301-1/+7
| | | | | If both items are optional, unit items should be processed first.
* Make notices about .list sources without signed-by less obnoxiousJulian Andres Klode2025-01-281-8/+12
|
* apt: Introduce the modernize-sources commandJulian Andres Klode2025-01-281-0/+85
| | | | | | | | | | | | | | | | | | | | | | This handles rewriting .list entries to .sources ones; the only options handled are - signed-by => Signed-By - trusted => Trusted The options to specify architectures, check-valid-until, etc. are not supported as they are not exposed in the code. We probably should refactor the parser at some point so we can actually read the parsed entry rather than the abstract objects we create from it. This can be improved further: - Keyrings from /etc/apt/trusted.gpg.d should be moved to /etc/apt/keyrings. - automatic detection by trying to verify using each keyring in trusted.gpg.d until we find the right one and then using that for Signed-By rather than leave some sources empty.
* apt-secure manual page is in manual section 8Andreas Rönnquist2025-01-281-2/+2
|
* sqv, gpgv: Do not fail if Dir::Etc::Trusted is set and trusted.gpg.d is missingJulian Andres Klode2025-01-221-0/+48
| | | | | | | | | | This should remain a warning case for now. So push the errors to a stack and see if we can find any file. Explictly warn again if Dir::Etc::trusted is set, as it is like _super_ deprecated. Reported-By: Cyril Brulebois <kibi@debian.org> in Bug#1032131
* Remove superseded warning about trusted.gpg fallbackJulian Andres Klode2025-01-211-65/+0
| | | | | | | | | | This warning has been superseded by the missing sources.list notice (which will also become a warning shortly) Adjust the sqv exit to avoid introducing a new spurious "No good signature" that we did not reach before. moo
* Warn about missing Signed-By in .list formatJulian Andres Klode2025-01-212-2/+13
| | | | | | Warn about the missing field there and suggest the transition to deb822 .sources files if we found any non-deb822 source without signed-by.
* Grep for more evil in test as a signature can contain evilDavid Kalnischkies2025-01-171-2/+2
| | | | | | | | The test was failing for me as the PGP signature contained "GeviljGB" which is clearly not the evil we are looking for. Sadly, reproducing this is hard, but random failures of the test suite are annoying, so lets grep for a bit longer strings that can't happen in the signature block to avoid this particular problem.
* Write Pre-Depends with a dash again in EDSPDavid Kalnischkies2025-01-171-5/+24
| | | | | | | | | | | | | | For some reason ~ that I don't know and can't come up with on the spot ~ we output the Pre-Depends field as PreDepends in the translated user-visible output and I accidentally copied that over to the untranslated variant that replaced the embedded copy (with a dash) in the EDSP generator. Code-copies and implementation details littered all over the place are apparently sometimes a good bug-free thing, too… Regression-of: 6828ae2c2f9268c8187f0fa91b3c464ed84a8476 Closes: #1093254
* t/i/test-method-gpgv-legacy-keyring: Switch to http, extendJulian Andres Klode2025-01-141-28/+41
| | | | | Add some more test cases where only one of the sources is bad; and switch to htp to make the order more reliable.
* Drop SourceListWarnings::NonFreeFirmware messagesDavid Kalnischkies2025-01-051-36/+0
| | | | | | | | | They were implemented to help a tiny bit in notifying users even if they didn't read the Release notes (or didn't realize it would apply to them) that they likely have to change their sources.list entries, but that transition is now over for a while, so everyone who should have seen it has hopefully reacted to it and so all this code now does is waste space, time and idly waiting for false positives.
* Resolve autopkgtest failures caused by dpkg >= 1.22.12David Kalnischkies2025-01-053-8/+12
| | | | | | | | | | | | | | | | | | | | Streamlining the rules around the Section field and especially what happens if its isn't present are a good idea in general, but for testing proposes we want a package without a section ~ something dpkg and co do not want us to have for good reasons (outside of tests). While a real nosection package would interact with more of our code, for this test we don't really need a real one and so we can adopt to the new reality of encountering such packages only if they are already installed instead of trying to build a sectionless package by hand. The other failure is a simple and trivial string change around the formatting of the version number in dpkg error message about file overrides, which we can resolve by just not looking at the version to sidestep dealing with the formatting difference. References: https://lists.debian.org/debian-dpkg/2024/12/msg00010.html Closes: #1092090
* test: Check "${METHODSDIR}/sqv" not "/usr/bin/sqv"Julian Andres Klode2025-01-013-4/+4
| | | | | This is the correct check to find out whether we have built with sqv support.
* sqv: Avoid propagating errors between filesJulian Andres Klode2025-01-011-4/+19
| | | | | | | | | | | We inadvertently did not discard pending errors when the fallback to Dir::Etc::trusted succeeded, do so. Also add some quick safety checks to make sure we enter the verification in a good state, and that if somehow errors are pending after a verification we return false. Closes: #1091679
* methods: Add new sqv methodJulian Andres Klode2024-12-226-44/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new sqv method uses sequoia's sqv tool to verify files. The tool's interface is quite simple: It returns 0 on success, and prints one line per good signer with the fingerprint. sqv has a configurable crypto policy. We have defined apt-specific override mechanisms for sequoia's standard policy, allowing both users, distributions, and apt package to provide overrides for Sequoia's default policy in meaningful ways. The sqv method will be built and be the standard method for verifying OpenPGP signatures provided that `sqv` is in the PATH during building. It is not built if there was no sqv at build time, so you need a rebuild to enable sqv later on. On the flip side, the gpgv method is always built, but it does need to be always built: If APT::Key::GPGVCommand is set, we need to fallback to it - this is important to support existing users of that interface such as mmdebstrap. Also we want to fall back to it when /usr/bin/sqv disappears - for example in our CI :D A couple of concessions have been made for test suite purposes: - Failure to split a clearsigned file only shows the summary, as the gpgv method also only showed it, and no details why it failed. - We write "Got GOODSIG <fingerprint>" in debug mode to mimic the gpgv code to keep the test suite happy. - In various places in the test suite we assert minimal output from sqv, but sqv's human output is not intended to be stable. This will incur additional work when it breaks. However we do not _parse_ the output, so actual operation of apt is unaffected. A couple of things are suboptimal here: - We are still doing clearsigned splitting ourselves. sqv only has support for detached signatures right now, whereas sqopv has support for clearsigned files as well (but sqopv does not provide any reasons for why signature verification fails or means to set a policy). - Deprecation of algorithms happens on a timebomb basis in sequoia. We have no means to give users warnings ahead of time if their configuration is outdated. We have implemented various bits that probably should be going away: - Fallback to trusted.gpg This is just annoying... - Support for fingerprints in Signed-By (no subkey matching though) The extra work here is arguably less compared to gpgv. - Check the keyring for correctness. With Signed-By everywhere, we should just error out rather than skip broken keyrings. Moo
* Use sq in the test suite, remove apt-keyJulian Andres Klode2024-12-198-116/+100
| | | | | | | | | | | | | Remove the test case for MD5 and expired signatures, as we can't create them (can't set signing digest, and can't set signature expiry). Tests for them have been added to test-method-gpgv instead. We override sq in a function with cert-store and key-store set to none. This supports both sq 0.40 and sq 1.0.
* Remove legacy apt-key testsJulian Andres Klode2024-12-192-469/+0
|
* apt show: Add pinning information for --fullJulian Andres Klode2024-12-191-0/+3
| | | | | Add new fields `APT-Pin`, `APT-Candidate`, and `APT-Release` to the output, same as in EDSP (limited to one release for now).
* Be more strict about which PAGER to execvp() directlyJulian Andres Klode2024-12-191-0/+11
| | | | | | | | | | | There could be | or ; or other stuff in PAGER that needs special handling, let's encode a list of known-good characters instead. This covers all known pagers. You could also try to execvp() PAGER in its entirety and if it fails with ENOENT fall back to the shell but then you don't get correct error handling if the pager really doesn't exist (in which case we should abort trying to page).
* Introduce automatic pager for read commandsJulian Andres Klode2024-12-182-2/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Automatically show the output of `show`, `policy`, `list`, `search`, `showsrc` in a pager. The pager setup is inspired by git's pager setup. Notably, the pager is found using APT_PAGER and PAGER variables. We wait for the pager to be setup somewhat correctly by using a notify pipe to figure out whether execvp() was succesful - then the pipe will read EOF as the other end got closed by CLOEXEC during exec - or not, then the pipe will contain an errno. We set up the correct handlers for signals and exit to close the fds and wait for the pager. Notably inside the signal handler we cannot flush our streams, only close them, so there is some duplication. We call the InitOutputPager() function from inside the various Do...() functions rather than setting it up generally in InitOutput(). Doing so allows us to first render the progress without a pager, and then setup the pager for the content only which improves user experience. When we setup a pager we also take care to disable standard input, as we should not be prompting users while a pager is running (the pager will be reading from the tty directly). We do this by dup2-ing() a /dev/null over it; if we just close()d the fd, another open() might reuse the fd number and problems could occur.
* Fix out-of-bounds read in `apt show :`Julian Andres Klode2024-12-161-0/+4
| | | | | | | | We tried to access str.size() - 1 on a 0-sized string, which does not work, and now that we build with assertions triggers it. Closes: #1090068
* methods/gpgv: Remove apt-key mentionsJulian Andres Klode2024-12-071-2/+2
|
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-07-18 23:48:19 +0000