summaryrefslogtreecommitdiff
path: root/test/integration/run-tests
Commit message (Collapse)AuthorAgeFilesLines
* test: Correctly skip tests when not using -jJulian Andres Klode2025-10-261-1/+1
| | | | | | We never implemented comment support in the skip test file for -j as we only run with -j, but the debian/tests/run-tests runs without parallelism.
* tests: Do not skip tests for --solver internalJulian Andres Klode2025-10-261-3/+3
| | | | | | | Remove the exports for the default as we don't want those inherited, they can just be reset. Set the skip file to /dev/null when a solver is selected, to make sure that tests don't fall back to the default skip file.
* Enable the new solver by default in 0.31, 1.21, 2.11, 3.1Julian Andres Klode2025-10-251-0/+3
| | | | | Enable the new solver by default and remove the vendor-specific configuration change in Ubuntu's drop-in.
* test: Support comments in --skip/--only filesJulian Andres Klode2025-02-141-2/+2
| | | | Strip everything after "#" and spaces before it.
* test: Fix skipping test casesArnaud Rebillout2025-02-121-2/+2
| | | | | | | | | | | The `--skip` option was introduced in commit 1fbdc04be7547b8c4bb84f400329415ea0bc60b5, and it was broken straight from the start for the case where `TESTTORUN` is not set (see last hunk in this commit, the mistake is obvious). It went unnoticed as the GitLab CI runs tests with the `-j` option, so `TESTTORUN` is set. While ci.debian.net runs without -j, but doesn't skip any test, so it doesn't trigger the bug either.
* Use sq in the test suite, remove apt-keyJulian Andres Klode2024-12-191-10/+0
| | | | | | | | | | | | | Remove the test case for MD5 and expired signatures, as we can't create them (can't set signing digest, and can't set signature expiry). Tests for them have been added to test-method-gpgv instead. We override sq in a function with cert-store and key-store set to none. This supports both sq 0.40 and sq 1.0.
* Make the failure mode for non-executable tests strongerJulian Andres Klode2024-06-131-1/+1
| | | | | | | | | When a test-* file is created but not executable, we'd ignore the argument and then our subprocess would run the entire test suite itself, sequentially. Silly. Let's just see if the file exists instead, then the test will fail.
* test: Show statistics for parallel tests tooJulian Andres Klode2024-05-241-1/+26
| | | | | | | | This makes things a bit nicer. We also include a list of passed tests here, to use with --only to validate solver3 changes, as well as the list of skipped tests. But too lazy to implement these for the non-parallel case.
* test: Run working tests against the 3.0 solverJulian Andres Klode2024-05-241-3/+28
| | | | | Implement --solver, --only, -skip flags for the test suite, and run the test suite with the broken tests skipped against solver3.
* Ignore stty failures in testcasesDavid Kalnischkies2022-05-071-2/+2
| | | | | | | | | | We use 'stty sane' to combat against stepped output and co caused by (especially) failed tests, but it does so many things that it occasionally fails to reset some bits in the parallel interaction we have with it which fails the tests without a real problem in apt… Ideally we would be better at stitching the output together, but for the time being lets ignore these failures instead to stabilize the tests.
* Use moreutils parallel even if GNU parallel is installedDavid Kalnischkies2022-02-021-0/+2
| | | | | | | | GNU parallel diverts moreutils implementation away. As we us moreutils features just installing parallel breaks the test runner hence. We have this already for another naming scheme, so fixing this is easy enough. Gbp-Dch: Ignore
* test suite: Do not exit 0 in trap for QUITJulian Andres Klode2017-01-241-1/+2
| | | | | | | | | This hides errors in the test suite because it will exit with 0 here. Instead, just do exit 1 in most traps, and do just the cleanup in the QUIT hook. This fixes a regression introduced with the caching of the GPG home directory in 4ce2f35248123ff2366c8c365ad6a94945578d66.
* tests: cache the apt-key homedir used for Release signingDavid Kalnischkies2016-12-211-0/+10
| | | | | | | | | Importing a new secret key into gpg(2) can be increadibly slow which prolongs the test runs significantly – by caching the homedir we gain a significant speedbonus as reimporting already present keys seems like a far less costly operation. Git-Dch: Ignore
* test, travis: Quieter testing with a new -qq modeJulian Andres Klode2016-08-291-5/+29
| | | | | | | | | | | | | | | Introduce a new -qq mode for our integration test framework, and make travis use it. The new -qq mode sets MSGLEVEL to 1. In MSGLEVEL=1, no messages are generated for passed tests, and all testcase filenames are printed in the same line. Also install first in travis, do not ls the installed output and run the install with chronic, so we only get output if it failed. Gbp-Dch: ignore
* test: Allow moreutils-parallel instead of parallelJulian Andres Klode2016-08-261-1/+5
| | | | | | That's what it's called on FreeBSD. Gbp-Dch: ignore
* Do not set the binary dir in run-tests, it breaks stuffJulian Andres Klode2016-08-101-6/+0
| | | | | | | This breaks -j and does all sort of other weird stuff I did not notice in the previous (non-parallel) runs. Gbp-Dch: ignore
* test: Automatically discover CMake build directoryJulian Andres Klode2016-08-101-0/+6
| | | | | | | Look at the project root, and all directories directly below it and pick the directory with the newest CMakeCache.txt file. Gbp-Dch: ignore
* tests: fix parallel execution to be working dir independentDavid Kalnischkies2016-03-161-1/+1
| | | | Git-Dch: Ignore
* tests: support spaces in path and TMPDIRDavid Kalnischkies2015-12-191-1/+2
| | | | | | | This doesn't allow all tests to run cleanly, but it at least allows to write tests which could run successfully in such environments. Git-Dch: Ignore
* replace run-parts with find|sort to avoid debianutils usageDavid Kalnischkies2015-12-061-1/+1
| | | | | | | After e75e5879 the reason for an implicit dependency on debianutils (which is essential for debian, but likely not on other systems) was just two uses of run-parts, which can be replaced with the a lot more portable find-piped-into-sort duo.
* tests: ensure sponge-files are cleaned up properlyDavid Kalnischkies2015-11-041-0/+2
| | | | Git-Dch: Ignore
* tests: add a -j $jobs mode to test runner for parallel executionDavid Kalnischkies2015-09-151-14/+66
| | | | | | | | | | Now that tests can be run in parallel, lets actually do it… The mode has some downsides like not collecting the failed tests, but it can be a lot faster than a sequential run and is therefore a good alternative in testing those "this shouldn't break anything" changes (which tend to break everything if untested). Git-Dch: Ignore
* tests: enhance output of grep and test failsDavid Kalnischkies2014-10-261-1/+5
| | | | Git-Dch: Ignore
* detect terminal output with 'test -t' in testsDavid Kalnischkies2014-09-071-1/+1
| | | | | | | | Instead of trying to inspect /proc and the fds inside we use "test -t 1" instead as this is available and working on kfreebsd as well – not that something breaks if we wouldn't, but we like color. Git-Dch: Ignore
* use printf instead of echo in testing frameworkMichele Orrù2014-07-071-2/+2
| | | | | | | | | The behaviour of echo "\tA\t" differs between dash/zsh which interprets the \t as tab and bash which prints it literally. Similar things happen for other escape sequences – without the -e flag. Switching to printf makes this more painless^Wportable, so that the tests are also working correctly with bash as sh. (commit message by committer, patch otherwise unmodified)
* show current/total testcase statistic also in concise modeDavid Kalnischkies2014-05-301-1/+1
| | | | Git-Dch: Ignore
* show progress in run-testsMichael Vogt2014-05-071-1/+2
|
* get color/msglevel handling for tests in lineDavid Kalnischkies2014-01-171-12/+23
| | | | | | | | | Without a PTY attached do not use color, but use the same MSGLEVEL with or without a PTY. The level is better adjust via flags – especially as it is likely that without a PTY you want fullblown logs instead of the reduced display you get with -q otherwise. Git-Dch: Ignore
* improve stdout/stderr usage correctness in test frameworkDavid Kalnischkies2014-01-171-7/+15
| | | | | | | | Also adds a friendly note about how many tests were run/passed so that the end of the testrun isn't all that negative by just showing fails. (It now tells us that we have 111 tests at the moment!) Git-Dch: Ignore
* test/integration/run-tests: output the failed test namesMichael Vogt2013-10-221-0/+5
|
* report failures via exit and ensure we don't overflowMichael Vogt2013-03-181-1/+2
|
* set the msglevel to a high dummy value if none is specifiedDavid Kalnischkies2011-08-171-0/+4
|
* do not output color in the testcases if stdout/stderr are not a /dev/ptsDavid Kalnischkies2011-08-171-4/+13
|
* test/integration/: remove a bunch of "local" statements from the ↵Michael Vogt2011-02-151-1/+8
| | | | shellscript, keep tests running even on failure but log failures
* enable the split out of LongDescriptions per default in the testcasesDavid Kalnischkies2011-01-251-1/+1
|
* implement i quiet run-tests mode which prints only one line per testcaseDavid Kalnischkies2011-01-161-1/+13
|
* tests/integration/test-*: remove a bunch of "local" that are used outside ↵Michael Vogt2010-10-131-1/+1
| | | | funtions (bash complains)
* - prefer non-virtual packages in FindPreferredPkg (Closes: #590041)David Kalnischkies2010-07-251-0/+8
* test/integration/*: - add with bug#590041 testcase a small test "framework"