apt/methods, branch 2.9.20

methods: Add new sqv method

2024-12-22T22:51:22Z

The new sqv method uses sequoia's sqv tool to verify files. The tool's interface is quite simple: It returns 0 on success, and prints one line per good signer with the fingerprint. sqv has a configurable crypto policy. We have defined apt-specific override mechanisms for sequoia's standard policy, allowing both users, distributions, and apt package to provide overrides for Sequoia's default policy in meaningful ways. The sqv method will be built and be the standard method for verifying OpenPGP signatures provided that `sqv` is in the PATH during building. It is not built if there was no sqv at build time, so you need a rebuild to enable sqv later on. On the flip side, the gpgv method is always built, but it does need to be always built: If APT::Key::GPGVCommand is set, we need to fallback to it - this is important to support existing users of that interface such as mmdebstrap. Also we want to fall back to it when /usr/bin/sqv disappears - for example in our CI :D A couple of concessions have been made for test suite purposes: - Failure to split a clearsigned file only shows the summary, as the gpgv method also only showed it, and no details why it failed. - We write "Got GOODSIG " in debug mode to mimic the gpgv code to keep the test suite happy. - In various places in the test suite we assert minimal output from sqv, but sqv's human output is not intended to be stable. This will incur additional work when it breaks. However we do not _parse_ the output, so actual operation of apt is unaffected. A couple of things are suboptimal here: - We are still doing clearsigned splitting ourselves. sqv only has support for detached signatures right now, whereas sqopv has support for clearsigned files as well (but sqopv does not provide any reasons for why signature verification fails or means to set a policy). - Deprecation of algorithms happens on a timebomb basis in sequoia. We have no means to give users warnings ahead of time if their configuration is outdated. We have implemented various bits that probably should be going away: - Fallback to trusted.gpg This is just annoying... - Support for fingerprints in Signed-By (no subkey matching though) The extra work here is arguably less compared to gpgv. - Check the keyring for correctness. With Signed-By everywhere, we should just error out rather than skip broken keyrings. Moo

Remove GnuTLS and gcrypt support

2024-12-22T22:45:15Z

OpenSSL is mandatory now, it is no longer possible to build without https support either.

hashes, methods: Add OpenSSL backends

2024-12-22T21:55:39Z

Introduce an OpenSSL::Crypto backend for the hashes library and an OpenSSL::SSL backend for the TLS support in our https method. Many thanks to curl for showing the way with how to handle a CRL file. There are some memory leaks here with the TlsFd itself as well as the proxy support; and we should reorganize the code to generate the ssl object as late as possible. A peculiar aspect of OpenSSL is that SSL_has_pending() returns 1 even if SSL_read() will fail to read anything and return the equivalent of EAGAIN. We work around this here by also peeking ahead 1 byte. I was running a very high RTT connection from Germany to Australia for testing, and with the peeking it's using negligible amounts of CPU; before that, it was busy looping at 100%. Bad OpenSSL!

methods/gpgv: Remove apt-key mentions

2024-12-07T14:14:34Z

Detect working gpgv using gpgv --dump-options

2024-11-28T18:01:36Z

This gains us support for not using an absolute path, which is nice, and we can now cache the invocation across fork() in the method. Still keep preferring absolute paths, this avoids execvp() having to iterate the path and reduces bug potential with custom gpgv in /usr/local. Fix the last entry in the list of gpgvs to say gpgv1

Do not implode key file name vector for calling gpgv

2024-11-28T09:17:21Z

Converting it to a comma-separated string and back loses precision.

Merge branch 'fix/partialfilemirror' into 'main'

2024-11-22T16:12:34Z

Support uncompressed indexes from partial file:/ mirrors See merge request apt-team/apt!235

Look at non by-hash paths in copy and file methods

2024-11-22T12:40:10Z

Ideally copy and file mirrors would support by-hash as well, but its harder to setup and maintain especially if you want to cache an online mirror who has by-hash enabled. We can avoid unneeded roundtrips (in the best case) and entire cache misses (in the usual worst case) by "just" telling methods that the URI we passed it has the requested file perhaps also in other paths. This is done in pseudo-relative paths as we would otherwise need to teach redirection code to rewrite those URIs as well. A method like http can easily ignore this value and await explicit instructions to look at that file, but inspecting the path in local sources via file or copy is (comparatively) free, so we just do it immediately. If that ends up being the wrong version of the file as by-hash would have protected us from we are in this feature branch now falling back to other mirrors which are like the ones online and in support of by-hash.

Do not assume mirror-URIs end in a filename causing a hang

2024-11-22T12:40:10Z

In practice most of them will, but making the acquire process hang in response if they don't seems like an overly excessive response if we can just support it gracefully to reply with an error (or actually succeed in the unlikely event this URI is actually correct).

Alternatively calculate alternative file hashes in file method

2024-11-22T12:21:24Z

If we do not have the requested file we haven't calculated the hashes for it either and it is likely that the alternative file will be used, so to save the main thread from being busy with calculating hashes we do the calculation here in the method.