Initial support for snapshot servers, apt --snapshot option

Provide snapshot support for offical Debian and Ubuntu archives.

There are two ways to enable snapshots for sources:

1. Add Snapshot: yes to your sources file ([snapshot=yes]). This
   will allow you to specify a snapshot to use when updating or
   installing using the --snapshot,-S option.

2. Add Snapshot: ID to your sources files to request a specific
   snapshot for this source.

Snapshots are discovered using Label and Origin fields in the Release
file of the main source, hence you need to have updated the source at
least once before you can use snapshots.

The Release file may also declare a snapshots server to use, similar
to Changelogs, it can contain a Snapshots field with the values:

1. `Snapshots: https://example.com/@SNAPSHOTID@` where `@SNAPSHOTID@`
   is a placeholder that is replaced with the requested snapshot id

2. `Snapshots: no` to disable snapshot support for this source.
   Requesting snapshots for this source will result in a failure
   to load the source.

The implementation adds a SHADOWED option to deb source entries,
and marks the main entry as SHADOWED when a snapshot has been
requested, which will cause it to be updated, but not included
in the generated cache.

The concern here was that we need to keep generating the shadowed
entries because the cleanup in `apt update` deletes any files not
queued for download, so we gotta keep downloading the main source.

This design is not entirely optimal, but avoids the pitfalls of
having to reimplement list cleanup.

Gaps:

- Ubuntu Pro repositories and PPAs are not yet supported.

4 files changed, 62 insertions, 0 deletions

diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml
index f4d347c70..beecfd27c 100644
--- a/doc/apt-get.8.xml
+++ b/doc/apt-get.8.xml
@@ -528,6 +528,16 @@
      Configuration Item: <literal>APT::Get::List-Cleanup</literal>.</para></listitem>
      </varlistentry>
 
+     <varlistentry><term><option>-S</option></term>
+                   <term><option>--snapshot</option></term>
+     <listitem><para>This option controls the snapshot chosen for archives with <literal>Snapshot: enable</literal>
+             in the source entry. For example, <option>-S 20220102T030405Z</option> selects a snapshot from January 2nd,
+             2022 at 03:04:05 UTC.
+     Configuration Item: <literal>APT::Snapshot</literal>;
+     see also the &sources-list; manual page.
+     </para></listitem>
+     </varlistentry>
+
      <varlistentry><term><option>-t</option></term>
                    <term><option>--target-release</option></term>
                    <term><option>--default-release</option></term>
diff --git a/doc/apt.conf.5.xml b/doc/apt.conf.5.xml
index 189fb7d7d..e815a58a1 100644
--- a/doc/apt.conf.5.xml
+++ b/doc/apt.conf.5.xml
@@ -211,6 +211,10 @@ APT::Compressor::rev {
      version is available. Contains release name, codename or release version. Examples: 'stable', 'testing',
      'unstable', '&debian-stable-codename;', '&debian-testing-codename;', '4.0', '5.0*'. See also &apt-preferences;.</para></listitem>
      </varlistentry>
+     <varlistentry><term><option>Snapshot</option></term>
+         <listitem><para>Snapshot to use for all repositories configured with <literal>Snapshot: yes</literal>. See also &sources-list;, the <option>--snapshot</option> option that sets this value, and <option>Acquire::Snapshots::URI</option> below.</para></listitem>
+     </varlistentry>
+
 
      <varlistentry><term><option>Ignore-Hold</option></term>
      <listitem><para>Ignore held packages; this global option causes the problem resolver to
@@ -647,6 +651,26 @@ APT::Compressor::rev {
 	  this source can't be used to acquire changelog files from. Another source will be tried
 	  if available in this case.
 	</para></listitem>
+</varlistentry>
+
+     <varlistentry><term><option>Snapshots::URI</option> scope</term>
+	<listitem><para>
+	  Like changelogs, snapshots can only be acquired if an URI is known from where to get them.
+	  Preferable the Release file indicates this in a 'Snapshots' field. If this isn't
+	  available the Label/Origin field of the Release file is used to check if a
+	  <literal>Acquire::Snapshots::URI::Label::<replaceable>LABEL</replaceable></literal> or
+	  <literal>Acquire::Snapshots::URI::Origin::<replaceable>ORIGIN</replaceable></literal> option
+	  exists and if so this value is taken. The value in the Release file can be overridden
+	  with <literal>Acquire::Snapshots::URI::Override::Label::<replaceable>LABEL</replaceable></literal>
+	  or <literal>Acquire::Snapshots::URI::Override::Origin::<replaceable>ORIGIN</replaceable></literal>.
+
+      The value should be a normal URI to a directory, except that the snapshot ID replaced with the
+      placeholder <literal>@SNAPSHOTID</literal>.
+
+	  The special value '<literal>no</literal>' is available for this option indicating that
+	  this source cannot be used to acquire snapshots from. Another source will be tried
+	  if available in this case.
+	</para></listitem>
     </varlistentry>
 
    </variablelist>
diff --git a/doc/examples/configure-index b/doc/examples/configure-index
index 5ec209766..e6d7c31ea 100644
--- a/doc/examples/configure-index
+++ b/doc/examples/configure-index
@@ -59,6 +59,9 @@ APT
   VersionedKernelPackages "<LIST>"; // regular expressions to be protected from autoremoval (kernel uname will be appended)
   Protect-Kernels "<BOOL>"; // whether to protect installed kernels against autoremoval (default: true)
 
+  // Currently active snapshot
+  Snapshot "<STRING>";
+
   // Options for apt-get
   Get
   {
@@ -398,6 +401,14 @@ Acquire
     Override::Origin::* "<STRING>";
     Override::Label::* "<STRING>";
   };
+  Snapshots::URI
+  {
+    // Origin::Debian "https://snapshot.debian.org/snapshot/@SNAPSHOTID@/";
+    Origin::* "<STRING>";
+    Label::* "<STRING>";
+    Override::Origin::* "<STRING>";
+    Override::Label::* "<STRING>";
+  };
   Changelogs::AlwaysOnline "<BOOL>"; // even if the changelog file exists get it online (as the file is incomplete)
   Changelogs::AlwaysOnline::Origin::* "<BOOL>";
 };
diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml
index 478694c77..4fa7a245b 100644
--- a/doc/sources.list.5.xml
+++ b/doc/sources.list.5.xml
@@ -392,6 +392,23 @@ Signed-By:
 		or, if that fails, a <filename>Release</filename> file and its associated <filename>Release.gpg</filename> file. By setting this option,
 		the specified path will be tried instead of the InRelease file,
 		and the fallback to <filename>Release</filename> files will be disabled.
+    </para></listitem>
+
+    <listitem><para><option>Snapshot</option> (<option>snapshot</option>)
+        allows selecting an earlier version of the archive from the snapshot service. Supported
+    values are: </para>
+        <itemizedlist>
+        <listitem>
+            <para><literal>enable</literal> to allow selecting a snapshot with the <option>--snapshot</option> option, or</para>
+        </listitem>
+        <listitem>
+            <para>a snapshot ID to select a specific snapshot.</para>
+        </listitem>
+    </itemizedlist>
+        <para>
+            Snapshot IDs are usually timestamps in the form of <literal>YYYYMMDDTHHMMSSZ</literal>, such as
+            <literal>20220102T030405Z</literal> which is the January 2nd, 2022 at 03:04:05 UTC, servers may
+            however support additional types of IDs, and APT does not perform any checks so far.
 	  </para></listitem>
 
        </itemizedlist>