Add support for Phased-Update-Percentage

This adds support for Phased-Update-Percentage by pinning
upgrades that are not to be installed down to 1.

The output of policy has been changed to add the level of
phasing, and documentation has been improved to document
how phased updates work.

The patch detects if it is running in a chroot, and if so, always
includes phased updates, restoring classic apt behavior to avoid
behavioral changes on buildd chroots.

Various options are added to control this all:

* APT::Get::{Always,Never}-Include-Phased-Updates and their legacy
  update-manager equivalents to always or never include phased updates
* APT::Machine-ID can be set to a UUID string to have all machines in a
  fleet phase the same
* Dir::Etc::Machine-ID is weird in that it's default is sort of like
  ../machine-id, but not really, as ../machine-id would look up
  $PWD/../machine-id and not relative to Dir::Etc; but it allows you to
  override the path to machine-id (as opposed to the value)
* Dir::Bin::ischroot is the path to the ischroot(1) binary which is used
  to detect whether we are running in a chroot.

2 files changed, 275 insertions, 0 deletions

diff --git a/test/integration/framework b/test/integration/framework
index 3973ad863..696fcd8cd 100644
--- a/test/integration/framework
+++ b/test/integration/framework
@@ -484,6 +484,8 @@ EOF
 	# Allow release files to be 10 hours in the future, rather than 10 seconds
 	echo 'Acquire::Max-FutureTime "'$((10 * 60 * 60))'";' > rootdir/etc/apt/apt.conf.d/future-time
 
+	echo 'APT::Machine-ID "912e43bd1c1d4ba481f9f8ccab25f9ee";' > rootdir/etc/apt/apt.conf.d/machine-id
+
 	configcompression '.' 'gz' #'bz2' 'lzma' 'xz'
 	confighashes 'SHA256' # these are tests, not security best-practices
 
@@ -505,6 +507,7 @@ EOF
 
 	# Make dpkg inherit testing path
 	echo 'DPkg::Path "";' >> aptconfig.conf
+	echo 'Dir::Bin::ischroot "/bin/false";' >> aptconfig.conf
 
 	# Make gcov shut up
 	export GCOV_ERROR_FILE=/dev/null
diff --git a/test/integration/test-phased-updates b/test/integration/test-phased-updates
new file mode 100755
index 000000000..50f151f79
--- /dev/null
+++ b/test/integration/test-phased-updates
@@ -0,0 +1,272 @@
+#!/bin/sh
+set -e
+
+TESTDIR="$(readlink -f "$(dirname "$0")")"
+. "$TESTDIR/framework"
+
+setupenvironment
+echo 'Debug::Phasing "1";' > rootdir/etc/apt/apt.conf.d/debug-phasing
+configarchitecture 'i386' 'armel'
+
+
+insertinstalledpackage 'phased1' 'all' '1'
+insertinstalledpackage 'phased2' 'all' '1'
+
+insertpackage 'unstable' 'phased1' 'all' '10' 'Phased-Update-Percentage: 10'
+insertpackage 'unstable' 'phased2' 'all' '10' 'Phased-Update-Percentage: 10'
+insertpackage 'unstable' 'phased3' 'all' '10' 'Phased-Update-Percentage: 10'
+
+insertpackage 'unstable' 'phased1' 'all' '100' 'Phased-Update-Percentage: 100'
+insertpackage 'unstable' 'phased2' 'all' '100' 'Phased-Update-Percentage: 100'
+insertpackage 'unstable' 'phased3' 'all' '100' 'Phased-Update-Percentage: 100'
+
+insertpackage 'unstable' 'phased1' 'all' '50' 'Phased-Update-Percentage: 50'
+insertpackage 'unstable' 'phased2' 'all' '50' 'Phased-Update-Percentage: 50'
+insertpackage 'unstable' 'phased3' 'all' '50' 'Phased-Update-Percentage: 50'
+
+setupaptarchive
+
+msgmsg "Basic test"
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 1 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased3:
+  Installed: (none)
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages" aptcache policy phased1 phased2 phased3
+
+msgmsg "Test for always-include-phased-updates"
+for always in Update-Manager::Always-Include-Phased-Updates APT::Get::Always-Include-Phased-Updates; do
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased3:
+  Installed: (none)
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages" aptcache policy phased1 phased2 phased3 -o $always=true
+done
+
+msgmsg "Test for never-include-phased-updates"
+for never in Update-Manager::Never-Include-Phased-Updates APT::Get::Never-Include-Phased-Updates; do
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 1 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 1 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased3:
+  Installed: (none)
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages" aptcache policy phased1 phased2 phased3 -o $never=true
+done
+
+msgmsg "Test that being in a chroot equals always-include-phased-updates"
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status" aptcache policy phased1 phased2 -o Dir::Bin::ischroot=/bin/true
+
+msgmsg "Test that empty machine-id equals always-include-phased-updates"
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status" aptcache policy phased1 phased2 -o Dir::Etc::machine-id=/dev/null -o APT::Machine-Id=""
+
+msgmsg "Test that never-include-phased-updates trumps empty machine-id"
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 1 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 1 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status" aptcache policy phased1 phased2 -o Dir::Etc::machine-id=/dev/null -o APT::Machine-Id="" -o APT::Get::Never-Include-Phased-Updates=1
+
+
+msgmsg "Test that SOURCE_DATE_EPOCH set equals always-include-phased-updates"
+export SOURCE_DATE_EPOCH=0
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 500 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status" aptcache policy phased1 phased2
+unset SOURCE_DATE_EPOCH
+
+msgmsg "Test that different machine-id produces different outcome"
+testsuccessequal "phased1:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status
+phased2:
+  Installed: 1
+  Candidate: 100
+  Version table:
+     100 500
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     50 500 (phased 50%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+     10 1 (phased 10%)
+        500 file:${TMPWORKINGDIRECTORY}/aptarchive unstable/main all Packages
+ *** 1 100
+        100 ${TMPWORKINGDIRECTORY}/rootdir/var/lib/dpkg/status" aptcache policy phased1 phased2 -o apt::machine-id="00000000000000000000000000000000"