From 798d79f10f828312db06fa7d25d8fa106fbe3c23 Mon Sep 17 00:00:00 2001 From: Stefano Zacchiroli Date: Wed, 30 Mar 2011 16:30:55 +0200 Subject: solver protocol (EDSP): first draft version 0.1 --- doc/apt-solver-protocol.mdwn | 235 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 doc/apt-solver-protocol.mdwn diff --git a/doc/apt-solver-protocol.mdwn b/doc/apt-solver-protocol.mdwn new file mode 100644 index 000000000..b1435102e --- /dev/null +++ b/doc/apt-solver-protocol.mdwn @@ -0,0 +1,235 @@ +** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** + +# APT External Dependency Solver Protocol (EDSP) - version 0.1 + +This document describes the communication protocol between APT and +external dependency solvers. The protocol is called APT EDSP, for "APT +External Dependency Solver Protocol". + + +## Components + +- **APT**: we know this one. +- APT is equipped with its own **internal solver** for dependencies, + which is identified by the string `internal`. +- **External solver**: an *external* software component able to resolve + dependencies on behalf of APT. Each external solver is identified by + an unique string (other than `internal`) called the solver **name**. + +At each interaction with APT, a single solver is in use. When there is +a total of 2 or more solvers, internals or externals, the user can +choose which one to use. + + +## Installation + +Each external solver is installed as a file under +`/usr/lib/apt/solvers`. The naming scheme is +`/usr/lib/apt/solvers/NAME`, where `NAME` is the name of the external +solver. + +Each file under `/usr/lib/apt/solvers` corresponding to an external +solver must be executable. + +No non-solver files must be installed under `/usr/lib/apt/solvers`, so +that an index of available external solvers can be obtained by simply +looking at the content of that directory. + + +## Configuration + +Several APT options can be used to affect dependency solving in APT. An +overview of them is given below. Please refer to proper APT +configuration documentation for more, and more up to date, information. + +- **APT::Solver::Name**: the name of the solver to be used for + dependency solving. Defaults to `internal` + +- **APT::Solver::Strict-Pinning**: whether pinning must be strictly + respected (as the internal solver does) or can be slightly deviated + from. Defaults to `yes`. + +- **APT::Solver::Preferences**: solver-specific user preferences used + during dependency solving. Check your solver documentation for what is + supported here. Default to empty. + + +## Protocol + +When configured to use an external solver, APT will resort to it to +decide which packages should be installed or removed. + +The interaction happens **in batch**: APT will invoke the external +solver passing the current status of installed and available packages, +as well as the user request to alter the set of installed packages. The +external solver will compute a new complete set of installed packages +and gives APT a "diff" listing of which *additional* packages should be +installed and of which currently installed packages should be +*removed*. (Note: the order in which those actions have to be performed +will be up to APT to decide.) + +External solvers are invoked by executing them. Communications happens +via the file descriptors: **stdin** (standard input) and **stdout** +(standard output). stderr is not used by the EDSP protocol. Solvers can +therefore use stderr to dump debugging information that could be +inspected separately. + +After invocation, the protocol passes through 3 separate phases: + +1. APT send to the solver a dependency solving **scenario** +2. The solver solves dependencies. No communication with APT happens + during this phase. +3. The solver sends back to APT an **answer**, i.e. either a *solution* + or an *error* report. + + +### Scenario + +A scenario is a text file encoded in a format very similar to the "Deb +822" format (AKA "the format used by Debian `Packages` files"). A +scenario consists of two distinct parts: a **request** and a **package +universe**, occurring in that order. The request consists of a single +Deb 822 stanza, while the package universe consists of several such +stanzas. All stanzas occurring in a scenario are separated by an empty +line. + + +#### Request + +Within a dependency solving scenario, a request represents the action on +installed packages requested by the user. + +A request is a single Deb 822 stanza opened by a mandatory Request field +and followed by a mixture of action and preference fields. + +The value of the **Request:** field is a string describing the EDSP +protocol which will be used to communicate. At present, the string must +be `EDSP 0.1`. + +a unique request identifier, such as an +UUID. Request fields are mainly used to identify the beginning of a +request stanza; their actual values are otherwise not used by the EDSP +protocol. + +The following **action fields** are supported in request stanzas: + +- **Install:** (optional, defaults to the empty string) A space + separated list of package names, with *no version attached*, to + install. This field denotes a list of packages that the user wants to + install, usually via an APT `install` request. + +- **Remove:** (optional, defaults to the empty string) Same syntax of + Install. This field denotes a list of packages that the user wants to + remove, usually via APT `remove` or `purge` requests. + +- **Upgrade:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, an upgrade of all installed packages has been + requested, usually via an APT `upgrade` request. + +- **Dist-Upgrade:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. Same as Upgrade, but for APT `dist-upgrade` requests. + +- **Autoremove:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, a clean up of unused automatically installed + packages has been requested, usually via an APT `autoremove` request. + +The following **preference fields** are supported in request stanzas: + +- **Strict-Pinning:** (optional, defaults to `yes`). Allowed values: + `yes`, `no`. When set to `yes`, APT pinning is strict, in the sense + that the solver must not propose to install packages which are not APT + candidates (see the `APT-Pin` and `APT-Candidate` fields in the + package universe). When set to `no`, the solver does only a best + effort attempt to install APT candidates. Usually, the value of this + field comes from the `APT::Solver::Strict-Pinning` configuration + option. + +- **Preferences:** a solver-specific optimization string, usually coming + from the `APT::Solver::Preferences` configuration option. + + +#### Package universe + +A package universe is a list of Deb 822 stanzas, one per package, called +**package stanzas**. Each package stanzas starts with a Package +field. The following fields are supported in package stanzas: + +- All fields supported by Debian Packages file (see one of the + `/var/lib/apt/lists/*Packages` file for an example), *with the + exception of the Description field* that is not allowed. + + Among those fields, the following are mandatory: Package, Version, + Architecture. + +- **Installed:** (optional, default value `no`). Allowed values: `yes`, + `no`. When set to `yes`, the corresponding package is currently + installed. + + ##TODO## changed with respect to current prototype, which uses Status + +- **APT-ID:** (mandatory). Unique package identifier, according to APT. + +- **APT-Pin:** (mandatory). Must be a non-negative integer. Package pin + value, according to current APT policy. + +- **APT-Candidate:** (optional, default value `no`). Allowed values: + `yes`, `no`. When set to `yes`, the corresponding package is granted + to have the highest pinning value among all the packages having the + same name. + + ##TODO## what about multi-arch? is the pin value granted to be the + higest also across different architectures? + + +### Answer + +An answer from the external solver to APT is either a *solution* or an +*error*. + +The following invariant on **exit codes** must hold true. When the +external solver is *able to find a solution*, it will write the solution +to standard output and then exit with an exit code of 0. When the +external solver is *unable to find a solution* (and aware of that), it +will write an error to standard output and then exit with an exit code +of 0. An exit code other than 0 will be interpreted as a solver crash +with no meaningful error about dependency resolution to convey to the +user. + + +#### Solution + +A solution is a single Deb 822 stanza, starting with the field +Solution. The following fields are supported in solution stanzas: + +- **Solution:** (mandatory). The value of this field is ignored, + although it should be a unique solution identifier, such as a UUID. + +- **Install:** (optional, defaults to the empty string). A space + separated list of strings of the form `PACKAGE=VERSION` where + `PACKAGE` is a package name and `VERSION` is an available version of + that package. The list denotes a set of packages that must be + installed to satisfy user request. + +- **Remove:** (optional, defaults to the empty string). Same as Install, + but denoting a set of packages that must be removed to satisfy user + request. + + +#### Error + +An error is a single Deb 822 stanza, starting the field Error. The +following fields are supported in error stanzas: + +- **Error:** (mandatory). The value of this field is ignored, although + it should be a unique error identifier, such as a UUID. + +- **Message:** (mandatory). The value of this field is a text string, + meant to be read by humans, that explains the cause of the solver + error. + + ##TODO## can we support line continuations throughout this format? If + yes, they might come handy both for error stanzas and for solution + stanzas (which might have very long install/remove lines) + + +** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** -- cgit v1.2.3-70-g09d2 From d911f277f40fec55b80b3c9cb6e805d39c480b59 Mon Sep 17 00:00:00 2001 From: Stefano Zacchiroli Date: Wed, 30 Mar 2011 22:28:23 +0200 Subject: second draft, addressing David and Michael comments EDSP version 0.2 --- doc/apt-solver-protocol.mdwn | 153 +++++++++++++++++++++++++++++-------------- 1 file changed, 104 insertions(+), 49 deletions(-) diff --git a/doc/apt-solver-protocol.mdwn b/doc/apt-solver-protocol.mdwn index b1435102e..85cd98d62 100644 --- a/doc/apt-solver-protocol.mdwn +++ b/doc/apt-solver-protocol.mdwn @@ -1,6 +1,6 @@ ** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** -# APT External Dependency Solver Protocol (EDSP) - version 0.1 +# APT External Dependency Solver Protocol (EDSP) - version 0.2 This document describes the communication protocol between APT and external dependency solvers. The protocol is called APT EDSP, for "APT @@ -13,13 +13,18 @@ External Dependency Solver Protocol". - APT is equipped with its own **internal solver** for dependencies, which is identified by the string `internal`. - **External solver**: an *external* software component able to resolve - dependencies on behalf of APT. Each external solver is identified by - an unique string (other than `internal`) called the solver **name**. - + dependencies on behalf of APT. + At each interaction with APT, a single solver is in use. When there is a total of 2 or more solvers, internals or externals, the user can choose which one to use. +Each solver is identified by an unique string, the **solver +name**. Solver names must be formed using only alphanumeric ASCII +characters, dashes, and underscores; solver names must start with a +lowercase ASCII letter. The special name `internal` denotes APT's +internal solver, is reserved, and cannot be used by external solvers. + ## Installation @@ -32,8 +37,8 @@ Each file under `/usr/lib/apt/solvers` corresponding to an external solver must be executable. No non-solver files must be installed under `/usr/lib/apt/solvers`, so -that an index of available external solvers can be obtained by simply -looking at the content of that directory. +that an index of available external solvers can be obtained by listing +the content of that directory. ## Configuration @@ -49,9 +54,10 @@ configuration documentation for more, and more up to date, information. respected (as the internal solver does) or can be slightly deviated from. Defaults to `yes`. -- **APT::Solver::Preferences**: solver-specific user preferences used - during dependency solving. Check your solver documentation for what is - supported here. Default to empty. +- **APT::Solver::NAME::Preferences** (where NAME is a solver name): + solver-specific user preference string used during dependency solving, + when the solver NAME is in use. Check solver-specific documentation + for what is supported here. Defaults to the empty string. ## Protocol @@ -74,13 +80,15 @@ via the file descriptors: **stdin** (standard input) and **stdout** therefore use stderr to dump debugging information that could be inspected separately. -After invocation, the protocol passes through 3 separate phases: +After invocation, the protocol passes through a sequence of phases: -1. APT send to the solver a dependency solving **scenario** -2. The solver solves dependencies. No communication with APT happens - during this phase. -3. The solver sends back to APT an **answer**, i.e. either a *solution* +1. APT invokes the external solver +2. APT send to the solver a dependency solving **scenario** +3. The solver solves dependencies. During this phase the solver may + send, repeatedly, **progress** information to APT. +4. The solver sends back to APT an **answer**, i.e. either a *solution* or an *error* report. +5. The external solver exits ### Scenario @@ -104,7 +112,7 @@ and followed by a mixture of action and preference fields. The value of the **Request:** field is a string describing the EDSP protocol which will be used to communicate. At present, the string must -be `EDSP 0.1`. +be `EDSP 0.2`. a unique request identifier, such as an UUID. Request fields are mainly used to identify the beginning of a @@ -154,32 +162,42 @@ A package universe is a list of Deb 822 stanzas, one per package, called **package stanzas**. Each package stanzas starts with a Package field. The following fields are supported in package stanzas: -- All fields supported by Debian Packages file (see one of the - `/var/lib/apt/lists/*Packages` file for an example), *with the - exception of the Description field* that is not allowed. +- All fields contained in the dpkg database, with the exception of + fields marked as "internal" (see the manpage `dpkg-query (1)`). Among + those fields, the following are mandatory for all package stanzas: + Package, Version, Architecture. - Among those fields, the following are mandatory: Package, Version, - Architecture. + It is recommended not to pass the Description field to external + solvers or, alternatively, to trim it to the short description only. -- **Installed:** (optional, default value `no`). Allowed values: `yes`, +- **Installed:** (optional, defaults to `no`). Allowed values: `yes`, `no`. When set to `yes`, the corresponding package is currently installed. + + Note: the Status field present in the dpkg database must not be passed + to the external solver, as it's an internal dpkg field. Installed and + other fields permit to encode the most relevant aspects of Status in + communications with solvers. - ##TODO## changed with respect to current prototype, which uses Status +- **Hold:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, the corresponding package is marked as "on + hold" by dpkg. - **APT-ID:** (mandatory). Unique package identifier, according to APT. -- **APT-Pin:** (mandatory). Must be a non-negative integer. Package pin - value, according to current APT policy. +- **APT-Pin:** (mandatory). Must be an integer. Package pin value, + according to APT policy. -- **APT-Candidate:** (optional, default value `no`). Allowed values: - `yes`, `no`. When set to `yes`, the corresponding package is granted - to have the highest pinning value among all the packages having the - same name. - - ##TODO## what about multi-arch? is the pin value granted to be the - higest also across different architectures? - +- **APT-Candidate:** (optional, defaults to `no`). Allowed values: + `yes`, `no`. When set to `yes`, the corresponding package is the APT + candidate for installation among all available packages with the same + name. + +- **APT-Automatic:** (optional, defaults to `no`). Allowed values: + `yes`, `no`. When set to `yes`, the corresponding package is marked by + APT as automatic installed. Note that automatic installed packages + should be removed by the solver only when the Autoremove action is + requested (see Request section). ### Answer @@ -189,7 +207,7 @@ An answer from the external solver to APT is either a *solution* or an The following invariant on **exit codes** must hold true. When the external solver is *able to find a solution*, it will write the solution to standard output and then exit with an exit code of 0. When the -external solver is *unable to find a solution* (and aware of that), it +external solver is *unable to find a solution* (and s aware of that), it will write an error to standard output and then exit with an exit code of 0. An exit code other than 0 will be interpreted as a solver crash with no meaningful error about dependency resolution to convey to the @@ -198,21 +216,29 @@ user. #### Solution -A solution is a single Deb 822 stanza, starting with the field -Solution. The following fields are supported in solution stanzas: +A solution is a list of Deb 822 stanzas. Each of them is either an +install stanza, telling APT to install a specific package, or a remove +stanza, telling APT to remove one. + +An **install stanza** starts with an Install field and supports the +following fields: + +- **Install:** (mandatory). The value is a package identifier, + referencing one of the package stanzas of the package universe via its + APT-ID field. -- **Solution:** (mandatory). The value of this field is ignored, - although it should be a unique solution identifier, such as a UUID. +- All fields supported by package stanzas. -- **Install:** (optional, defaults to the empty string). A space - separated list of strings of the form `PACKAGE=VERSION` where - `PACKAGE` is a package name and `VERSION` is an available version of - that package. The list denotes a set of packages that must be - installed to satisfy user request. +**Remove stanzas** are similar to install stanzas, but have **Remove** +fields instead of Install fields. -- **Remove:** (optional, defaults to the empty string). Same as Install, - but denoting a set of packages that must be removed to satisfy user - request. +In terms of expressivity, install and remove stanzas can carry one +single field each, as APT-IDs are enough to pinpoint packages to be +installed/removed. Nonetheless, for protocol readability, it is +recommended that solvers either add unconditionally the fields Package, +Version, and Architecture to all install/remove stanzas or, +alternatively, that they support a `--verbose` command line flag that +explicitly enables the output of those fields in solutions. #### Error @@ -225,11 +251,40 @@ following fields are supported in error stanzas: - **Message:** (mandatory). The value of this field is a text string, meant to be read by humans, that explains the cause of the solver - error. + error. Message fields might be multi-line, like the Description field + in the dpkg database. The first line conveys a short message, which + can be explained in more details using subsequent lines. + + +### Progress + +During dependency solving, an external solver may send progress +information to APT using **progress stanzas**. A progress stanza starts +with the Progress field and might contain the following fields: + +- **Progress:** (mandatory). The value of this field is a date and time + timestamp, in RFC 2822 format. The timestamp provides a time + annotation for the progress report. + +- **Percentage:** (optional). An integer from 0 to 100, representing the + completion of the dependency solving process, as declared by the + solver. + +- **Message:** (optional). A textual message, meant to be read by the + APT user, telling what is going on within the dependency solving + (e.g. the current phase of dependency solving, as declared by the + solver). + + +# Future extensions + +Potential future extensions to this protocol, listed in no specific +order, include: - ##TODO## can we support line continuations throughout this format? If - yes, they might come handy both for error stanzas and for solution - stanzas (which might have very long install/remove lines) +- fixed error types to identify common failures across solvers and + enable APT to translate error messages +- structured error data to explain failures in terms of packages and + dependencies ** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** -- cgit v1.2.3-70-g09d2 From 825780ef6c39e9fd89f761a8a9c873c848fb894d Mon Sep 17 00:00:00 2001 From: Stefano Zacchiroli Date: Thu, 31 Mar 2011 11:49:34 +0200 Subject: add indicative Autoremove stanzas in solutions as suggested by David EDSP version 0.3 --- doc/apt-solver-protocol.mdwn | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/doc/apt-solver-protocol.mdwn b/doc/apt-solver-protocol.mdwn index 85cd98d62..4c520b037 100644 --- a/doc/apt-solver-protocol.mdwn +++ b/doc/apt-solver-protocol.mdwn @@ -1,6 +1,6 @@ ** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** -# APT External Dependency Solver Protocol (EDSP) - version 0.2 +# APT External Dependency Solver Protocol (EDSP) - version 0.3 This document describes the communication protocol between APT and external dependency solvers. The protocol is called APT EDSP, for "APT @@ -112,7 +112,7 @@ and followed by a mixture of action and preference fields. The value of the **Request:** field is a string describing the EDSP protocol which will be used to communicate. At present, the string must -be `EDSP 0.2`. +be `EDSP 0.3`. a unique request identifier, such as an UUID. Request fields are mainly used to identify the beginning of a @@ -216,9 +216,11 @@ user. #### Solution -A solution is a list of Deb 822 stanzas. Each of them is either an -install stanza, telling APT to install a specific package, or a remove -stanza, telling APT to remove one. +A solution is a list of Deb 822 stanzas. Each of them could be an +install stanza (telling APT to install a specific package), a remove +stanza (telling APT to remove one), or an autoremove stanza (telling APT +about the *future* possibility of removing a package using the +Autoremove action). An **install stanza** starts with an Install field and supports the following fields: @@ -232,6 +234,14 @@ following fields: **Remove stanzas** are similar to install stanzas, but have **Remove** fields instead of Install fields. +**Autoremove stanzas** are similar to install stanzas, but have +**Autoremove** fields instead of Install fields. Autoremove stanzas +should be output so that APT can inform the user of which packages they +can now autoremove, as a consequence of the executed action. However, +this protocol makes no assumption on the fact that a subsequent +invocation of an Autoremove action will actually remove the very same +packages indicated by Autoremove stanzas in the former solution. + In terms of expressivity, install and remove stanzas can carry one single field each, as APT-IDs are enough to pinpoint packages to be installed/removed. Nonetheless, for protocol readability, it is -- cgit v1.2.3-70-g09d2 From 90bcac9e663d1a37eb2e0f1ba0878d0a545c9991 Mon Sep 17 00:00:00 2001 From: Stefano Zacchiroli Date: Mon, 4 Apr 2011 11:54:54 +0200 Subject: edsp: rename and drop very-draft warnings --- doc/apt-solver-protocol.mdwn | 300 ---------------------------- doc/external-dependency-solver-protocol.txt | 295 +++++++++++++++++++++++++++ 2 files changed, 295 insertions(+), 300 deletions(-) delete mode 100644 doc/apt-solver-protocol.mdwn create mode 100644 doc/external-dependency-solver-protocol.txt diff --git a/doc/apt-solver-protocol.mdwn b/doc/apt-solver-protocol.mdwn deleted file mode 100644 index 4c520b037..000000000 --- a/doc/apt-solver-protocol.mdwn +++ /dev/null @@ -1,300 +0,0 @@ -** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** - -# APT External Dependency Solver Protocol (EDSP) - version 0.3 - -This document describes the communication protocol between APT and -external dependency solvers. The protocol is called APT EDSP, for "APT -External Dependency Solver Protocol". - - -## Components - -- **APT**: we know this one. -- APT is equipped with its own **internal solver** for dependencies, - which is identified by the string `internal`. -- **External solver**: an *external* software component able to resolve - dependencies on behalf of APT. - -At each interaction with APT, a single solver is in use. When there is -a total of 2 or more solvers, internals or externals, the user can -choose which one to use. - -Each solver is identified by an unique string, the **solver -name**. Solver names must be formed using only alphanumeric ASCII -characters, dashes, and underscores; solver names must start with a -lowercase ASCII letter. The special name `internal` denotes APT's -internal solver, is reserved, and cannot be used by external solvers. - - -## Installation - -Each external solver is installed as a file under -`/usr/lib/apt/solvers`. The naming scheme is -`/usr/lib/apt/solvers/NAME`, where `NAME` is the name of the external -solver. - -Each file under `/usr/lib/apt/solvers` corresponding to an external -solver must be executable. - -No non-solver files must be installed under `/usr/lib/apt/solvers`, so -that an index of available external solvers can be obtained by listing -the content of that directory. - - -## Configuration - -Several APT options can be used to affect dependency solving in APT. An -overview of them is given below. Please refer to proper APT -configuration documentation for more, and more up to date, information. - -- **APT::Solver::Name**: the name of the solver to be used for - dependency solving. Defaults to `internal` - -- **APT::Solver::Strict-Pinning**: whether pinning must be strictly - respected (as the internal solver does) or can be slightly deviated - from. Defaults to `yes`. - -- **APT::Solver::NAME::Preferences** (where NAME is a solver name): - solver-specific user preference string used during dependency solving, - when the solver NAME is in use. Check solver-specific documentation - for what is supported here. Defaults to the empty string. - - -## Protocol - -When configured to use an external solver, APT will resort to it to -decide which packages should be installed or removed. - -The interaction happens **in batch**: APT will invoke the external -solver passing the current status of installed and available packages, -as well as the user request to alter the set of installed packages. The -external solver will compute a new complete set of installed packages -and gives APT a "diff" listing of which *additional* packages should be -installed and of which currently installed packages should be -*removed*. (Note: the order in which those actions have to be performed -will be up to APT to decide.) - -External solvers are invoked by executing them. Communications happens -via the file descriptors: **stdin** (standard input) and **stdout** -(standard output). stderr is not used by the EDSP protocol. Solvers can -therefore use stderr to dump debugging information that could be -inspected separately. - -After invocation, the protocol passes through a sequence of phases: - -1. APT invokes the external solver -2. APT send to the solver a dependency solving **scenario** -3. The solver solves dependencies. During this phase the solver may - send, repeatedly, **progress** information to APT. -4. The solver sends back to APT an **answer**, i.e. either a *solution* - or an *error* report. -5. The external solver exits - - -### Scenario - -A scenario is a text file encoded in a format very similar to the "Deb -822" format (AKA "the format used by Debian `Packages` files"). A -scenario consists of two distinct parts: a **request** and a **package -universe**, occurring in that order. The request consists of a single -Deb 822 stanza, while the package universe consists of several such -stanzas. All stanzas occurring in a scenario are separated by an empty -line. - - -#### Request - -Within a dependency solving scenario, a request represents the action on -installed packages requested by the user. - -A request is a single Deb 822 stanza opened by a mandatory Request field -and followed by a mixture of action and preference fields. - -The value of the **Request:** field is a string describing the EDSP -protocol which will be used to communicate. At present, the string must -be `EDSP 0.3`. - -a unique request identifier, such as an -UUID. Request fields are mainly used to identify the beginning of a -request stanza; their actual values are otherwise not used by the EDSP -protocol. - -The following **action fields** are supported in request stanzas: - -- **Install:** (optional, defaults to the empty string) A space - separated list of package names, with *no version attached*, to - install. This field denotes a list of packages that the user wants to - install, usually via an APT `install` request. - -- **Remove:** (optional, defaults to the empty string) Same syntax of - Install. This field denotes a list of packages that the user wants to - remove, usually via APT `remove` or `purge` requests. - -- **Upgrade:** (optional, defaults to `no`). Allowed values: `yes`, - `no`. When set to `yes`, an upgrade of all installed packages has been - requested, usually via an APT `upgrade` request. - -- **Dist-Upgrade:** (optional, defaults to `no`). Allowed values: `yes`, - `no`. Same as Upgrade, but for APT `dist-upgrade` requests. - -- **Autoremove:** (optional, defaults to `no`). Allowed values: `yes`, - `no`. When set to `yes`, a clean up of unused automatically installed - packages has been requested, usually via an APT `autoremove` request. - -The following **preference fields** are supported in request stanzas: - -- **Strict-Pinning:** (optional, defaults to `yes`). Allowed values: - `yes`, `no`. When set to `yes`, APT pinning is strict, in the sense - that the solver must not propose to install packages which are not APT - candidates (see the `APT-Pin` and `APT-Candidate` fields in the - package universe). When set to `no`, the solver does only a best - effort attempt to install APT candidates. Usually, the value of this - field comes from the `APT::Solver::Strict-Pinning` configuration - option. - -- **Preferences:** a solver-specific optimization string, usually coming - from the `APT::Solver::Preferences` configuration option. - - -#### Package universe - -A package universe is a list of Deb 822 stanzas, one per package, called -**package stanzas**. Each package stanzas starts with a Package -field. The following fields are supported in package stanzas: - -- All fields contained in the dpkg database, with the exception of - fields marked as "internal" (see the manpage `dpkg-query (1)`). Among - those fields, the following are mandatory for all package stanzas: - Package, Version, Architecture. - - It is recommended not to pass the Description field to external - solvers or, alternatively, to trim it to the short description only. - -- **Installed:** (optional, defaults to `no`). Allowed values: `yes`, - `no`. When set to `yes`, the corresponding package is currently - installed. - - Note: the Status field present in the dpkg database must not be passed - to the external solver, as it's an internal dpkg field. Installed and - other fields permit to encode the most relevant aspects of Status in - communications with solvers. - -- **Hold:** (optional, defaults to `no`). Allowed values: `yes`, - `no`. When set to `yes`, the corresponding package is marked as "on - hold" by dpkg. - -- **APT-ID:** (mandatory). Unique package identifier, according to APT. - -- **APT-Pin:** (mandatory). Must be an integer. Package pin value, - according to APT policy. - -- **APT-Candidate:** (optional, defaults to `no`). Allowed values: - `yes`, `no`. When set to `yes`, the corresponding package is the APT - candidate for installation among all available packages with the same - name. - -- **APT-Automatic:** (optional, defaults to `no`). Allowed values: - `yes`, `no`. When set to `yes`, the corresponding package is marked by - APT as automatic installed. Note that automatic installed packages - should be removed by the solver only when the Autoremove action is - requested (see Request section). - -### Answer - -An answer from the external solver to APT is either a *solution* or an -*error*. - -The following invariant on **exit codes** must hold true. When the -external solver is *able to find a solution*, it will write the solution -to standard output and then exit with an exit code of 0. When the -external solver is *unable to find a solution* (and s aware of that), it -will write an error to standard output and then exit with an exit code -of 0. An exit code other than 0 will be interpreted as a solver crash -with no meaningful error about dependency resolution to convey to the -user. - - -#### Solution - -A solution is a list of Deb 822 stanzas. Each of them could be an -install stanza (telling APT to install a specific package), a remove -stanza (telling APT to remove one), or an autoremove stanza (telling APT -about the *future* possibility of removing a package using the -Autoremove action). - -An **install stanza** starts with an Install field and supports the -following fields: - -- **Install:** (mandatory). The value is a package identifier, - referencing one of the package stanzas of the package universe via its - APT-ID field. - -- All fields supported by package stanzas. - -**Remove stanzas** are similar to install stanzas, but have **Remove** -fields instead of Install fields. - -**Autoremove stanzas** are similar to install stanzas, but have -**Autoremove** fields instead of Install fields. Autoremove stanzas -should be output so that APT can inform the user of which packages they -can now autoremove, as a consequence of the executed action. However, -this protocol makes no assumption on the fact that a subsequent -invocation of an Autoremove action will actually remove the very same -packages indicated by Autoremove stanzas in the former solution. - -In terms of expressivity, install and remove stanzas can carry one -single field each, as APT-IDs are enough to pinpoint packages to be -installed/removed. Nonetheless, for protocol readability, it is -recommended that solvers either add unconditionally the fields Package, -Version, and Architecture to all install/remove stanzas or, -alternatively, that they support a `--verbose` command line flag that -explicitly enables the output of those fields in solutions. - - -#### Error - -An error is a single Deb 822 stanza, starting the field Error. The -following fields are supported in error stanzas: - -- **Error:** (mandatory). The value of this field is ignored, although - it should be a unique error identifier, such as a UUID. - -- **Message:** (mandatory). The value of this field is a text string, - meant to be read by humans, that explains the cause of the solver - error. Message fields might be multi-line, like the Description field - in the dpkg database. The first line conveys a short message, which - can be explained in more details using subsequent lines. - - -### Progress - -During dependency solving, an external solver may send progress -information to APT using **progress stanzas**. A progress stanza starts -with the Progress field and might contain the following fields: - -- **Progress:** (mandatory). The value of this field is a date and time - timestamp, in RFC 2822 format. The timestamp provides a time - annotation for the progress report. - -- **Percentage:** (optional). An integer from 0 to 100, representing the - completion of the dependency solving process, as declared by the - solver. - -- **Message:** (optional). A textual message, meant to be read by the - APT user, telling what is going on within the dependency solving - (e.g. the current phase of dependency solving, as declared by the - solver). - - -# Future extensions - -Potential future extensions to this protocol, listed in no specific -order, include: - -- fixed error types to identify common failures across solvers and - enable APT to translate error messages -- structured error data to explain failures in terms of packages and - dependencies - - -** TENTATIVE PROPOSAL, VERY VERY VERY DRAFT ** diff --git a/doc/external-dependency-solver-protocol.txt b/doc/external-dependency-solver-protocol.txt new file mode 100644 index 000000000..8c13b185c --- /dev/null +++ b/doc/external-dependency-solver-protocol.txt @@ -0,0 +1,295 @@ +# APT External Dependency Solver Protocol (EDSP) - version 0.3 + +This document describes the communication protocol between APT and +external dependency solvers. The protocol is called APT EDSP, for "APT +External Dependency Solver Protocol". + + +## Components + +- **APT**: we know this one. +- APT is equipped with its own **internal solver** for dependencies, + which is identified by the string `internal`. +- **External solver**: an *external* software component able to resolve + dependencies on behalf of APT. + +At each interaction with APT, a single solver is in use. When there is +a total of 2 or more solvers, internals or externals, the user can +choose which one to use. + +Each solver is identified by an unique string, the **solver +name**. Solver names must be formed using only alphanumeric ASCII +characters, dashes, and underscores; solver names must start with a +lowercase ASCII letter. The special name `internal` denotes APT's +internal solver, is reserved, and cannot be used by external solvers. + + +## Installation + +Each external solver is installed as a file under +`/usr/lib/apt/solvers`. The naming scheme is +`/usr/lib/apt/solvers/NAME`, where `NAME` is the name of the external +solver. + +Each file under `/usr/lib/apt/solvers` corresponding to an external +solver must be executable. + +No non-solver files must be installed under `/usr/lib/apt/solvers`, so +that an index of available external solvers can be obtained by listing +the content of that directory. + + +## Configuration + +Several APT options can be used to affect dependency solving in APT. An +overview of them is given below. Please refer to proper APT +configuration documentation for more, and more up to date, information. + +- **APT::Solver::Name**: the name of the solver to be used for + dependency solving. Defaults to `internal` + +- **APT::Solver::Strict-Pinning**: whether pinning must be strictly + respected (as the internal solver does) or can be slightly deviated + from. Defaults to `yes`. + +- **APT::Solver::NAME::Preferences** (where NAME is a solver name): + solver-specific user preference string used during dependency solving, + when the solver NAME is in use. Check solver-specific documentation + for what is supported here. Defaults to the empty string. + + +## Protocol + +When configured to use an external solver, APT will resort to it to +decide which packages should be installed or removed. + +The interaction happens **in batch**: APT will invoke the external +solver passing the current status of installed and available packages, +as well as the user request to alter the set of installed packages. The +external solver will compute a new complete set of installed packages +and gives APT a "diff" listing of which *additional* packages should be +installed and of which currently installed packages should be +*removed*. (Note: the order in which those actions have to be performed +will be up to APT to decide.) + +External solvers are invoked by executing them. Communications happens +via the file descriptors: **stdin** (standard input) and **stdout** +(standard output). stderr is not used by the EDSP protocol. Solvers can +therefore use stderr to dump debugging information that could be +inspected separately. + +After invocation, the protocol passes through a sequence of phases: + +1. APT invokes the external solver +2. APT send to the solver a dependency solving **scenario** +3. The solver solves dependencies. During this phase the solver may + send, repeatedly, **progress** information to APT. +4. The solver sends back to APT an **answer**, i.e. either a *solution* + or an *error* report. +5. The external solver exits + + +### Scenario + +A scenario is a text file encoded in a format very similar to the "Deb +822" format (AKA "the format used by Debian `Packages` files"). A +scenario consists of two distinct parts: a **request** and a **package +universe**, occurring in that order. The request consists of a single +Deb 822 stanza, while the package universe consists of several such +stanzas. All stanzas occurring in a scenario are separated by an empty +line. + + +#### Request + +Within a dependency solving scenario, a request represents the action on +installed packages requested by the user. + +A request is a single Deb 822 stanza opened by a mandatory Request field +and followed by a mixture of action and preference fields. + +The value of the **Request:** field is a string describing the EDSP +protocol which will be used to communicate. At present, the string must +be `EDSP 0.3`. + +a unique request identifier, such as an +UUID. Request fields are mainly used to identify the beginning of a +request stanza; their actual values are otherwise not used by the EDSP +protocol. + +The following **action fields** are supported in request stanzas: + +- **Install:** (optional, defaults to the empty string) A space + separated list of package names, with *no version attached*, to + install. This field denotes a list of packages that the user wants to + install, usually via an APT `install` request. + +- **Remove:** (optional, defaults to the empty string) Same syntax of + Install. This field denotes a list of packages that the user wants to + remove, usually via APT `remove` or `purge` requests. + +- **Upgrade:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, an upgrade of all installed packages has been + requested, usually via an APT `upgrade` request. + +- **Dist-Upgrade:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. Same as Upgrade, but for APT `dist-upgrade` requests. + +- **Autoremove:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, a clean up of unused automatically installed + packages has been requested, usually via an APT `autoremove` request. + +The following **preference fields** are supported in request stanzas: + +- **Strict-Pinning:** (optional, defaults to `yes`). Allowed values: + `yes`, `no`. When set to `yes`, APT pinning is strict, in the sense + that the solver must not propose to install packages which are not APT + candidates (see the `APT-Pin` and `APT-Candidate` fields in the + package universe). When set to `no`, the solver does only a best + effort attempt to install APT candidates. Usually, the value of this + field comes from the `APT::Solver::Strict-Pinning` configuration + option. + +- **Preferences:** a solver-specific optimization string, usually coming + from the `APT::Solver::Preferences` configuration option. + + +#### Package universe + +A package universe is a list of Deb 822 stanzas, one per package, called +**package stanzas**. Each package stanzas starts with a Package +field. The following fields are supported in package stanzas: + +- All fields contained in the dpkg database, with the exception of + fields marked as "internal" (see the manpage `dpkg-query (1)`). Among + those fields, the following are mandatory for all package stanzas: + Package, Version, Architecture. + + It is recommended not to pass the Description field to external + solvers or, alternatively, to trim it to the short description only. + +- **Installed:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, the corresponding package is currently + installed. + + Note: the Status field present in the dpkg database must not be passed + to the external solver, as it's an internal dpkg field. Installed and + other fields permit to encode the most relevant aspects of Status in + communications with solvers. + +- **Hold:** (optional, defaults to `no`). Allowed values: `yes`, + `no`. When set to `yes`, the corresponding package is marked as "on + hold" by dpkg. + +- **APT-ID:** (mandatory). Unique package identifier, according to APT. + +- **APT-Pin:** (mandatory). Must be an integer. Package pin value, + according to APT policy. + +- **APT-Candidate:** (optional, defaults to `no`). Allowed values: + `yes`, `no`. When set to `yes`, the corresponding package is the APT + candidate for installation among all available packages with the same + name. + +- **APT-Automatic:** (optional, defaults to `no`). Allowed values: + `yes`, `no`. When set to `yes`, the corresponding package is marked by + APT as automatic installed. Note that automatic installed packages + should be removed by the solver only when the Autoremove action is + requested (see Request section). + +### Answer + +An answer from the external solver to APT is either a *solution* or an +*error*. + +The following invariant on **exit codes** must hold true. When the +external solver is *able to find a solution*, it will write the solution +to standard output and then exit with an exit code of 0. When the +external solver is *unable to find a solution* (and s aware of that), it +will write an error to standard output and then exit with an exit code +of 0. An exit code other than 0 will be interpreted as a solver crash +with no meaningful error about dependency resolution to convey to the +user. + + +#### Solution + +A solution is a list of Deb 822 stanzas. Each of them could be an +install stanza (telling APT to install a specific package), a remove +stanza (telling APT to remove one), or an autoremove stanza (telling APT +about the *future* possibility of removing a package using the +Autoremove action). + +An **install stanza** starts with an Install field and supports the +following fields: + +- **Install:** (mandatory). The value is a package identifier, + referencing one of the package stanzas of the package universe via its + APT-ID field. + +- All fields supported by package stanzas. + +**Remove stanzas** are similar to install stanzas, but have **Remove** +fields instead of Install fields. + +**Autoremove stanzas** are similar to install stanzas, but have +**Autoremove** fields instead of Install fields. Autoremove stanzas +should be output so that APT can inform the user of which packages they +can now autoremove, as a consequence of the executed action. However, +this protocol makes no assumption on the fact that a subsequent +invocation of an Autoremove action will actually remove the very same +packages indicated by Autoremove stanzas in the former solution. + +In terms of expressivity, install and remove stanzas can carry one +single field each, as APT-IDs are enough to pinpoint packages to be +installed/removed. Nonetheless, for protocol readability, it is +recommended that solvers either add unconditionally the fields Package, +Version, and Architecture to all install/remove stanzas or, +alternatively, that they support a `--verbose` command line flag that +explicitly enables the output of those fields in solutions. + + +#### Error + +An error is a single Deb 822 stanza, starting the field Error. The +following fields are supported in error stanzas: + +- **Error:** (mandatory). The value of this field is ignored, although + it should be a unique error identifier, such as a UUID. + +- **Message:** (mandatory). The value of this field is a text string, + meant to be read by humans, that explains the cause of the solver + error. Message fields might be multi-line, like the Description field + in the dpkg database. The first line conveys a short message, which + can be explained in more details using subsequent lines. + + +### Progress + +During dependency solving, an external solver may send progress +information to APT using **progress stanzas**. A progress stanza starts +with the Progress field and might contain the following fields: + +- **Progress:** (mandatory). The value of this field is a date and time + timestamp, in RFC 2822 format. The timestamp provides a time + annotation for the progress report. + +- **Percentage:** (optional). An integer from 0 to 100, representing the + completion of the dependency solving process, as declared by the + solver. + +- **Message:** (optional). A textual message, meant to be read by the + APT user, telling what is going on within the dependency solving + (e.g. the current phase of dependency solving, as declared by the + solver). + + +# Future extensions + +Potential future extensions to this protocol, listed in no specific +order, include: + +- fixed error types to identify common failures across solvers and + enable APT to translate error messages +- structured error data to explain failures in terms of packages and + dependencies -- cgit v1.2.3-70-g09d2 From a1421e1e452e36b3f82a1cf23d2bf806ca9262c1 Mon Sep 17 00:00:00 2001 From: Stefano Zacchiroli Date: Mon, 4 Apr 2011 11:57:04 +0200 Subject: install EDSP protocol description in apt-doc --- debian/apt-doc.docs | 1 + 1 file changed, 1 insertion(+) diff --git a/debian/apt-doc.docs b/debian/apt-doc.docs index 86aa69ceb..4ec23f55d 100644 --- a/debian/apt-doc.docs +++ b/debian/apt-doc.docs @@ -1,2 +1,3 @@ README.progress-reporting README.MultiArch +doc/external-dependency-solver-protocol.txt -- cgit v1.2.3-70-g09d2