diff options
Diffstat (limited to 'doc/files.sgml')
| -rw-r--r-- | doc/files.sgml | 491 |
1 files changed, 491 insertions, 0 deletions
diff --git a/doc/files.sgml b/doc/files.sgml new file mode 100644 index 000000000..d98d6f68a --- /dev/null +++ b/doc/files.sgml @@ -0,0 +1,491 @@ +<!doctype debiandoc system> +<!-- -*- mode: sgml; mode: fold -*- --> +<book> +<title>APT Files</title> + +<author>Jason Gunthorpe <email>jgg@debian.org</email></author> +<version>$Id: files.sgml,v 1.1 1998/07/02 02:58:12 jgg Exp $</version> + +<abstract> +This document describes the complete implementation and format of the +installed APT directory structure. It also serves as guide to how APT +views the Debian archive. +</abstract> + +<copyright> +Copyright © Jason Gunthorpe, 1998. +<p> +"APT" and this document are free software; you can redistribute them and/or +modify them under the terms of the GNU General Public License as published +by the Free Software Foundation; either version 2 of the License, or (at your +option) any later version. + +<p> +For more details, on Debian GNU/Linux systems, see the file +/usr/doc/copyright/GPL for the full license. +</copyright> + +<toc sect> + +<chapt>Introduction +<!-- General {{{ --> +<!-- ===================================================================== --> +<sect>General + +<p> +This document serves two purposes. The first is to document the installed +directory structure and the format and purpose of each file. The second +purpose is to document how APT views the Debian archive and deals with +multiple package files. + +<p> +The var directory structure is as follows: +<example> + /var/state/apt/ + lists/ + partial/ + xstatus + /var/cache/apt/ + pkgcache.bin + srcpkgcache.bin + archives/ + partial/ + /etc/apt/ + sources.list + cdromdevs.list + /usr/lib/apt/ + methods/ + cdrom + ftp + http +</example> + +<p> +As is specified in the FHS 2.0 /var/state/apt is used for application +data that is not expected to be user modified. /var/cache/apt is used +for regeneratable data and is where the package cache and downloaded .debs +go. +</sect> + <!-- }}} --> + +<chapt>Files +<!-- Distribution Source List {{{ --> +<!-- ===================================================================== --> +<sect>Distribution Source list (sources.list) + +<p> +The distribution source list is used to locate archives of the debian +distribution. It is designed to support any number of active sources and to +support a mix of source media. The file lists one source per line, with the +fastest source listed first. The format of each line is: + +<p> +<var>type ui args</var> + +<p> +The first item, <var>type</var>, indicates the format for the remainder +of the line. It is designed to indicate the structure of the distribution +the line is talking about. Currently the only defined value is <em>deb</em> +which indicates a standard debian archive with a dists dir. + +<sect1>The deb Type + <p> + The <em>deb</em> type is to be a typical two level debian distributions, + dist/<var>distribution</var>/<var>component</var>. Typically distribution + is one of stable, unstable or frozen while component is one of main, + contrib, non-free or non-us. The format for the deb line is as follows: + + <p> + deb <var>uri</var> <var>distribution</var> <var>compontent</var> + [<var>component</var> ...] + + <p> + <var>uri</var> for the <em>deb</em> type must specify the base of the + debian distribution. APT will automatically generate the proper longer + URIs to get the information it needs. <var>distribution</var> can specify + an exact path, in this case the components must be omitted and + <var>distribution</var> must end in a slash. + + <p> + Since only one distribution can be specified per deb line it may be + necessary to list a number of deb lines for the same URI. APT will + sort the URI list after it has generated a complete set to allow + connection reuse. It is important to order things in the sourcelist + from most prefered to least prefered (fastest to slowest). +</sect1> + +<sect1>URI specification +<p> +URIs in the source list support a large number of access schemes. + +<taglist> +<tag>cdrom<item> + The cdrom scheme is special in that If Modifed Since queries are never + performed and that APT knows how to match a cdrom to the name it + was given when first inserted. It does this by examining the date + and size of the package file. APT also knows all of the possible + prefix paths for the cdrom drives and that the user should be prompted + to insert a CD if it cannot be found. The path is relative to an + arbitary mount point (of APT's choosing) and must not start with a + slash. The first pathname component is the given name and is purely + descriptive and of the users choice. However, if a file in the root of + the cdrom is called 'cdname' its contents will be used instead of + prompting. The name serves as a tag for the cdrom and should be unique. + APT will track the CDROM's based on their tag and package file + properties. + <example> + cdrom:Debian 1.3/debian + </example> + +<tag>http<item> + This scheme specifies a HTTP server for the debian archive. HTTP is prefered + over FTP because If Modified Since queries against the Package file are + possible. Newer HTTP protcols may even support reget which would make + http the protocol of choice. + <example> + http://www.debian.org/archive + </example> + +<tag>ftp<item> + This scheme specifies a FTP connection to the server. FTP is limited because + there is no support for IMS and is hard to proxy over firewalls. + <example> + ftp://ftp.debian.org/debian + </example> + +<tag>file<item> + The file scheme allows an arbitary directory in the file system to be + considered as a debian archive. This is usefull for NFS mounts and + local mirrors/archives. + <example> + file:/var/debian + </example> + +<tag>mirror<item> + The mirror scheme is special in that it does not specify the location of a + debian archive but specifies the location of a list of mirrors to use + to access the archive. Some technique will be used to determine the + best choice for a mirror. The mirror file is specified in the Mirror File + section. If/when URIs take off they should obsolete this field. + <example> + mirror:http://www.debian.org/archivemirrors + </example> + +<tag>smb<item> + A possible future expansion may be to have direct support for smb (Samba + servers). + <example> + smb://ftp.kernel.org/pub/mirrors/debian + </example> +</taglist> +</sect1> + +<sect1>Hashing the URI +<p> +All permanent information aquired from any of the sources is stored in the +lists directory. Thus, there must be a way to relate the filename in the +lists directory to a line in the sourcelist. To simplify things this is +done by quoting the URI and treating ='s as quoteable characters and +converting / to =. The URI spec says this is done by converting a +sensitive character into %xx where xx is the hexadecimal representation +from the ascii character set. Examples: + +<example> +http://www.debian.org/archive/dists/stable/binary-i386/Packages +/var/state/apt/lists/www.debian.org=archive=dists=stable=binary-i386=Packages + +cdrom:Debian 1.3/debian/Packages +/var/state/apt/info/Debian%201.3=debian=Packages +</example> + +<p> +The other alternative that was considered was to use a deep directory +structure but this poses two problems, it makes it very difficult to prune +directories back when sources are no longer used and complicates the handling +of the partial directory. This gives a very simple way to deal with all +of the situations that can arise. The equals sign was choosen on the +suggestion of Manoj because it is very infrequently used in filenames. +Also note that the same rules described in the <em>Archive Directory</> +section regarding the partial sub dir apply here as well. +</sect1> + +</sect> + <!-- }}} --> +<!-- Extra Status {{{ --> +<!-- ===================================================================== --> +<sect>Extra Status File (xstatus) + +<p> +The extra status file serves the same purpose as the normal dpkg status file +(/var/lib/dpkg/status) except that it stores information unique to diety. +This includes the autoflag, target distribution and version and any other +uniqe features that come up over time. It duplicates nothing from the normal +dpkg status file. Please see other APT documentation for a discussion +of the exact internal behavior of these fields. The Package field is +placed directly before the new fields to indicate which package they +apply to. The new fields are as follows: + +<taglist> +<tag>X-Auto<item> + The Auto flag can be Yes or No and controls whether the package is in + auto mode. + +<tag>X-TargetDist<item> + The TargetDist item indicates which distribution versions are offered for + installation from. It should be stable, unstable or frozen. + +<tag>X-TargetVersion<item> + The target version item is set if the user selects a specific version, it + overrides the TargetDist selection if both are present. +</taglist> +</sect> + <!-- }}} --> +<!-- Binary Package Cache {{{ --> +<!-- ===================================================================== --> +<sect>Binary Package Cache (pkgcache.bin) + +<p> +Please see cache.sgml for a complete description of what this file is. The +cache file is updated whenever the contents of the lists directory changes. +If the cache is erased, corrupted or of a non-matching version it will +be automatically rebuilt by all of the tools that need it. +<em>srcpkgcache.bin</> contains a cache of all of the package files in the +source list. This allows regeneration of the cache when the status files +change to use a prebuilt version for greater speed. +</sect> + <!-- }}} --> +<!-- Downloads Directory {{{ --> +<!-- ===================================================================== --> +<sect>Downloads Directory (archives) + +<p> +The archives directory is where all downloaded .deb archives go. When the +file transfer is initiated the deb is placed in partial. Once the file +is fully downloaded and its MD5 hash and size are verifitied it is moved +from partial into archives/. Any files found in archives/ can be assumed +to be verified. + +<p> +No dirctory structure is transfered from the receiving site and all .deb +file names conform to debian conventions. No short (msdos) filename should +be placed in archives. If the need arises .debs should be unpacked, scanned +and renamed to their correct internal names. This is mostly to prevent +file name conflicts but other programs may depend on this if convenient. +Downloaded .debs must be found in one of the package lists with an exact +name + version match.. +</sect> + <!-- }}} --> +<!-- The Methods Directory {{{ --> +<!-- ===================================================================== --> +<sect> The Methods Directory (/usr/lib/apt/methods) + +<p> +Like dselect, APT will support plugable acquisition methods to complement +its internaly supported methods. The files in +this directory are execultables named after the URI type. APT will +sort the required URIs and spawn these programs giving a full sorted, quoted +list of URIs. + +<p> +The interface is simple, the program will be given a list +of URIs on the command line. The URIs will be in pairs, the first +being the actual URI and the second being the filename to write the data to. +The current directory will be set properly by APT and it is +expected the method will put files relative to the current directory. +The output of these programs is strictly speficied. The programs must accept +nothing from stdin (stdin will be an invalid fd) and they must output +status information to stdout according to the format below. +Stderr will be redirected to the logging facility. + +<p> +Each line sent to stdout must be a line that has a single letter and a +space. Strings after the first letter do not need quoting, they are taken +as is till the end of the line. The tag letters, listed in expected order, +is as follows: + +<taglist> + +<tag>F - Change URI<item> +This specifies a change in URI. All information after this will be applied +to the new URI. When the URI is changed it is assumed that the old URI has +completed unless an error is set. The format is <var>F URI</> + +<tag>S - Object Size<item> +This specifies the expected size of the object. APT will use this to +compute percent done figures. If it is not sent then a kilobyte meter +will be used instead of a percent display. The foramat is <var>S INTEGER</> + +<tag>E - Error Information<item> +Exactly one line of error information can be set for each URI. The +information will be summarized for the user. If an E tag is send before +any F tags then the error is assumed to be a fatal method error and all URI +fetches for that method are aborted with that error string. The format +is <var>E String</> + +<tag>I - Informative progress information<item> +The I tag allows the method to specify the status of the connection. +Typically the GUI will show the last recieved I line. The format is +<var>I String</> As a general rule an I tag should be ommitted before a +lengthy operation only. Things that always take a short period are not +suited for I tags. I tags should change wnenever the methods state changes. +Some standard forms, in order of occurance, are <var>Connecting to SITE</>, +<var>Connecting to SITE (1.1.1.1)</>, <var>Waiting for file</>, +<var>Authenticating</>, <var>Downloading</>, <var>Resuming (size)</>, +<var>Computing MD5</> <var>I</> lines should never print out information that +APT is already aware of, such as file names. + +<tag>R - Set final path<item> +The R tag allows the method to tell APT that the file is present in the +local file system. APT might copy it into a the download directory. The format +is <var>R String</> + +<tag>M - MD5Sum of the file<item> +The method is expected to compute the md5 hash on the fly as the download +progresses. The final md5 of the file is to be output when the file is +completed. If the md5 is not output it will not be checked! Some methods +such as the file method will not check md5's because they are most +commonly used on mirrors or local CD-ROM's, a paranoid option may be +provided in future to force checking. The format is <var>M MD5-String</> + +<tag>L - Log output<item> +This tag indicates a string that should be dumped to some log file. The +string is for debugging and is not ment to be seen by the user. The format +is <var>L String</> Log things should only be used in a completed method +if they have special relavence to what is happening. +</taglist> + +<p> +APT monitors the progress of the transfer by watching the file size. This +means the method must not create any temp files and must use a fairly small +buffer. The method is also responsible for If-Modified-Since (IMS) queries +for the object. It should check ../outputname to get the time stamp but not +size. The size may be different because the file was uncompressed after +it was transfed. A method must <em>never</> change the file in .., it may +only change the output file in the current directory. + +<p> +The APT 'http' program is the reference implementation of this specification, +it implements all of the features a method is expected to do. +</sect> + <!-- }}} --> +<!-- The Mirror List {{{ --> +<!-- ===================================================================== --> +<sect> The Mirror List + +<p> +The mirror list is stored on the primary debian web server (www.debian.org) +and contains a machine readable list of all known debian mirrors. The mirror +URI type will cause this list to be downloaded and considered. It has the +same form as the source list. When the source list specifies mirror +as the target the mirror list is scanned to find the nescessary parts for +the requested distributions and components. This means the user could +have a line like: + +<var>deb mirror:http://www.debian.org/mirrorlist stable main non-us</var> + +which would likely cause APT to choose two separate sites to download from, +one for main and another for non-us. + +<p> +Some form of network measurement will have to be used to gauge performance +of each of the mirrors. This will be discussed later, initial versions +will use the first found URI. +</sect> + <!-- }}} --> +<!-- The Release File {{{ --> +<!-- ===================================================================== --> +<sect> The Release File + +<p> +This file plays and important role in how APT presents the archive to the +user. Its main purpose is to present a descriptive name for the source +of each version of each package. It also is used to detect when new versions +of debian are released. It augments the package file it is associated with +by providing meta information about the entire archive which the Packages +file describes. + +<p> +The full name of the distribution for presentation to the user is formed +as 'label version archive', with a possible extended name being +'label version archive component'. + +<p> +The file is formed as the package file (RFC-822) with the following tags +defined: + +<taglist> +<tag>Archive<item> +This is the common name we give our archives, such as <em>stable</> or +<em>unstable</>. + +<tag>Component<item> +Referes to the sub-component of the archive, <em>main</>, <em>contrib</> +etc. + +<tag>Version<item> +This is a version string with the same properties as in the Packages file. +It represents the release level of the archive. + +<tag>Origin<item> +This specifies who is providing this archive. In the case of Debian the +string will read 'Debian'. Other providers may use their own string + +<tag>Label<item> +This carries the encompassing name of the distribution. For Debian proper +this field reads 'Debian'. For derived distributions it should contain their +proper name. + +<tag>Architecture<item> +When the archive has packages for a single architecture then the Architecture +is listed here. If a mixed set of systems are represented then this should +contain the keyword <em>mixed</em>. + +<tag>NotAutomatic<item> +A Yes/No flag indicating that the archive is extremely unstable and its +version's should never be automatically selected. This is to be used by +experimental. + +<tag>Description<item> +Description is used to describe the release. For instance experimental would +contain a warning that the packages have problems. +</taglist> + +<p> +The location of the Release file in the archive is very important, it must +be located in the same location as the packages file so that it can be +located in all situations. The following is an example for the current stable +release, 1.3.1r6 + +<example> +Archive: stable +Compontent: main +Version: 1.3.1r6 +Origin: Debian +Label: Debian +Architecture: i386 +</example> + +This is an example of experimental, +<example> +Archive: experimental +Version: 0 +Origin: Debian +Label: Debian +Architecture: mixed +NotAutomatic: Yes +</example> + +And unstable, +<example> +Archive: unstable +Compontent: main +Version: 2.1 +Origin: Debian +Label: Debian +Architecture: i386 +</example> + +</sect> + <!-- }}} --> + +</book> |