From 1e0f0f28e1025f42a8172eb72f3e87984eb2b939 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Tue, 9 Jun 2015 11:59:22 +0200 Subject: configureable acquire targets to download additional files First pass at making the acquire system capable of downloading files based on configuration rather than hardcoded entries. It is now possible to instruct 'deb' and 'deb-src' sources.list lines to download more than just Packages/Translation-* and Sources files. Details on how to do that can be found in the included documentation file. --- doc/acquire-additional-files.txt | 142 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 doc/acquire-additional-files.txt (limited to 'doc') diff --git a/doc/acquire-additional-files.txt b/doc/acquire-additional-files.txt new file mode 100644 index 000000000..5a07c2bec --- /dev/null +++ b/doc/acquire-additional-files.txt @@ -0,0 +1,142 @@ +# Acquire additional files in 'update' operations + +The download and verification of data from multiple sources in different +compression formats, with partial downloads and patches is an involved +process which is hard to implement correctly and securely. + +APT frontends share the code and binaries to make this happen in libapt +with the Acquire system, supported by helpers shipped in the apt package +itself and additional transports in individual packages like +apt-transport-https. + +For its own operation libapt needs or can make use of Packages, Sources +and Translation-* files, which it will acquire by default, but +a repository might contain more data files (e.g. Contents) a frontend +might want to use and would therefore need to be downloaded as well +(e.g. apt-file). + +This file describes the configuration scheme such a frontend can use to +instruct the Acquire system to download those additional files. + +Note that you can't download files from other sources. It must be files +in the same repository and below the Release file. The Release file must +also contain hashes for the file itself as well as for the compressed +file if wanted, otherwise a download isn't even tried! + + +# The Configuration Stanza + +The Acquire system uses the same configuration settings to implement the +files it downloads by default. These settings are the default, but if +they would be written in a configuration file the configuration +instructing the Acquire system to download the Packages files would look +like this (see also apt.conf(5) manpage for configuration file syntax): + + APT::Acquire::Targets::deb::Packages { + URI "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"; + ShortDescription "Packages"; + Description "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages"; + + flatURI "Packages"; + flatDescription "$(SITE) $(RELEASE) Packages"; + + Optional "false"; + }; + +All files which should be downloaded (nicknamed 'Targets') are mentioned +below the APT::Acquire::Targets scope. 'deb' is here the type of the +sources.list entry the file should be acquired for. The only other +supported value is hence 'deb-src'. Beware: You can't specify multiple +types here and you can't download the same URI for multiple types! + +After the type you can pick any valid and unique string which preferable +refers to the file it downloads (In the example we picked 'Packages'). +This string is never shown or used. + +All targets have three main properties you can define: +* URI: The identifier of the file to be downloaded as used in the + Release file. It is also the relative location of the file from the + Release file. You can neither download from a different server + entirely (absolute URI) nor should you try to access directories above + the Release file (e.g. "../../"). +* ShortDescription: Very short string intended to be displayed to the + user e.g. while reporting progress. apt will e.g. use this string in + the last line to indicate progress of e.g. the download of a specific + item. +* Description: A preferable human understandable and readable identifier + of which file is acquired exactly. Mainly used for progress reporting + and error messages. apt will e.g. use this string in the Get/Hit/Err + progress lines. + +Additional optional properties: +* flat{URI,Description}: APT supports two types of repositories: + dists-style repositories which are the default and by far the most + common which are named after the fact that the files are in an + elaborated directory structure. In contrast a flat-style repositories + lumps all files together in one directory. Support for these flat + repositories exists mainly for legacy purposes only. It is therefore + recommend to not set these values. +* Optional: The default value is 'true' and should be kept at this + value. If enabled the acquire system will skip the download if the + file isn't mentioned in the Release file. Otherwise this is treated as + a hard error and the update process fails. + + +Note that the acquire system will automatically choose to download +a compressed file if it is available and uncompress it for you, just as +it will also use pdiff patching if provided by the repository and +enabled by the user. You only have to ensure that the Release file +contains the information about the compressed files/pdiffs to make this +happen. NO properties have to be set to enable this. + +# More examples + +The stanzas for Translation-* files as well as for Sources files would +look like this: + +APT::Acquire::Targets { + deb::Translations { + URI "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"; + ShortDescription "Translation-$(LANGUAGE)"; + Description "$(SITE) $(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)"; + + flatURI "$(LANGUAGE)"; + flatDescription "$(SITE) $(RELEASE) Translation-$(LANGUAGE)"; + }; + + deb-src::Sources { + URI "$(COMPONENT)/source/Sources"; + ShortDescription "Sources"; + Description "$(SITE) $(RELEASE)/$(COMPONENT) Sources"; + + flatURI "Sources"; + flatDescription "$(SITE) $(RELEASE) Sources"; + + Optional "false"; + }; +}; + +# Substitution variables + +As seen in the examples, properties can contain placeholders filled in +by the acquire system. The following variables are known; note that +unknown variables have no default value nor are they touched: They are +printed literally. + +* $(SITE): An identifier of the site we access, e.g. + "http://example.org/". +* $(RELEASE): This is usually an archive- or codename, e.g. "stable" or + "stretch". Note that flat-style repositories do not have a archive- + or codename per-se, so the value might very well be just "/" or so. +* $(COMPONENT): as given in the sources.list, e.g. "main", "non-free" or + "universe". Note that flat-style repositories again do not really + have a meaningful value here. +* $(LANGUAGE): Values are all entries (expect "none") of configuration + option Acquire::Languages, e.g. "en", "de" or "de_AT". + +These values are defined both for 'deb' as well as 'deb-src' targets. +'deb' targets additional have the variable: + +* $(ARCHITECTURE): Values are all entries of configuration option + APT::Architectures (potentially modified by sources.list options), + e.g. "amd64", "i386" or "armel". -- cgit v1.2.3-70-g09d2 From d3a869e35503638e3483228fbfc95b7143568ad0 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Wed, 10 Jun 2015 21:24:47 +0200 Subject: store all targets data in IndexTarget struct We still need an API for the targets, so slowly prepare the IndexTargets to let them take this job. Git-Dch: Ignore --- apt-pkg/acquire-item.cc | 15 ++- apt-pkg/acquire-item.h | 31 ++---- apt-pkg/deb/debmetaindex.cc | 114 ++++++++++----------- apt-pkg/init.cc | 12 +-- doc/acquire-additional-files.txt | 29 +++--- test/integration/test-apt-acquire-additional-files | 4 +- 6 files changed, 97 insertions(+), 108 deletions(-) (limited to 'doc') diff --git a/apt-pkg/acquire-item.cc b/apt-pkg/acquire-item.cc index a1357fb15..e92ccc08b 100644 --- a/apt-pkg/acquire-item.cc +++ b/apt-pkg/acquire-item.cc @@ -382,6 +382,15 @@ bool pkgAcqDiffIndex::TransactionState(TransactionStates const state) } /*}}}*/ +// IndexTarget - Constructor /*{{{*/ +IndexTarget::IndexTarget(std::string const &MetaKey, std::string const &ShortDesc, + std::string const &LongDesc, std::string const &URI, bool const IsOptional, + std::map const &Options) : + URI(URI), Description(LongDesc), ShortDesc(ShortDesc), MetaKey(MetaKey), IsOptional(IsOptional), Options(Options) +{ +} + /*}}}*/ + class APT_HIDDEN NoActionItem : public pkgAcquire::Item /*{{{*/ /* The sole purpose of this class is having an item which does nothing to reach its done state to prevent cleanup deleting the mentioned file. @@ -955,7 +964,7 @@ void pkgAcqMetaBase::QueueIndexes(bool const verify) /*{{{*/ if (TransactionManager->MetaIndexParser->Exists((*Target)->MetaKey) == false) { // optional targets that we do not have in the Release file are skipped - if ((*Target)->IsOptional()) + if ((*Target)->IsOptional) continue; Status = StatAuthError; @@ -2388,7 +2397,7 @@ string pkgAcqIndex::Custom600Headers() const if (stat(Final.c_str(),&Buf) == 0) msg += "\nLast-Modified: " + TimeRFC1123(Buf.st_mtime); - if(Target->IsOptional()) + if(Target->IsOptional) msg += "\nFail-Ignore: true"; return msg; @@ -2410,7 +2419,7 @@ void pkgAcqIndex::Failed(string const &Message,pkgAcquire::MethodConfig const * } } - if(Target->IsOptional() && GetExpectedHashes().empty() && Stage == STAGE_DOWNLOAD) + if(Target->IsOptional && GetExpectedHashes().empty() && Stage == STAGE_DOWNLOAD) Status = StatDone; else TransactionManager->AbortTransaction(); diff --git a/apt-pkg/acquire-item.h b/apt-pkg/acquire-item.h index 38a7a8662..a2571e1cd 100644 --- a/apt-pkg/acquire-item.h +++ b/apt-pkg/acquire-item.h @@ -28,6 +28,7 @@ #include #include +#include #ifndef APT_8_CLEANER_HEADERS #include @@ -49,7 +50,7 @@ class pkgSourceList; class IndexTarget; class pkgAcqMetaBase; -class APT_HIDDEN IndexTarget /*{{{*/ +class IndexTarget /*{{{*/ /** \brief Information about an index file. */ { public: @@ -63,30 +64,18 @@ class APT_HIDDEN IndexTarget /*{{{*/ std::string const ShortDesc; /** \brief The key by which this index file should be - * looked up within the meta signature file. - */ + looked up within the meta index file. */ std::string const MetaKey; - virtual bool IsOptional() const { - return false; - } + /** \brief Is it okay if the file isn't found in the meta index */ + bool const IsOptional; - IndexTarget(std::string const &MetaKey, std::string const &ShortDesc, - std::string const &LongDesc, std::string const &URI) : - URI(URI), Description(LongDesc), ShortDesc(ShortDesc), MetaKey(MetaKey) {} -}; - /*}}}*/ -class APT_HIDDEN OptionalIndexTarget : public IndexTarget /*{{{*/ -/** \brief Information about an optional index file. */ -{ - public: - virtual bool IsOptional() const { - return true; - } + /** \brief Target specific options defined by the implementation */ + std::map const Options; - OptionalIndexTarget(std::string const &MetaKey, std::string const &ShortDesc, - std::string const &LongDesc, std::string const &URI) : - IndexTarget(MetaKey, ShortDesc, LongDesc, URI) {} + IndexTarget(std::string const &MetaKey, std::string const &ShortDesc, + std::string const &LongDesc, std::string const &URI, bool const IsOptional, + std::map const &Options); }; /*}}}*/ class pkgAcquire::Item : public WeakPointable /*{{{*/ diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index c35f3b0a4..f038f12f5 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -116,12 +116,12 @@ void foreachTarget(std::string const URI, std::string const Dist, for (std::vector::const_iterator T = targets.begin(); T != targets.end(); ++T) { #define APT_T_CONFIG(X) _config->Find(std::string("APT::Acquire::Targets::deb-src::") + *T + "::" + (X)) - std::string const URI = APT_T_CONFIG(flatArchive ? "flatURI" : "URI"); + std::string const MetaKey = APT_T_CONFIG(flatArchive ? "flatMetaKey" : "MetaKey"); std::string const ShortDesc = APT_T_CONFIG("ShortDescription"); std::string const LongDesc = APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); bool const IsOptional = _config->FindB(std::string("APT::Acquire::Targets::deb-src::") + *T + "::Optional", true); #undef APT_T_CONFIG - if (URI.empty()) + if (MetaKey.empty()) continue; vector const SectionEntries = src->second; @@ -130,23 +130,24 @@ void foreachTarget(std::string const URI, std::string const Dist, { for (vector::const_iterator l = lang.begin(); l != lang.end(); ++l) { - if (*l == "none" && URI.find("$(LANGUAGE)") != std::string::npos) + if (*l == "none" && MetaKey.find("$(LANGUAGE)") != std::string::npos) continue; - struct SubstVar subst[] = { - { "$(SITE)", &Site }, - { "$(RELEASE)", &Release }, - { "$(COMPONENT)", &((*I)->Section) }, - { "$(LANGUAGE)", &(*l) }, - { NULL, NULL } - }; - Call(baseURI, *T, URI, ShortDesc, LongDesc, IsOptional, subst); - - if (URI.find("$(LANGUAGE)") == std::string::npos) + std::map Options; + Options.insert(std::make_pair("SITE", Site)); + Options.insert(std::make_pair("RELEASE", Release)); + Options.insert(std::make_pair("COMPONENT", (*I)->Section)); + Options.insert(std::make_pair("LANGUAGE", *l)); + Options.insert(std::make_pair("ARCHITECTURE", "source")); + Options.insert(std::make_pair("BASE_URI", baseURI)); + Options.insert(std::make_pair("CREATED_BY", *T)); + Call(MetaKey, ShortDesc, LongDesc, IsOptional, Options); + + if (MetaKey.find("$(LANGUAGE)") == std::string::npos) break; } - if (URI.find("$(COMPONENT)") == std::string::npos) + if (MetaKey.find("$(COMPONENT)") == std::string::npos) break; } } @@ -156,12 +157,12 @@ void foreachTarget(std::string const URI, std::string const Dist, for (std::vector::const_iterator T = targets.begin(); T != targets.end(); ++T) { #define APT_T_CONFIG(X) _config->Find(std::string("APT::Acquire::Targets::deb::") + *T + "::" + (X)) - std::string const URI = APT_T_CONFIG(flatArchive ? "flatURI" : "URI"); + std::string const MetaKey = APT_T_CONFIG(flatArchive ? "flatMetaKey" : "MetaKey"); std::string const ShortDesc = APT_T_CONFIG("ShortDescription"); std::string const LongDesc = APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); bool const IsOptional = _config->FindB(std::string("APT::Acquire::Targets::deb::") + *T + "::Optional", true); #undef APT_T_CONFIG - if (URI.empty()) + if (MetaKey.empty()) continue; for (map >::const_iterator a = ArchEntries.begin(); @@ -175,28 +176,28 @@ void foreachTarget(std::string const URI, std::string const Dist, for (vector::const_iterator l = lang.begin(); l != lang.end(); ++l) { - if (*l == "none" && URI.find("$(LANGUAGE)") != std::string::npos) + if (*l == "none" && MetaKey.find("$(LANGUAGE)") != std::string::npos) continue; - struct SubstVar subst[] = { - { "$(SITE)", &Site }, - { "$(RELEASE)", &Release }, - { "$(COMPONENT)", &((*I)->Section) }, - { "$(LANGUAGE)", &(*l) }, - { "$(ARCHITECTURE)", &(a->first) }, - { NULL, NULL } - }; - Call(baseURI, *T, URI, ShortDesc, LongDesc, IsOptional, subst); - - if (URI.find("$(LANGUAGE)") == std::string::npos) + std::map Options; + Options.insert(std::make_pair("SITE", Site)); + Options.insert(std::make_pair("RELEASE", Release)); + Options.insert(std::make_pair("COMPONENT", (*I)->Section)); + Options.insert(std::make_pair("LANGUAGE", *l)); + Options.insert(std::make_pair("ARCHITECTURE", a->first)); + Options.insert(std::make_pair("BASE_URI", baseURI)); + Options.insert(std::make_pair("CREATED_BY", *T)); + Call(MetaKey, ShortDesc, LongDesc, IsOptional, Options); + + if (MetaKey.find("$(LANGUAGE)") == std::string::npos) break; } - if (URI.find("$(COMPONENT)") == std::string::npos) + if (MetaKey.find("$(COMPONENT)") == std::string::npos) break; } - if (URI.find("$(ARCHITECTURE)") == std::string::npos) + if (MetaKey.find("$(ARCHITECTURE)") == std::string::npos) break; } } @@ -207,30 +208,23 @@ struct ComputeIndexTargetsClass { vector * const IndexTargets; - void operator()(std::string const &baseURI, std::string const &/*TargetName*/, - std::string const &URI, std::string const &ShortDesc, std::string const &LongDesc, - bool const IsOptional, struct SubstVar const * const subst) + void operator()(std::string MetaKey, std::string ShortDesc, std::string LongDesc, + bool const IsOptional, std::map Options) { - std::string const name = SubstVar(URI, subst); - IndexTarget * Target; - if (IsOptional == true) - { - Target = new OptionalIndexTarget( - name, - SubstVar(ShortDesc, subst), - SubstVar(LongDesc, subst), - baseURI + name - ); - } - else + for (std::map::const_iterator O = Options.begin(); O != Options.end(); ++O) { - Target = new IndexTarget( - name, - SubstVar(ShortDesc, subst), - SubstVar(LongDesc, subst), - baseURI + name - ); + MetaKey = SubstVar(MetaKey, std::string("$(") + O->first + ")", O->second); + ShortDesc = SubstVar(ShortDesc, std::string("$(") + O->first + ")", O->second); + LongDesc = SubstVar(LongDesc, std::string("$(") + O->first + ")", O->second); } + IndexTarget * Target = new IndexTarget( + MetaKey, + ShortDesc, + LongDesc, + Options.find("BASE_URI")->second + MetaKey, + IsOptional, + Options + ); IndexTargets->push_back(Target); } @@ -256,7 +250,7 @@ bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const // special case for --print-uris vector const * const targets = ComputeIndexTargets(); -#define APT_TARGET(X) IndexTarget("", X, MetaIndexInfo(X), MetaIndexURI(X)) +#define APT_TARGET(X) IndexTarget("", X, MetaIndexInfo(X), MetaIndexURI(X), false, std::map()) pkgAcqMetaBase * const TransactionManager = new pkgAcqMetaClearSig(Owner, APT_TARGET("InRelease"), APT_TARGET("Release"), APT_TARGET("Release.gpg"), targets, iR); @@ -309,33 +303,33 @@ struct GetIndexFilesClass std::string const Release; bool const IsTrusted; - void operator()(std::string const &/*baseURI*/, std::string const &TargetName, - std::string const &/*URI*/, std::string const &/*ShortDesc*/, std::string const &/*LongDesc*/, - bool const /*IsOptional*/, struct SubstVar const * const subst) + void operator()(std::string const &/*URI*/, std::string const &/*ShortDesc*/, std::string const &/*LongDesc*/, + bool const /*IsOptional*/, std::map Options) { + std::string const TargetName = Options.find("CREATED_BY")->second; if (TargetName == "Packages") { Indexes->push_back(new debPackagesIndex( URI, Release, - SubstVar("$(COMPONENT)", subst), + Options.find("COMPONENT")->second, IsTrusted, - SubstVar("$(ARCHITECTURE)", subst) + Options.find("ARCHITECTURE")->second )); } else if (TargetName == "Sources") Indexes->push_back(new debSourcesIndex( URI, Release, - SubstVar("$(COMPONENT)", subst), + Options.find("COMPONENT")->second, IsTrusted )); else if (TargetName == "Translations") Indexes->push_back(new debTranslationsIndex( URI, Release, - SubstVar("$(COMPONENT)", subst), - SubstVar("$(LANGUAGE)", subst) + Options.find("COMPONENT")->second, + Options.find("LANGUAGE")->second )); } diff --git a/apt-pkg/init.cc b/apt-pkg/init.cc index 50ea2b49e..e2239a906 100644 --- a/apt-pkg/init.cc +++ b/apt-pkg/init.cc @@ -101,19 +101,19 @@ bool pkgInitConfig(Configuration &Cnf) // The default user we drop to in the methods Cnf.CndSet("APT::Sandbox::User", "_apt"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::URI", "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::flatURI", "Packages"); + Cnf.CndSet("APT::Acquire::Targets::deb::Packages::MetaKey", "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"); + Cnf.CndSet("APT::Acquire::Targets::deb::Packages::flatMetaKey", "Packages"); Cnf.CndSet("APT::Acquire::Targets::deb::Packages::ShortDescription", "Packages"); Cnf.CndSet("APT::Acquire::Targets::deb::Packages::Description", "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages"); Cnf.CndSet("APT::Acquire::Targets::deb::Packages::flatDescription", "$(SITE) $(RELEASE) Packages"); Cnf.CndSet("APT::Acquire::Targets::deb::Packages::Optional", false); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::URI", "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::flatURI", "$(LANGUAGE)"); + Cnf.CndSet("APT::Acquire::Targets::deb::Translations::MetaKey", "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"); + Cnf.CndSet("APT::Acquire::Targets::deb::Translations::flatMetaKey", "$(LANGUAGE)"); Cnf.CndSet("APT::Acquire::Targets::deb::Translations::ShortDescription", "Translation-$(LANGUAGE)"); Cnf.CndSet("APT::Acquire::Targets::deb::Translations::Description", "$(SITE) $(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)"); Cnf.CndSet("APT::Acquire::Targets::deb::Translations::flatDescription", "$(SITE) $(RELEASE) Translation-$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::URI", "$(COMPONENT)/source/Sources"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::flatURI", "Sources"); + Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::MetaKey", "$(COMPONENT)/source/Sources"); + Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::flatMetaKey", "Sources"); Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::ShortDescription", "Sources"); Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::Description", "$(SITE) $(RELEASE)/$(COMPONENT) Sources"); Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::flatDescription", "$(SITE) $(RELEASE) Sources"); diff --git a/doc/acquire-additional-files.txt b/doc/acquire-additional-files.txt index 5a07c2bec..ab833440b 100644 --- a/doc/acquire-additional-files.txt +++ b/doc/acquire-additional-files.txt @@ -33,11 +33,11 @@ instructing the Acquire system to download the Packages files would look like this (see also apt.conf(5) manpage for configuration file syntax): APT::Acquire::Targets::deb::Packages { - URI "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"; + MetaKey "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"; ShortDescription "Packages"; Description "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages"; - flatURI "Packages"; + flatMetaKey "Packages"; flatDescription "$(SITE) $(RELEASE) Packages"; Optional "false"; @@ -47,18 +47,18 @@ All files which should be downloaded (nicknamed 'Targets') are mentioned below the APT::Acquire::Targets scope. 'deb' is here the type of the sources.list entry the file should be acquired for. The only other supported value is hence 'deb-src'. Beware: You can't specify multiple -types here and you can't download the same URI for multiple types! +types here and you can't download the same MetaKey form multiple types! After the type you can pick any valid and unique string which preferable refers to the file it downloads (In the example we picked 'Packages'). This string is never shown or used. All targets have three main properties you can define: -* URI: The identifier of the file to be downloaded as used in the +* MetaKey: The identifier of the file to be downloaded as used in the Release file. It is also the relative location of the file from the Release file. You can neither download from a different server - entirely (absolute URI) nor should you try to access directories above - the Release file (e.g. "../../"). + entirely (absolute URI) nor access directories above the Release file + (e.g. "../../"). * ShortDescription: Very short string intended to be displayed to the user e.g. while reporting progress. apt will e.g. use this string in the last line to indicate progress of e.g. the download of a specific @@ -69,7 +69,7 @@ All targets have three main properties you can define: progress lines. Additional optional properties: -* flat{URI,Description}: APT supports two types of repositories: +* flat{MetaKey,Description}: APT supports two types of repositories: dists-style repositories which are the default and by far the most common which are named after the fact that the files are in an elaborated directory structure. In contrast a flat-style repositories @@ -96,20 +96,20 @@ look like this: APT::Acquire::Targets { deb::Translations { - URI "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"; + MetaKey "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"; ShortDescription "Translation-$(LANGUAGE)"; Description "$(SITE) $(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)"; - flatURI "$(LANGUAGE)"; + flatMetaKey "$(LANGUAGE)"; flatDescription "$(SITE) $(RELEASE) Translation-$(LANGUAGE)"; }; deb-src::Sources { - URI "$(COMPONENT)/source/Sources"; + MetaKey "$(COMPONENT)/source/Sources"; ShortDescription "Sources"; Description "$(SITE) $(RELEASE)/$(COMPONENT) Sources"; - flatURI "Sources"; + flatMetaKey "Sources"; flatDescription "$(SITE) $(RELEASE) Sources"; Optional "false"; @@ -133,10 +133,7 @@ printed literally. have a meaningful value here. * $(LANGUAGE): Values are all entries (expect "none") of configuration option Acquire::Languages, e.g. "en", "de" or "de_AT". - -These values are defined both for 'deb' as well as 'deb-src' targets. -'deb' targets additional have the variable: - * $(ARCHITECTURE): Values are all entries of configuration option APT::Architectures (potentially modified by sources.list options), - e.g. "amd64", "i386" or "armel". + e.g. "amd64", "i386" or "armel" for the 'deb' type. In type 'deb-src' + this variable has the value "source". diff --git a/test/integration/test-apt-acquire-additional-files b/test/integration/test-apt-acquire-additional-files index 0e59233e9..4a6845f40 100755 --- a/test/integration/test-apt-acquire-additional-files +++ b/test/integration/test-apt-acquire-additional-files @@ -31,7 +31,7 @@ testempty find rootdir/var/lib/apt/lists -name '*Contents*' cat > rootdir/etc/apt/apt.conf.d/content-target.conf < rootdir/etc/apt/apt.conf.d/content-target.conf < Date: Thu, 11 Jun 2015 16:40:45 +0200 Subject: implement 'apt-get files' to access index targets Downloading additional files is only half the job. We still need a way to allow external tools to know where the files are they requested for download given that we don't want them to choose their own location. 'apt-get files' is our answer to this showing by default in a deb822 format information about each IndexTarget with the potential to filter the records based on lines and an option to change the output format. The command serves also as an example on how to get to this information via libapt. --- apt-pkg/deb/debmetaindex.cc | 17 +++-- apt-pkg/indexfile.cc | 18 ++++- apt-pkg/indexfile.h | 3 + apt-private/private-cmndline.cc | 4 + cmdline/apt-get.cc | 87 ++++++++++++++++++++++ doc/acquire-additional-files.txt | 87 +++++++++++++++++++++- doc/apt-get.8.xml | 15 ++++ test/integration/test-apt-acquire-additional-files | 2 + 8 files changed, 223 insertions(+), 10 deletions(-) (limited to 'doc') diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index c05e7cdae..8e4c2be2d 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -136,11 +136,14 @@ void foreachTarget(std::string const URI, std::string const Dist, std::map Options; Options.insert(std::make_pair("SITE", Site)); Options.insert(std::make_pair("RELEASE", Release)); - Options.insert(std::make_pair("COMPONENT", (*I)->Section)); - Options.insert(std::make_pair("LANGUAGE", *l)); + if (MetaKey.find("$(COMPONENT)") != std::string::npos) + Options.insert(std::make_pair("COMPONENT", (*I)->Section)); + if (MetaKey.find("$(LANGUAGE)") != std::string::npos) + Options.insert(std::make_pair("LANGUAGE", *l)); Options.insert(std::make_pair("ARCHITECTURE", "source")); Options.insert(std::make_pair("BASE_URI", baseURI)); Options.insert(std::make_pair("REPO_URI", URI)); + Options.insert(std::make_pair("TARGET_OF", "deb-src")); Options.insert(std::make_pair("CREATED_BY", *T)); Call(MetaKey, ShortDesc, LongDesc, IsOptional, Options); @@ -183,11 +186,15 @@ void foreachTarget(std::string const URI, std::string const Dist, std::map Options; Options.insert(std::make_pair("SITE", Site)); Options.insert(std::make_pair("RELEASE", Release)); - Options.insert(std::make_pair("COMPONENT", (*I)->Section)); - Options.insert(std::make_pair("LANGUAGE", *l)); - Options.insert(std::make_pair("ARCHITECTURE", a->first)); + if (MetaKey.find("$(COMPONENT)") != std::string::npos) + Options.insert(std::make_pair("COMPONENT", (*I)->Section)); + if (MetaKey.find("$(LANGUAGE)") != std::string::npos) + Options.insert(std::make_pair("LANGUAGE", *l)); + if (MetaKey.find("$(ARCHITECTURE)") != std::string::npos) + Options.insert(std::make_pair("ARCHITECTURE", a->first)); Options.insert(std::make_pair("BASE_URI", baseURI)); Options.insert(std::make_pair("REPO_URI", URI)); + Options.insert(std::make_pair("TARGET_OF", "deb")); Options.insert(std::make_pair("CREATED_BY", *T)); Call(MetaKey, ShortDesc, LongDesc, IsOptional, Options); diff --git a/apt-pkg/indexfile.cc b/apt-pkg/indexfile.cc index 72d35ddcc..33fb48e35 100644 --- a/apt-pkg/indexfile.cc +++ b/apt-pkg/indexfile.cc @@ -133,8 +133,10 @@ std::string IndexTarget::Option(OptionKeys const EnumKey) const /*{{{*/ APT_CASE(ARCHITECTURE); APT_CASE(BASE_URI); APT_CASE(REPO_URI); + APT_CASE(TARGET_OF); APT_CASE(CREATED_BY); #undef APT_CASE + case FILENAME: return _config->FindDir("Dir::State::lists") + URItoFileName(URI); } std::map::const_iterator const M = Options.find(Key); if (M == Options.end()) @@ -142,6 +144,20 @@ std::string IndexTarget::Option(OptionKeys const EnumKey) const /*{{{*/ return M->second; } /*}}}*/ +std::string IndexTarget::Format(std::string format) const /*{{{*/ +{ + for (std::map::const_iterator O = Options.begin(); O != Options.end(); ++O) + { + format = SubstVar(format, std::string("$(") + O->first + ")", O->second); + } + format = SubstVar(format, "$(METAKEY)", MetaKey); + format = SubstVar(format, "$(SHORTDESC)", ShortDesc); + format = SubstVar(format, "$(DESCRIPTION)", Description); + format = SubstVar(format, "$(URI)", URI); + format = SubstVar(format, "$(FILENAME)", Option(IndexTarget::FILENAME)); + return format; +} + /*}}}*/ pkgIndexTargetFile::pkgIndexTargetFile(IndexTarget const &Target, bool const Trusted) :/*{{{*/ pkgIndexFile(Trusted), Target(Target) @@ -162,7 +178,7 @@ std::string pkgIndexTargetFile::Describe(bool Short) const /*{{{*/ /*}}}*/ std::string pkgIndexTargetFile::IndexFileName() const /*{{{*/ { - std::string const s =_config->FindDir("Dir::State::lists") + URItoFileName(Target.URI); + std::string const s = Target.Option(IndexTarget::FILENAME); if (FileExists(s)) return s; diff --git a/apt-pkg/indexfile.h b/apt-pkg/indexfile.h index c38cf0bf0..220c415ac 100644 --- a/apt-pkg/indexfile.h +++ b/apt-pkg/indexfile.h @@ -79,8 +79,11 @@ class IndexTarget /*{{{*/ BASE_URI, REPO_URI, CREATED_BY, + TARGET_OF, + FILENAME, }; std::string Option(OptionKeys const Key) const; + std::string Format(std::string format) const; }; /*}}}*/ diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index 41aab81f6..458051bf2 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -163,6 +163,10 @@ static bool addArgumentsAPTGet(std::vector &Args, char const // once sbuild is fixed, this option can be removed addArg('f', "fix-broken", "APT::Get::Fix-Broken", 0); } + else if (CmdMatches("files")) + { + addArg(0,"format","APT::Get::Files::Format", CommandLine::HasArg); + } else if (CmdMatches("clean", "autoclean", "check", "download", "changelog") || CmdMatches("markauto", "unmarkauto")) // deprecated commands ; diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc index c1f78523c..0fff2db58 100644 --- a/cmdline/apt-get.cc +++ b/cmdline/apt-get.cc @@ -86,6 +86,7 @@ #include #include #include +#include #include #include #include @@ -1602,6 +1603,91 @@ static bool DoChangelog(CommandLine &CmdL) return true; } /*}}}*/ +// DoFiles - Lists all IndexTargets /*{{{*/ +static std::string format_key(std::string key) +{ + // deb822 is case-insensitive, but the human eye prefers candy + std::transform(key.begin(), key.end(), key.begin(), ::tolower); + key[0] = ::toupper(key[0]); + size_t found = key.find("_uri"); + if (found != std::string::npos) + key.replace(found, 4, "-URI"); + while ((found = key.find('_')) != std::string::npos) + { + key[found] = '-'; + key[found + 1] = ::toupper(key[found + 1]); + } + return key; +} +static bool DoFiles(CommandLine &CmdL) +{ + pkgCacheFile CacheFile; + pkgSourceList *SrcList = CacheFile.GetSourceList(); + + if (SrcList == NULL) + return false; + + std::string const Format = _config->Find("APT::Get::Files::Format"); + bool Filtered = CmdL.FileSize() > 1; + for (pkgSourceList::const_iterator S = SrcList->begin(); S != SrcList->end(); ++S) + { + std::vector const targets = (*S)->GetIndexTargets(); + std::map AddOptions; + AddOptions.insert(std::make_pair("TRUSTED", ((*S)->IsTrusted() ? "yes" : "no"))); + + for (std::vector::const_iterator T = targets.begin(); T != targets.end(); ++T) + { + std::ostringstream stanza; + if (Filtered || Format.empty()) + { + stanza << "MetaKey: " << T->MetaKey << "\n" + << "ShortDesc: " << T->ShortDesc << "\n" + << "Description: " << T->Description << "\n" + << "URI: " << T->URI << "\n" + << "Filename: " << T->Option(IndexTarget::FILENAME) << "\n" + << "Optional: " << (T->IsOptional ? "yes" : "no") << "\n"; + for (std::map::const_iterator O = AddOptions.begin(); O != AddOptions.end(); ++O) + stanza << format_key(O->first) << ": " << O->second << "\n"; + for (std::map::const_iterator O = T->Options.begin(); O != T->Options.end(); ++O) + stanza << format_key(O->first) << ": " << O->second << "\n"; + stanza << "\n"; + + if (Filtered) + { + // that is a bit crude, but good enough for now + bool found = true; + std::string haystack = std::string("\n") + stanza.str() + "\n"; + std::transform(haystack.begin(), haystack.end(), haystack.begin(), ::tolower); + size_t const filesize = CmdL.FileSize() - 1; + for (size_t i = 0; i != filesize; ++i) + { + std::string needle = std::string("\n") + CmdL.FileList[i + 1] + "\n"; + std::transform(needle.begin(), needle.end(), needle.begin(), ::tolower); + if (haystack.find(needle) != std::string::npos) + continue; + found = false; + break; + } + if (found == false) + continue; + } + } + + if (Format.empty()) + cout << stanza.str(); + else + { + std::string out = T->Format(Format); + for (std::map::const_iterator O = AddOptions.begin(); O != AddOptions.end(); ++O) + out = SubstVar(out, std::string("$(") + O->first + ")", O->second); + cout << out << std::endl; + } + } + } + + return true; +} + /*}}}*/ // ShowHelp - Show a help screen /*{{{*/ // --------------------------------------------------------------------- /* */ @@ -1716,6 +1802,7 @@ int main(int argc,const char *argv[]) /*{{{*/ {"source",&DoSource}, {"download",&DoDownload}, {"changelog",&DoChangelog}, + {"files",&DoFiles}, {"moo",&DoMoo}, {"help",&ShowHelp}, {0,0}}; diff --git a/doc/acquire-additional-files.txt b/doc/acquire-additional-files.txt index ab833440b..a47dac2d4 100644 --- a/doc/acquire-additional-files.txt +++ b/doc/acquire-additional-files.txt @@ -47,11 +47,13 @@ All files which should be downloaded (nicknamed 'Targets') are mentioned below the APT::Acquire::Targets scope. 'deb' is here the type of the sources.list entry the file should be acquired for. The only other supported value is hence 'deb-src'. Beware: You can't specify multiple -types here and you can't download the same MetaKey form multiple types! +types here and you can't download the same (evaluated) MetaKey from +multiple types! After the type you can pick any valid and unique string which preferable refers to the file it downloads (In the example we picked 'Packages'). -This string is never shown or used. +This string is used as identifier for the target class and accessible as +'Created-By' e.g. in the "apt-get files" output as detailed below. All targets have three main properties you can define: * MetaKey: The identifier of the file to be downloaded as used in the @@ -123,11 +125,12 @@ by the acquire system. The following variables are known; note that unknown variables have no default value nor are they touched: They are printed literally. -* $(SITE): An identifier of the site we access, e.g. - "http://example.org/". +* $(SITE): An identifier of the site we access as seen in sources.list, + e.g. "http://example.org/debian" or "file:/path/to/a/repository". * $(RELEASE): This is usually an archive- or codename, e.g. "stable" or "stretch". Note that flat-style repositories do not have a archive- or codename per-se, so the value might very well be just "/" or so. + Again, as seen in the sources.list. * $(COMPONENT): as given in the sources.list, e.g. "main", "non-free" or "universe". Note that flat-style repositories again do not really have a meaningful value here. @@ -137,3 +140,79 @@ printed literally. APT::Architectures (potentially modified by sources.list options), e.g. "amd64", "i386" or "armel" for the 'deb' type. In type 'deb-src' this variable has the value "source". + +Note that while more variables might exist in the implementation, these +are to be considered undefined and their usage strongly discouraged. If +you have a need for another variable contact us instead. + +# Accessing files + +Do NOT hardcode specific file locations, names or compression types in +your application! You will notice that the configuration options give +you no choice over where the downloaded files will be stored. This is by +design so multiple applications can download and use the same file +rather than each and every one of them potentially downloads and uses +its own copy somewhere on disk. + +"apt-get files" can be used to get the location as well as other +information about all files downloaded (aka: you will see Packages, +Sources and Translation-* files here as well). Provide a line of the +default output format as parameter to filter out all entries which do +not have such a line. With --format, you can further more define your +own output style. The variables are what you see in the output, just all +uppercase and wrapped in $(), as in the configuration file. + +To get all the filenames of all Translation-en files you can e.g. call: + apt-get files --format '$(FILENAME)' "Created-By: Translations" "Language: en" + +Accessing this information via libapt is done by reading the +sources.lists (pkgSourceList), iterating over the metaIndex objects this +creates and calling GetIndexTargets() on them. See the sourcecode of +"apt-get files" for a complete example. + +Remarks on the available fields: +* MetaKey, ShortDesc, Description, Site, Release: as defined + by the configuration and described further above. +* Created-By: configuration entity responsible for this target +* Target-Of: type of the sources.list entry +* URI, Repo-URI: avoid using. Contains potentially username/password. + Prefer 'Site', especially for display. +* Filename: The mentioned file doesn't need to exist, e.g. because the + file wasn't downloaded yet – or it does exist compressed. libapt users + are encouraged to use FileFd to open such files as it can handle + decompression transparently. +* Trusted: As of the last 'apt-get update' call denoting if e.g. apt-get + would print the unauthenticated warning while installing packages + mentioned in this file (example assumes Packages file) [Not really + a property of the target itself, but of the metaIndex]. +* Optional: Decodes the option of the same name from the configuration. + Note that it is using 'yes' and 'no' instead of 'true' and 'false'. +* Language, Architecture, Component: as defined further above, but with + the catch that they might be missing if they don't effect the target + (aka: They weren't used while evaluating the MetaKey template). + +Again, additional fields might be visible in certain implementation, but +you should avoid using them and instead talk to us about a portable +implementation. + +# Multiple application requiring the same files + +It is highly encouraged that applications talk to each other and to us +about which files they require. It is usually best to have a common +package ship the configuration needed to get the files, but specific +needs might require specific solutions. Again: talk to us. + +# Acquiring files not mentioned in the Release file + +You can't. This is by design as these files couldn't be verified to not +be modified in transit, corrupted by the download process or simple if +they are present at all on the server, which would require apt to probe +for them. APT did this in the past for legacy reasons, we do not intend +to go back to these dark times. + +This is also why you can't request files from a different server. It +would have the additional problem that this server might not even be +accessible (e.g. proxy settings) or that local sources (file:/, cdrom:/) +start requesting online files… + +In other words: We would be opening Pandora's box. diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml index a372a0d30..da077afa7 100644 --- a/doc/apt-get.8.xml +++ b/doc/apt-get.8.xml @@ -243,6 +243,21 @@ + + Displays by default a deb822 formatted listing of + information about all data files apt-get + update would download. Supports a + option to modify the output format as + well as accepts lines of the default output to filter the records + by. The command is mainly used as an interface for external tools + working with APT to get information as well as filenames for + downloaded files so they can use them as well instead of + downloading them again on their own. Detailed documentation is + omitted here and can instead be found in the source tree in + doc/acquire-additional-files.txt. + + + diff --git a/test/integration/test-apt-acquire-additional-files b/test/integration/test-apt-acquire-additional-files index 4a6845f40..971cfac2e 100755 --- a/test/integration/test-apt-acquire-additional-files +++ b/test/integration/test-apt-acquire-additional-files @@ -48,6 +48,7 @@ Get:1 http://localhost:8080 unstable/main amd64 Contents [$(stat -c%s aptarchive Reading package lists..." aptget update testequal 'rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64' find rootdir/var/lib/apt/lists -name '*Contents*' +testequal "$(readlink -f ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64)" aptget files --format '$(FILENAME)' 'Created-By: Contents' testsuccess cmp 'rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64' 'aptarchive/dists/unstable/main/Contents-amd64' # no automatic uncompress based on the name please, @@ -72,6 +73,7 @@ Get:1 http://localhost:8080 unstable/main amd64 Contents.gz [$(stat -c%s aptarch Reading package lists..." aptget update testequal 'rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz' find rootdir/var/lib/apt/lists -name '*Contents*' +testequal "$(readlink -f ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz)" aptget files --format '$(FILENAME)' 'Created-By: Contents' testsuccess cmp 'rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz' 'aptarchive/dists/unstable/main/Contents-amd64.gz' rm -f rootdir/etc/apt/apt.conf.d/content-target.conf -- cgit v1.2.3-70-g09d2 From 3fd89e62e985c89b1f9a545ab72c20987b756aff Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Fri, 12 Jun 2015 12:06:29 +0200 Subject: implement default apt-get file --release-info mode Selecting targets based on the Release they belong to isn't to unrealistic. In fact, it is assumed to be the most used case so it is made the default especially as this allows to bundle another thing we have to be careful with: Filenames and only showing targets we have acquired. Closes: 752702 --- apt-pkg/deb/debmetaindex.cc | 4 +- apt-pkg/deb/debmetaindex.h | 2 +- apt-pkg/indexfile.cc | 12 ++++ apt-pkg/indexfile.h | 1 + apt-pkg/metaindex.cc | 2 +- apt-pkg/metaindex.h | 2 +- apt-pkg/pkgcachegen.cc | 4 +- apt-private/private-cmndline.cc | 1 + cmdline/apt-get.cc | 26 ++++++++- doc/acquire-additional-files.txt | 65 +++++++++++++--------- test/integration/test-apt-acquire-additional-files | 6 ++ 11 files changed, 88 insertions(+), 37 deletions(-) (limited to 'doc') diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index 994b95849..b328fcea8 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -422,7 +422,7 @@ bool debReleaseIndex::Merge(pkgCacheGenerator &Gen,OpProgress * /*Prog*/) const/ } /*}}}*/ // ReleaseIndex::FindInCache - Find this index /*{{{*/ -pkgCache::RlsFileIterator debReleaseIndex::FindInCache(pkgCache &Cache) const +pkgCache::RlsFileIterator debReleaseIndex::FindInCache(pkgCache &Cache, bool const ModifyCheck) const { std::string ReleaseFile; bool const releaseExists = ReleaseFileName(this, ReleaseFile); @@ -434,7 +434,7 @@ pkgCache::RlsFileIterator debReleaseIndex::FindInCache(pkgCache &Cache) const continue; // empty means the file does not exist by "design" - if (releaseExists == false && File->Size == 0) + if (ModifyCheck == false || (releaseExists == false && File->Size == 0)) return File; struct stat St; diff --git a/apt-pkg/deb/debmetaindex.h b/apt-pkg/deb/debmetaindex.h index 7e9942710..b448ecc53 100644 --- a/apt-pkg/deb/debmetaindex.h +++ b/apt-pkg/deb/debmetaindex.h @@ -51,7 +51,7 @@ class APT_HIDDEN debReleaseIndex : public metaIndex { virtual std::vector GetIndexTargets() const; virtual std::string Describe() const; - virtual pkgCache::RlsFileIterator FindInCache(pkgCache &Cache) const; + virtual pkgCache::RlsFileIterator FindInCache(pkgCache &Cache, bool const ModifyCheck) const; virtual bool Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const; std::string MetaIndexInfo(const char *Type) const; diff --git a/apt-pkg/indexfile.cc b/apt-pkg/indexfile.cc index 33fb48e35..605bbeb47 100644 --- a/apt-pkg/indexfile.cc +++ b/apt-pkg/indexfile.cc @@ -137,6 +137,18 @@ std::string IndexTarget::Option(OptionKeys const EnumKey) const /*{{{*/ APT_CASE(CREATED_BY); #undef APT_CASE case FILENAME: return _config->FindDir("Dir::State::lists") + URItoFileName(URI); + case EXISTING_FILENAME: + std::string const filename = Option(FILENAME); + std::vector const types = APT::Configuration::getCompressionTypes(); + for (std::vector::const_iterator t = types.begin(); t != types.end(); ++t) + { + if (t->empty()) + continue; + std::string const file = (*t == "uncompressed") ? filename : (filename + "." + *t); + if (FileExists(file)) + return file; + } + return ""; } std::map::const_iterator const M = Options.find(Key); if (M == Options.end()) diff --git a/apt-pkg/indexfile.h b/apt-pkg/indexfile.h index 220c415ac..042e5c2f7 100644 --- a/apt-pkg/indexfile.h +++ b/apt-pkg/indexfile.h @@ -81,6 +81,7 @@ class IndexTarget /*{{{*/ CREATED_BY, TARGET_OF, FILENAME, + EXISTING_FILENAME, }; std::string Option(OptionKeys const Key) const; std::string Format(std::string format) const; diff --git a/apt-pkg/metaindex.cc b/apt-pkg/metaindex.cc index 35ab6c53b..3c1b696bd 100644 --- a/apt-pkg/metaindex.cc +++ b/apt-pkg/metaindex.cc @@ -28,7 +28,7 @@ std::string metaIndex::Describe() const return "Release"; } -pkgCache::RlsFileIterator metaIndex::FindInCache(pkgCache &Cache) const +pkgCache::RlsFileIterator metaIndex::FindInCache(pkgCache &Cache, bool const) const { return pkgCache::RlsFileIterator(Cache); } diff --git a/apt-pkg/metaindex.h b/apt-pkg/metaindex.h index 20c879a89..e1810fb27 100644 --- a/apt-pkg/metaindex.h +++ b/apt-pkg/metaindex.h @@ -54,7 +54,7 @@ class metaIndex virtual bool IsTrusted() const = 0; virtual std::string Describe() const; - virtual pkgCache::RlsFileIterator FindInCache(pkgCache &Cache) const; + virtual pkgCache::RlsFileIterator FindInCache(pkgCache &Cache, bool const ModifyCheck) const; virtual bool Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const; metaIndex(std::string const &URI, std::string const &Dist, diff --git a/apt-pkg/pkgcachegen.cc b/apt-pkg/pkgcachegen.cc index 2d0e3960d..9db4ef41f 100644 --- a/apt-pkg/pkgcachegen.cc +++ b/apt-pkg/pkgcachegen.cc @@ -1224,7 +1224,7 @@ static bool CheckValidity(const string &CacheFile, { if (Debug == true) std::clog << "Checking RlsFile " << (*i)->Describe() << ": "; - pkgCache::RlsFileIterator const RlsFile = (*i)->FindInCache(Cache); + pkgCache::RlsFileIterator const RlsFile = (*i)->FindInCache(Cache, true); if (RlsFile.end() == true) { if (Debug == true) @@ -1348,7 +1348,7 @@ static bool BuildCache(pkgCacheGenerator &Gen, { for (pkgSourceList::const_iterator i = List->begin(); i != List->end(); ++i) { - if ((*i)->FindInCache(Gen.GetCache()).end() == false) + if ((*i)->FindInCache(Gen.GetCache(), false).end() == false) { _error->Warning("Duplicate sources.list entry %s", (*i)->Describe().c_str()); diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index 458051bf2..11e88b1e7 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -166,6 +166,7 @@ static bool addArgumentsAPTGet(std::vector &Args, char const else if (CmdMatches("files")) { addArg(0,"format","APT::Get::Files::Format", CommandLine::HasArg); + addArg(0,"release-info","APT::Get::Files::ReleaseInfo", 0); } else if (CmdMatches("clean", "autoclean", "check", "download", "changelog") || CmdMatches("markauto", "unmarkauto")) // deprecated commands diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc index 0fff2db58..a4cd3c377 100644 --- a/cmdline/apt-get.cc +++ b/cmdline/apt-get.cc @@ -1628,15 +1628,34 @@ static bool DoFiles(CommandLine &CmdL) return false; std::string const Format = _config->Find("APT::Get::Files::Format"); + bool const ReleaseInfo = _config->FindB("APT::Get::Files::ReleaseInfo", true); bool Filtered = CmdL.FileSize() > 1; for (pkgSourceList::const_iterator S = SrcList->begin(); S != SrcList->end(); ++S) { std::vector const targets = (*S)->GetIndexTargets(); std::map AddOptions; - AddOptions.insert(std::make_pair("TRUSTED", ((*S)->IsTrusted() ? "yes" : "no"))); + if (ReleaseInfo) + { + AddOptions.insert(std::make_pair("TRUSTED", ((*S)->IsTrusted() ? "yes" : "no"))); + pkgCache &Cache = *CacheFile.GetPkgCache(); + pkgCache::RlsFileIterator const RlsFile = (*S)->FindInCache(Cache, false); + if (RlsFile.end()) + continue; +#define APT_RELEASE(X,Y) if (RlsFile.Y() != NULL) AddOptions.insert(std::make_pair(X, RlsFile.Y())) + APT_RELEASE("CODENAME", Codename); + APT_RELEASE("SUITE", Archive); + APT_RELEASE("VERSION", Version); + APT_RELEASE("ORIGIN", Origin); + APT_RELEASE("LABEL", Label); +#undef APT_RELEASE + } for (std::vector::const_iterator T = targets.begin(); T != targets.end(); ++T) { + std::string filename = T->Option(ReleaseInfo ? IndexTarget::EXISTING_FILENAME : IndexTarget::FILENAME); + if (filename.empty()) + continue; + std::ostringstream stanza; if (Filtered || Format.empty()) { @@ -1644,7 +1663,7 @@ static bool DoFiles(CommandLine &CmdL) << "ShortDesc: " << T->ShortDesc << "\n" << "Description: " << T->Description << "\n" << "URI: " << T->URI << "\n" - << "Filename: " << T->Option(IndexTarget::FILENAME) << "\n" + << "Filename: " << filename << "\n" << "Optional: " << (T->IsOptional ? "yes" : "no") << "\n"; for (std::map::const_iterator O = AddOptions.begin(); O != AddOptions.end(); ++O) stanza << format_key(O->first) << ": " << O->second << "\n"; @@ -1677,7 +1696,8 @@ static bool DoFiles(CommandLine &CmdL) cout << stanza.str(); else { - std::string out = T->Format(Format); + std::string out = SubstVar(Format, "$(FILENAME)", filename); + out = T->Format(out); for (std::map::const_iterator O = AddOptions.begin(); O != AddOptions.end(); ++O) out = SubstVar(out, std::string("$(") + O->first + ")", O->second); cout << out << std::endl; diff --git a/doc/acquire-additional-files.txt b/doc/acquire-additional-files.txt index a47dac2d4..f9a16318d 100644 --- a/doc/acquire-additional-files.txt +++ b/doc/acquire-additional-files.txt @@ -18,12 +18,6 @@ might want to use and would therefore need to be downloaded as well This file describes the configuration scheme such a frontend can use to instruct the Acquire system to download those additional files. -Note that you can't download files from other sources. It must be files -in the same repository and below the Release file. The Release file must -also contain hashes for the file itself as well as for the compressed -file if wanted, otherwise a download isn't even tried! - - # The Configuration Stanza The Acquire system uses the same configuration settings to implement the @@ -81,15 +75,17 @@ Additional optional properties: * Optional: The default value is 'true' and should be kept at this value. If enabled the acquire system will skip the download if the file isn't mentioned in the Release file. Otherwise this is treated as - a hard error and the update process fails. + a hard error and the update process fails. Note that failures while + downloading (e.g. 404 or hash verification errors) are failures, + regardless of this setting. -Note that the acquire system will automatically choose to download -a compressed file if it is available and uncompress it for you, just as -it will also use pdiff patching if provided by the repository and -enabled by the user. You only have to ensure that the Release file -contains the information about the compressed files/pdiffs to make this -happen. NO properties have to be set to enable this. +The acquire system will automatically choose to download a compressed +file if it is available and uncompress it for you, just as it will also +use pdiff patching if provided by the repository and enabled by the +user. You only have to ensure that the Release file contains the +information about the compressed files/pdiffs to make this happen. +NO properties have to be set to enable this. # More examples @@ -123,10 +119,12 @@ APT::Acquire::Targets { As seen in the examples, properties can contain placeholders filled in by the acquire system. The following variables are known; note that unknown variables have no default value nor are they touched: They are -printed literally. +printed as-is. * $(SITE): An identifier of the site we access as seen in sources.list, - e.g. "http://example.org/debian" or "file:/path/to/a/repository". + e.g. "http://example.org/debian" or "file:/path/to/a/repository". You + can't use this field in {,flat}MetaKey, it is for description proposes + only. * $(RELEASE): This is usually an archive- or codename, e.g. "stable" or "stretch". Note that flat-style repositories do not have a archive- or codename per-se, so the value might very well be just "/" or so. @@ -143,7 +141,7 @@ printed literally. Note that while more variables might exist in the implementation, these are to be considered undefined and their usage strongly discouraged. If -you have a need for another variable contact us instead. +you have a need for other variables contact us. # Accessing files @@ -170,29 +168,42 @@ sources.lists (pkgSourceList), iterating over the metaIndex objects this creates and calling GetIndexTargets() on them. See the sourcecode of "apt-get files" for a complete example. -Remarks on the available fields: +Note that by default targets are not listed if they weren't downloaded. +If you want to see all targets, you can use the --no-release-info, which +also removes the Codename, Suite, Version, Origin, Label and Trusted +fields from the output as these also display data which needs to be +downloaded first and could hence be inaccurate [on the pro-side: This +mode is faster as it doesn't require a valid binary cache to operate]. +The most notable difference perhaps is in the Filename field through: By +default it indicates an existing file, potentially compressed (Hint: +libapt users can use FileFd to open compressed files transparently). In +the --no-release-info mode the indicated file doesn't need to exist and +it will always refer to an uncompressed file, even if the index would be +(or is) stored compressed. + +Remarks on fields only available in (default) --release-info mode: +* Trusted: Denotes with a 'yes' or 'no' if the data in this file is + authenticated by a trustchain rooted in a trusted gpg key. You should + be careful with untrusted data and warn the user if you use it. +* Codename, Suite, Version, Origin and Label are fields from the Release + file, are only present if they are present in the Release file and + contain the same data. + +Remarks on other available fields: * MetaKey, ShortDesc, Description, Site, Release: as defined by the configuration and described further above. * Created-By: configuration entity responsible for this target * Target-Of: type of the sources.list entry * URI, Repo-URI: avoid using. Contains potentially username/password. Prefer 'Site', especially for display. -* Filename: The mentioned file doesn't need to exist, e.g. because the - file wasn't downloaded yet – or it does exist compressed. libapt users - are encouraged to use FileFd to open such files as it can handle - decompression transparently. -* Trusted: As of the last 'apt-get update' call denoting if e.g. apt-get - would print the unauthenticated warning while installing packages - mentioned in this file (example assumes Packages file) [Not really - a property of the target itself, but of the metaIndex]. * Optional: Decodes the option of the same name from the configuration. Note that it is using 'yes' and 'no' instead of 'true' and 'false'. * Language, Architecture, Component: as defined further above, but with the catch that they might be missing if they don't effect the target (aka: They weren't used while evaluating the MetaKey template). -Again, additional fields might be visible in certain implementation, but -you should avoid using them and instead talk to us about a portable +Again, additional fields might be visible in certain implementations, +but you should avoid using them and instead talk to us about a portable implementation. # Multiple application requiring the same files diff --git a/test/integration/test-apt-acquire-additional-files b/test/integration/test-apt-acquire-additional-files index 971cfac2e..ffb9f3135 100755 --- a/test/integration/test-apt-acquire-additional-files +++ b/test/integration/test-apt-acquire-additional-files @@ -37,6 +37,12 @@ APT::Acquire::Targets::deb::Contents { }; EOF +testequal "$(readlink -f ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64)" aptget files --no-release-info --format '$(FILENAME)' 'Created-By: Contents' +testempty aptget files --format '$(FILENAME)' 'Created-By: Contents' +# lets fake the existence of a compressed Contents file +touch ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz +testequal "$(readlink -f ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz)" aptget files --format '$(FILENAME)' 'Created-By: Contents' + testequal "'http://localhost:8080/dists/unstable/InRelease' localhost:8080_dists_unstable_InRelease 0 'http://localhost:8080/dists/unstable/main/source/Sources.bz2' localhost:8080_dists_unstable_main_source_Sources 0 'http://localhost:8080/dists/unstable/main/binary-amd64/Packages.bz2' localhost:8080_dists_unstable_main_binary-amd64_Packages 0 -- cgit v1.2.3-70-g09d2 From d56e2917f27a722b54685de13aeb1bb7592fc61b Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Sat, 13 Jun 2015 11:13:45 +0200 Subject: provide a public interface for acquiring changelogs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Provided is a specialized acquire item which given a version can figure out the correct URI to try by itself and if not provides an error message alongside with static methods to get just the URI it would try to download if it should just be displayed or similar such. The URI is constructed as follows: Release files can provide an URI template in the "Changelogs" field, otherwise we lookup a configuration item based on the "Label" or "Origin" of the Release file to get a (hopefully known) default value for now. This template should contain the string CHANGEPATH which is replaced with the information about the version we want the changelog for (e.g. main/a/apt/apt_1.1). This middleway was choosen as this path part was consistent over the three known implementations (+1 defunct), while the rest of the URI varies widely between them. The benefit of this construct is that it is now easy to get changelogs for Debian packages on Ubuntu and vice versa – even at the moment where the Changelogs field is present nowhere. Strictly better than what apt-get had before as it would even fail to get changelogs from security… Now it will notice that security identifies as Origin: Debian and pick this setting (assuming again that no Changelogs field exists). If on the other hand security would ship its changelogs in a different location we could set it via the Label option overruling Origin. Closes: 687147, 739854, 784027, 787190 --- apt-pkg/acquire-item.cc | 211 +++++++++++++++++++++ apt-pkg/acquire-item.h | 114 +++++++++++ apt-pkg/init.cc | 4 + cmdline/apt-get.cc | 204 ++++---------------- doc/apt-get.8.xml | 16 +- doc/apt.conf.5.xml | 27 +++ doc/examples/configure-index | 11 +- test/integration/framework | 24 ++- test/integration/test-apt-get-changelog | 104 +++++++--- .../test-bug-722207-print-uris-even-if-very-quiet | 3 +- test/integration/test-bug-738785-switch-protocol | 7 +- vendor/ubuntu/apt.conf-01-vendor-ubuntu | 6 - 12 files changed, 507 insertions(+), 224 deletions(-) (limited to 'doc') diff --git a/apt-pkg/acquire-item.cc b/apt-pkg/acquire-item.cc index d6e9ccbe0..4bf4e62f8 100644 --- a/apt-pkg/acquire-item.cc +++ b/apt-pkg/acquire-item.cc @@ -31,6 +31,7 @@ #include #include #include +#include #include #include @@ -2836,6 +2837,216 @@ std::string pkgAcqArchive::ShortDesc() const /*{{{*/ } /*}}}*/ +// AcqChangelog::pkgAcqChangelog - Constructors /*{{{*/ +pkgAcqChangelog::pkgAcqChangelog(pkgAcquire * const Owner, pkgCache::VerIterator const &Ver, + std::string const &DestDir, std::string const &DestFilename) : + pkgAcquire::Item(Owner), d(NULL), SrcName(Ver.SourcePkgName()), SrcVersion(Ver.SourceVerStr()) +{ + Desc.URI = URI(Ver); + Init(DestDir, DestFilename); +} +// some parameters are char* here as they come likely from char* interfaces – which can also return NULL +pkgAcqChangelog::pkgAcqChangelog(pkgAcquire * const Owner, pkgCache::RlsFileIterator const &RlsFile, + char const * const Component, char const * const SrcName, char const * const SrcVersion, + const string &DestDir, const string &DestFilename) : + pkgAcquire::Item(Owner), d(NULL), SrcName(SrcName), SrcVersion(SrcVersion) +{ + Desc.URI = URI(RlsFile, Component, SrcName, SrcVersion); + Init(DestDir, DestFilename); +} +pkgAcqChangelog::pkgAcqChangelog(pkgAcquire * const Owner, + std::string const &URI, char const * const SrcName, char const * const SrcVersion, + const string &DestDir, const string &DestFilename) : + pkgAcquire::Item(Owner), d(NULL), SrcName(SrcName), SrcVersion(SrcVersion) +{ + Desc.URI = URI; + Init(DestDir, DestFilename); +} +void pkgAcqChangelog::Init(std::string const &DestDir, std::string const &DestFilename) +{ + if (Desc.URI.empty()) + { + Status = StatError; + // TRANSLATOR: %s=%s is sourcename=sourceversion, e.g. apt=1.1 + strprintf(ErrorText, _("Changelog unavailable for %s=%s"), SrcName.c_str(), SrcVersion.c_str()); + // Let the error message print something sensible rather than "Failed to fetch /" + if (DestFilename.empty()) + DestFile = SrcName + ".changelog"; + else + DestFile = DestFilename; + Desc.URI = "changelog:/" + DestFile; + return; + } + + if (DestDir.empty()) + { + std::string const systemTemp = GetTempDir(); + char tmpname[100]; + snprintf(tmpname, sizeof(tmpname), "%s/apt-changelog-XXXXXX", systemTemp.c_str()); + if (NULL == mkdtemp(tmpname)) + { + _error->Errno("mkdtemp", "mkdtemp failed in changelog acquire of %s %s", SrcName.c_str(), SrcVersion.c_str()); + Status = StatError; + return; + } + DestFile = TemporaryDirectory = tmpname; + } + else + DestFile = DestDir; + + if (DestFilename.empty()) + DestFile = flCombine(DestFile, SrcName + ".changelog"); + else + DestFile = flCombine(DestFile, DestFilename); + + Desc.ShortDesc = "Changelog"; + strprintf(Desc.Description, "%s %s %s Changelog", URI::SiteOnly(Desc.URI).c_str(), SrcName.c_str(), SrcVersion.c_str()); + Desc.Owner = this; + QueueURI(Desc); + + if (Status == StatDone) // this happens if we queue the same changelog two times + { + Complete = true; + for (pkgAcquire::UriIterator I = Owner->UriBegin(); I != Owner->UriEnd(); ++I) + if (I->URI == Desc.URI) + if (DestFile != I->Owner->DestFile) + if (symlink(I->Owner->DestFile.c_str(), DestFile.c_str()) != 0) + { + ; // ignore error, there isn't anthing we could do to handle the edgecase of an edgecase + } + } +} + /*}}}*/ +std::string pkgAcqChangelog::URI(pkgCache::VerIterator const &Ver) /*{{{*/ +{ + char const * const SrcName = Ver.SourcePkgName(); + char const * const SrcVersion = Ver.SourceVerStr(); + pkgCache::PkgFileIterator PkgFile; + // find the first source for this version which promises a changelog + for (pkgCache::VerFileIterator VF = Ver.FileList(); VF.end() == false; ++VF) + { + pkgCache::PkgFileIterator const PF = VF.File(); + if (PF.Flagged(pkgCache::Flag::NotSource) || PF->Release == 0) + continue; + PkgFile = PF; + pkgCache::RlsFileIterator const RF = PF.ReleaseFile(); + std::string const uri = URI(RF, PF.Component(), SrcName, SrcVersion); + if (uri.empty()) + continue; + return uri; + } + return ""; +} +std::string pkgAcqChangelog::URITemplate(pkgCache::RlsFileIterator const &Rls) +{ + if (Rls.end() == true || (Rls->Label == 0 && Rls->Origin == 0)) + return ""; + std::string const serverConfig = "Acquire::Changelogs::URI"; + std::string server; +#define APT_EMPTY_SERVER \ + if (server.empty() == false) \ + { \ + if (server != "no") \ + return server; \ + return ""; \ + } +#define APT_CHECK_SERVER(X, Y) \ + if (Rls->X != 0) \ + { \ + std::string const specialServerConfig = serverConfig + "::" + Y + #X + "::" + Rls.X(); \ + server = _config->Find(specialServerConfig); \ + APT_EMPTY_SERVER \ + } + // this way e.g. Debian-Security can fallback to Debian + APT_CHECK_SERVER(Label, "Override::") + APT_CHECK_SERVER(Origin, "Override::") + + if (RealFileExists(Rls.FileName())) + { + _error->PushToStack(); + FileFd rf; + /* This can be costly. A caller wanting to get millions of URIs might + want to do this on its own once and use Override settings. + We don't do this here as Origin/Label are not as unique as they + should be so this could produce request order-dependent anomalies */ + if (OpenMaybeClearSignedFile(Rls.FileName(), rf) == true) + { + pkgTagFile TagFile(&rf, rf.Size()); + pkgTagSection Section; + if (TagFile.Step(Section) == true) + server = Section.FindS("Changelogs"); + } + _error->RevertToStack(); + APT_EMPTY_SERVER + } + + APT_CHECK_SERVER(Label, "") + APT_CHECK_SERVER(Origin, "") +#undef APT_CHECK_SERVER +#undef APT_EMPTY_SERVER + return ""; +} +std::string pkgAcqChangelog::URI(pkgCache::RlsFileIterator const &Rls, + char const * const Component, char const * const SrcName, + char const * const SrcVersion) +{ + return URI(URITemplate(Rls), Component, SrcName, SrcVersion); +} +std::string pkgAcqChangelog::URI(std::string const &Template, + char const * const Component, char const * const SrcName, + char const * const SrcVersion) +{ + if (Template.find("CHANGEPATH") == std::string::npos) + return ""; + + // the path is: COMPONENT/SRC/SRCNAME/SRCNAME_SRCVER, e.g. main/a/apt/1.1 or contrib/liba/libapt/2.0 + std::string Src = SrcName; + std::string path = APT::String::Startswith(SrcName, "lib") ? Src.substr(0, 4) : Src.substr(0,1); + path.append("/").append(Src).append("/"); + path.append(Src).append("_").append(StripEpoch(SrcVersion)); + // we omit component for releases without one (= flat-style repositories) + if (Component != NULL && strlen(Component) != 0) + path = std::string(Component) + "/" + path; + + return SubstVar(Template, "CHANGEPATH", path); +} + /*}}}*/ +// AcqChangelog::Failed - Failure handler /*{{{*/ +void pkgAcqChangelog::Failed(string const &Message, pkgAcquire::MethodConfig const * const Cnf) +{ + Item::Failed(Message,Cnf); + + std::string errText; + // TRANSLATOR: %s=%s is sourcename=sourceversion, e.g. apt=1.1 + strprintf(errText, _("Changelog unavailable for %s=%s"), SrcName.c_str(), SrcVersion.c_str()); + + // Error is probably something techy like 404 Not Found + if (ErrorText.empty()) + ErrorText = errText; + else + ErrorText = errText + " (" + ErrorText + ")"; + return; +} + /*}}}*/ +// AcqChangelog::Done - Item downloaded OK /*{{{*/ +void pkgAcqChangelog::Done(string const &Message,HashStringList const &CalcHashes, + pkgAcquire::MethodConfig const * const Cnf) +{ + Item::Done(Message,CalcHashes,Cnf); + + Complete = true; +} + /*}}}*/ +pkgAcqChangelog::~pkgAcqChangelog() /*{{{*/ +{ + if (TemporaryDirectory.empty() == false) + { + unlink(DestFile.c_str()); + rmdir(TemporaryDirectory.c_str()); + } +} + /*}}}*/ + // AcqFile::pkgAcqFile - Constructor /*{{{*/ pkgAcqFile::pkgAcqFile(pkgAcquire * const Owner,string const &URI, HashStringList const &Hashes, unsigned long long const Size,string const &Dsc,string const &ShortDesc, diff --git a/apt-pkg/acquire-item.h b/apt-pkg/acquire-item.h index b6d569737..606fd4173 100644 --- a/apt-pkg/acquire-item.h +++ b/apt-pkg/acquire-item.h @@ -1031,6 +1031,120 @@ class pkgAcqArchive : public pkgAcquire::Item std::string &StoreFilename); }; /*}}}*/ +/** \brief Retrieve the changelog for the given version {{{ + * + * Downloads the changelog to a temporary file it will also remove again + * while it is deconstructed or downloads it to a named location. + */ +class pkgAcqChangelog : public pkgAcquire::Item +{ + void *d; + std::string TemporaryDirectory; + std::string const SrcName; + std::string const SrcVersion; + + public: + // we will never have hashes for changelogs. + // If you need verified ones, download the deb and extract the changelog. + virtual HashStringList GetExpectedHashes() const { return HashStringList(); } + virtual bool HashesRequired() const { return false; } + + // Specialized action members + virtual void Failed(std::string const &Message,pkgAcquire::MethodConfig const * const Cnf); + virtual void Done(std::string const &Message, HashStringList const &CalcHashes, + pkgAcquire::MethodConfig const * const Cnf); + virtual std::string DescURI() const {return Desc.URI;}; + + /** returns the URI to the changelog of this version + * + * @param Ver is the version to get the changelog for + * @return the URI which will be used to acquire the changelog + */ + static std::string URI(pkgCache::VerIterator const &Ver); + + /** returns the URI to the changelog of this version + * + * \param Rls is the Release file the package comes from + * \param Component in which the package resides, can be empty + * \param SrcName is the source package name + * \param SrcVersion is the source package version + * @return the URI which will be used to acquire the changelog + */ + static std::string URI(pkgCache::RlsFileIterator const &Rls, + char const * const Component, char const * const SrcName, + char const * const SrcVersion); + + /** returns the URI to the changelog of this version + * + * \param Template URI where CHANGEPATH has to be filled in + * \param Component in which the package resides, can be empty + * \param SrcName is the source package name + * \param SrcVersion is the source package version + * @return the URI which will be used to acquire the changelog + */ + static std::string URI(std::string const &Template, + char const * const Component, char const * const SrcName, + char const * const SrcVersion); + + /** returns the URI template for this release file + * + * \param Rls is a Release file + * @return the URI template to use for this release file + */ + static std::string URITemplate(pkgCache::RlsFileIterator const &Rls); + + /** \brief Create a new pkgAcqChangelog object. + * + * \param Owner The pkgAcquire object with which this object is + * associated. + * \param Ver is the version to get the changelog for + * \param DestDir The directory the file should be downloaded into. + * Will be an autocreated (and cleaned up) temporary directory if not set. + * \param DestFilename The filename the file should have in #DestDir + * Defaults to sourcepackagename.changelog if not set. + */ + pkgAcqChangelog(pkgAcquire * const Owner, pkgCache::VerIterator const &Ver, + std::string const &DestDir="", std::string const &DestFilename=""); + + /** \brief Create a new pkgAcqChangelog object. + * + * \param Owner The pkgAcquire object with which this object is + * associated. + * \param Rls is the Release file the package comes from + * \param Component in which the package resides, can be empty + * \param SrcName is the source package name + * \param SrcVersion is the source package version + * \param DestDir The directory the file should be downloaded into. + * Will be an autocreated (and cleaned up) temporary directory if not set. + * \param DestFilename The filename the file should have in #DestDir + * Defaults to sourcepackagename.changelog if not set. + */ + pkgAcqChangelog(pkgAcquire * const Owner, pkgCache::RlsFileIterator const &Rls, + char const * const Component, char const * const SrcName, char const * const SrcVersion, + std::string const &DestDir="", std::string const &DestFilename=""); + + /** \brief Create a new pkgAcqChangelog object. + * + * \param Owner The pkgAcquire object with which this object is + * associated. + * \param URI is to be used to get the changelog + * \param SrcName is the source package name + * \param SrcVersion is the source package version + * \param DestDir The directory the file should be downloaded into. + * Will be an autocreated (and cleaned up) temporary directory if not set. + * \param DestFilename The filename the file should have in #DestDir + * Defaults to sourcepackagename.changelog if not set. + */ + pkgAcqChangelog(pkgAcquire * const Owner, std::string const &URI, + char const * const SrcName, char const * const SrcVersion, + std::string const &DestDir="", std::string const &DestFilename=""); + + virtual ~pkgAcqChangelog(); + +private: + APT_HIDDEN void Init(std::string const &DestDir, std::string const &DestFilename); +}; + /*}}}*/ /** \brief Retrieve an arbitrary file to the current directory. {{{ * * The file is retrieved even if it is accessed via a URL type that diff --git a/apt-pkg/init.cc b/apt-pkg/init.cc index e2239a906..96966b249 100644 --- a/apt-pkg/init.cc +++ b/apt-pkg/init.cc @@ -119,6 +119,10 @@ bool pkgInitConfig(Configuration &Cnf) Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::flatDescription", "$(SITE) $(RELEASE) Sources"); Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::Optional", false); + Cnf.CndSet("Acquire::Changelogs::URI::Origin::Debian", "http://metadata.ftp-master.debian.org/changelogs/CHANGEPATH_changelog"); + Cnf.CndSet("Acquire::Changelogs::URI::Origin::Ubuntu", "http://changelogs.ubuntu.com/changelogs/pool/CHANGEPATH/changelog"); + Cnf.CndSet("Acquire::Changelogs::URI::Origin::Ultimedia", "http://packages.ultimediaos.com/changelogs/pool/CHANGEPATH/changelog.txt"); + bool Res = true; // Read an alternate config file diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc index a4cd3c377..184b51d23 100644 --- a/cmdline/apt-get.cc +++ b/cmdline/apt-get.cc @@ -1410,196 +1410,70 @@ static bool DoBuildDep(CommandLine &CmdL) return true; } /*}}}*/ -// GetChangelogPath - return a path pointing to a changelog file or dir /*{{{*/ -// --------------------------------------------------------------------- -/* This returns a "path" string for the changelog url construction. - * Please note that its not complete, it either needs a "/changelog" - * appended (for the packages.debian.org/changelogs site) or a - * ".changelog" (for third party sites that store the changelog in the - * pool/ next to the deb itself) - * Example return: "pool/main/a/apt/apt_0.8.8ubuntu3" - */ -static string GetChangelogPath(CacheFile &Cache, - pkgCache::VerIterator Ver) -{ - pkgRecords Recs(Cache); - pkgRecords::Parser &rec=Recs.Lookup(Ver.FileList()); - string path = flNotFile(rec.FileName()); -#if APT_PKG_ABI >= 413 - path.append(Ver.SourcePkgName()); - path.append("_"); - path.append(StripEpoch(Ver.SourceVerStr())); -#else - string srcpkg = rec.SourcePkg().empty() ? Ver.ParentPkg().Name() : rec.SourcePkg(); - string ver = Ver.VerStr(); - // if there is a source version it always wins - if (rec.SourceVer() != "") - ver = rec.SourceVer(); - path += srcpkg + "_" + StripEpoch(ver); -#endif - return path; -} - /*}}}*/ -// GuessThirdPartyChangelogUri - return url /*{{{*/ -// --------------------------------------------------------------------- -/* Contruct a changelog file path for third party sites that do not use - * packages.debian.org/changelogs - * This simply uses the ArchiveURI() of the source pkg and looks for - * a .changelog file there, Example for "mediabuntu": - * apt-get changelog mplayer-doc: - * http://packages.medibuntu.org/pool/non-free/m/mplayer/mplayer_1.0~rc4~try1.dsfg1-1ubuntu1+medibuntu1.changelog - */ -static bool GuessThirdPartyChangelogUri(CacheFile &Cache, - pkgCache::VerIterator Ver, - string &out_uri) -{ - // get the binary deb server path - pkgCache::VerFileIterator Vf = Ver.FileList(); - if (Vf.end() == true) - return false; - pkgCache::PkgFileIterator F = Vf.File(); - pkgIndexFile *index; - pkgSourceList *SrcList = Cache.GetSourceList(); - if(SrcList->FindIndex(F, index) == false) - return false; - - // get archive uri for the binary deb - string path_without_dot_changelog = GetChangelogPath(Cache, Ver); - out_uri = index->ArchiveURI(path_without_dot_changelog + ".changelog"); - - // now strip away the filename and add srcpkg_srcver.changelog - return true; -} - /*}}}*/ -// DownloadChangelog - Download the changelog /*{{{*/ -// --------------------------------------------------------------------- -static bool DownloadChangelog(CacheFile &CacheFile, pkgAcquire &Fetcher, - pkgCache::VerIterator Ver, string targetfile) -/* Download a changelog file for the given package version to - * targetfile. This will first try the server from Apt::Changelogs::Server - * (http://packages.debian.org/changelogs by default) and if that gives - * a 404 tries to get it from the archive directly (see - * GuessThirdPartyChangelogUri for details how) - */ -{ - // make the server root configurable - string const server = _config->Find("Apt::Changelogs::Server", - "http://packages.debian.org/changelogs"); - string const path = GetChangelogPath(CacheFile, Ver); - string changelog_uri; - if (APT::String::Endswith(server, "/") == true) - strprintf(changelog_uri, "%s%s/changelog", server.c_str(), path.c_str()); - else - strprintf(changelog_uri, "%s/%s/changelog", server.c_str(), path.c_str()); - if (_config->FindB("APT::Get::Print-URIs", false) == true) - { - std::cout << '\'' << changelog_uri << '\'' << std::endl; - return true; - } - pkgCache::PkgIterator const Pkg = Ver.ParentPkg(); - - string descr; - strprintf(descr, _("Changelog for %s (%s)"), Pkg.Name(), changelog_uri.c_str()); - // queue it - pkgAcquire::Item const * itm = new pkgAcqFile(&Fetcher, changelog_uri, "", 0, descr, Pkg.Name(), "ignored", targetfile); - - // Disable drop-privs if "_apt" can not write to the target dir - CheckDropPrivsMustBeDisabled(Fetcher); - - // try downloading it, if that fails, try third-party-changelogs location - // FIXME: Fetcher.Run() is "Continue" even if I get a 404?!? - Fetcher.Run(); - if (itm->Status != pkgAcquire::Item::StatDone) - { - string third_party_uri; - if (GuessThirdPartyChangelogUri(CacheFile, Ver, third_party_uri)) - { - strprintf(descr, _("Changelog for %s (%s)"), Pkg.Name(), third_party_uri.c_str()); - itm = new pkgAcqFile(&Fetcher, third_party_uri, "", 0, descr, Pkg.Name(), "ignored", targetfile); - Fetcher.Run(); - } - } - - if (itm->Status == pkgAcquire::Item::StatDone) - return true; - - // error - return _error->Error("changelog download failed"); -} - /*}}}*/ // DoChangelog - Get changelog from the command line /*{{{*/ -// --------------------------------------------------------------------- static bool DoChangelog(CommandLine &CmdL) { CacheFile Cache; if (Cache.ReadOnlyOpen() == false) return false; - + APT::CacheSetHelper helper; APT::VersionList verset = APT::VersionList::FromCommandLine(Cache, CmdL.FileList + 1, APT::CacheSetHelper::CANDIDATE, helper); if (verset.empty() == true) return false; pkgAcquire Fetcher; + AcqTextStatus Stat(std::cout, ScreenWidth,_config->FindI("quiet",0)); + Fetcher.SetLog(&Stat); - if (_config->FindB("APT::Get::Print-URIs", false) == true) + bool const downOnly = _config->FindB("APT::Get::Download-Only", false); + bool const printOnly = _config->FindB("APT::Get::Print-URIs", false); + + for (APT::VersionList::const_iterator Ver = verset.begin(); + Ver != verset.end(); + ++Ver) { - bool Success = true; - for (APT::VersionList::const_iterator Ver = verset.begin(); - Ver != verset.end(); ++Ver) - Success &= DownloadChangelog(Cache, Fetcher, Ver, ""); - return Success; + if (printOnly) + new pkgAcqChangelog(&Fetcher, Ver, "/dev/null"); + else if (downOnly) + new pkgAcqChangelog(&Fetcher, Ver, "."); + else + new pkgAcqChangelog(&Fetcher, Ver); } - AcqTextStatus Stat(std::cout, ScreenWidth,_config->FindI("quiet",0)); - Fetcher.SetLog(&Stat); + if (printOnly == false) + { + // Disable drop-privs if "_apt" can not write to the target dir + CheckDropPrivsMustBeDisabled(Fetcher); + if (_error->PendingError() == true) + return false; - bool const downOnly = _config->FindB("APT::Get::Download-Only", false); + bool Failed = false; + if (AcquireRun(Fetcher, 0, &Failed, NULL) == false || Failed == true) + return false; + } - char tmpname[100]; - const char* tmpdir = NULL; - if (downOnly == false) + if (downOnly == false || printOnly == true) { - std::string systemTemp = GetTempDir(); - snprintf(tmpname, sizeof(tmpname), "%s/apt-changelog-XXXXXX", - systemTemp.c_str()); - tmpdir = mkdtemp(tmpname); - if (tmpdir == NULL) - return _error->Errno("mkdtemp", "mkdtemp failed"); - - std::string const SandboxUser = _config->Find("APT::Sandbox::User"); - if (getuid() == 0 && SandboxUser.empty() == false) // if we aren't root, we can't chown, so don't try it + bool Failed = false; + for (pkgAcquire::ItemIterator I = Fetcher.ItemsBegin(); I != Fetcher.ItemsEnd(); ++I) { - struct passwd const * const pw = getpwnam(SandboxUser.c_str()); - struct group const * const gr = getgrnam("root"); - if (pw != NULL && gr != NULL) + if (printOnly) { - // chown the tmp dir directory we use to the sandbox user - if(chown(tmpdir, pw->pw_uid, gr->gr_gid) != 0) - _error->WarningE("DoChangelog", "chown to %s:%s of directory %s failed", SandboxUser.c_str(), "root", tmpdir); + if ((*I)->ErrorText.empty() == false) + { + Failed = true; + _error->Error("%s", (*I)->ErrorText.c_str()); + } + else + cout << '\'' << (*I)->DescURI() << "' " << flNotDir((*I)->DestFile) << std::endl; } + else + DisplayFileInPager((*I)->DestFile); } + return Failed == false; } - for (APT::VersionList::const_iterator Ver = verset.begin(); - Ver != verset.end(); - ++Ver) - { - string changelogfile; - if (downOnly == false) - changelogfile.append(tmpname).append("/changelog"); - else - changelogfile.append(Ver.ParentPkg().Name()).append(".changelog"); - if (DownloadChangelog(Cache, Fetcher, Ver, changelogfile) && downOnly == false) - { - DisplayFileInPager(changelogfile); - // cleanup temp file - unlink(changelogfile.c_str()); - } - } - // clenaup tmp dir - if (tmpdir != NULL) - rmdir(tmpdir); return true; } /*}}}*/ diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml index da077afa7..5b6788ed4 100644 --- a/doc/apt-get.8.xml +++ b/doc/apt-get.8.xml @@ -230,16 +230,12 @@ - changelog downloads a package changelog and displays - it through sensible-pager. The server name and base - directory is defined in the APT::Changelogs::Server - variable (e.g. packages.debian.org/changelogs for - Debian or changelogs.ubuntu.com/changelogs for - Ubuntu). - By default it displays the changelog for the version that is - installed. However, you can specify the same options as for - the command. - + changelog tries to download the + changelog of a package and displays it through + sensible-pager. By default it + displays the changelog for the version that is installed. + However, you can specify the same options as for the + command. diff --git a/doc/apt.conf.5.xml b/doc/apt.conf.5.xml index efe986ea8..7d5f7e9b3 100644 --- a/doc/apt.conf.5.xml +++ b/doc/apt.conf.5.xml @@ -618,6 +618,33 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; + scope + + Acquiring changelogs can only be done if an URI is known from where to get them. + Preferable the Release file indicates this in a 'Changelogs' field. If this isn't + available the Label/Origin field of the Release file is used to check if a + Acquire::Changelogs::URI::Label::LABEL or + Acquire::Changelogs::URI::Origin::ORIGIN option + exists and if so this value is taken. The value in the Release file can be overridden + with Acquire::Changelogs::URI::Override::Label::LABEL + or Acquire::Changelogs::URI::Override::Origin::ORIGIN. + + The value should be a normal URI to a text file, expect that package specific data is + replaced with the placeholder CHANGEPATH. The + value for it is: 1. if the package is from a component (e.g. main) + this is the first part otherwise it is omitted, 2. the first letter of source package name, + expect if the source package name starts with 'lib' in which case it will + be the first four letters. 3. The complete source package name. 4. the complete name again and + 5. the source version. + The first (if present), second, third and fourth part are separated by a slash ('/') + and between the fourth and fifth part is an underscore ('_'). + + The special value 'no' is available for this option indicating that + this source can't be used to acquire changelog files from. Another source will be tried + if available in this case. + + + diff --git a/doc/examples/configure-index b/doc/examples/configure-index index ef1ae056d..1339335fa 100644 --- a/doc/examples/configure-index +++ b/doc/examples/configure-index @@ -117,14 +117,6 @@ APT // does a ExecFork) Keep-Fds {}; - Changelogs - { - // server the provides the changelogs, the code will assume - // the changlogs are in the pool/ under a srcpkg_ver directory - // with the name "changelog" - Server "http://packages.debian.org/changelogs"; - }: - // control parameters for cron jobs by /etc/cron.daily/apt Periodic { @@ -305,6 +297,9 @@ Acquire "none"; "fr"; }; + + // Location of the changelogs with the placeholder CHANGEPATH (e.g. "main/a/apt/apt_1.1") + Changelogs::URI::Origin::Debian "http://metadata.ftp-master.debian.org/changelogs/CHANGEPATH_changelog"; }; // Directory layout diff --git a/test/integration/framework b/test/integration/framework index 7b03c09ef..1b99929e2 100644 --- a/test/integration/framework +++ b/test/integration/framework @@ -520,6 +520,12 @@ Package: $NAME" > debian/control buildsimplenativepackage() { local NAME="$1" + local NM + if [ "$(echo "$NAME" | cut -c 1-3)" = 'lib' ]; then + NM="$(echo "$NAME" | cut -c 1-4)" + else + NM="$(echo "$NAME" | cut -c 1)" + fi local ARCH="$2" local VERSION="$3" local RELEASE="${4:-unstable}" @@ -600,15 +606,15 @@ Package: $NAME" >> ${BUILDDIR}/debian/control (cd ${BUILDDIR}; dpkg-gencontrol -DArchitecture=$arch) (cd ${BUILDDIR}/debian/tmp; md5sum $(find usr/ -type f) > DEBIAN/md5sums) local LOG="${BUILDDIR}/../${NAME}_${VERSION}_${arch}.dpkg-deb.log" - # ensure the right permissions as dpkg-deb ensists + # ensure the right permissions as dpkg-deb insists chmod 755 ${BUILDDIR}/debian/tmp/DEBIAN testsuccess --nomsg dpkg-deb -Z${COMPRESS_TYPE} --build ${BUILDDIR}/debian/tmp ${BUILDDIR}/.. echo "pool/${NAME}_${VERSION}_${arch}.deb" >> ${BUILDDIR}/../${RELEASE}.${DISTSECTION}.pkglist done - mkdir -p ${BUILDDIR}/../${NAME}_${VERSION} - cp ${BUILDDIR}/debian/changelog ${BUILDDIR}/../${NAME}_${VERSION}/ - cp ${BUILDDIR}/debian/changelog ${BUILDDIR}/../${NAME}_${VERSION}.changelog + local CHANGEPATH="${BUILDDIR}/../${DISTSECTION}/${NM}/${NAME}/${NAME}_${VERSION}" + mkdir -p $CHANGEPATH + cp ${BUILDDIR}/debian/changelog $CHANGEPATH rm -rf "${BUILDDIR}" msgdone "info" } @@ -876,6 +882,7 @@ getcodenamefromsuite() { } getreleaseversionfromsuite() { true; } getlabelfromsuite() { true; } +getoriginfromsuite() { true; } generatereleasefiles() { # $1 is the Date header and $2 is the ValidUntil header to be set @@ -887,16 +894,21 @@ generatereleasefiles() { local CODENAME="$(getcodenamefromsuite $SUITE)" local VERSION="$(getreleaseversionfromsuite $SUITE)" local LABEL="$(getlabelfromsuite $SUITE)" + local ORIGIN="$(getoriginfromsuite $SUITE)" if [ -n "$VERSION" ]; then VERSION="-o APT::FTPArchive::Release::Version=${VERSION}" fi if [ -n "$LABEL" ]; then LABEL="-o APT::FTPArchive::Release::Label=${LABEL}" fi + if [ -n "$ORIGIN" ]; then + ORIGIN="-o APT::FTPArchive::Release::Origin=${ORIGIN}" + fi aptftparchive -qq release $dir \ -o APT::FTPArchive::Release::Suite="${SUITE}" \ -o APT::FTPArchive::Release::Codename="${CODENAME}" \ ${LABEL} \ + ${ORIGIN} \ ${VERSION} \ | sed -e '/0 Release$/ d' > $dir/Release # remove the self reference if [ "$SUITE" = "experimental" -o "$SUITE" = "experimental2" ]; then @@ -1450,9 +1462,9 @@ testfilestats() { msgpass else echo >&2 - ls -ld >&2 "$1" + ls -ld >&2 "$1" || true echo -n >&2 "stat(1) reports for $2: " - stat --format "$2" "$1" + stat --format "$2" "$1" || true msgfail fi } diff --git a/test/integration/test-apt-get-changelog b/test/integration/test-apt-get-changelog index 7e81c71b6..5fa8543b9 100755 --- a/test/integration/test-apt-get-changelog +++ b/test/integration/test-apt-get-changelog @@ -5,44 +5,98 @@ TESTDIR=$(readlink -f $(dirname $0)) . $TESTDIR/framework setupenvironment -configarchitecture "i386" +configarchitecture 'native' -buildsimplenativepackage 'apt' 'all' '1.0' 'stable' +buildsimplenativepackage 'foo' 'all' '1.0' 'stable' +buildsimplenativepackage 'libbar' 'all' '1.0' 'stable' + +getlabelfromsuite() { echo 'Testcases'; } +getoriginfromsuite() { echo 'Debian'; } setupaptarchive --no-update changetowebserver testsuccess aptget update -# simulate normal user with non-existent root-owned directories -rm -rf rootdir/var/cache/apt/archives/ -mkdir rootdir/var/cache/apt/archives/ -addtrap 'prefix' "chmod -f -R +w $PWD/rootdir/var/cache/apt/archives || true;" -chmod -R -w rootdir/var/cache/apt/archives +testsuccessequal "'http://metadata.ftp-master.debian.org/changelogs/main/f/foo/foo_1.0_changelog' foo.changelog +'http://metadata.ftp-master.debian.org/changelogs/main/libb/libbar/libbar_1.0_changelog' libbar.changelog" aptget changelog foo libbar --print-uris + +releasechanger() { + # modifying the Release files in lists… bad stuff. Good that this is only a test… + sed -i "s#^${1}: .*#${1}: ${2}#" $(find rootdir/var/lib/apt/lists -name '*Release') + rm -f rootdir/var/cache/apt/*.bin +} +releasechanger 'Origin' 'Ubuntu' +testsuccessequal "'http://changelogs.ubuntu.com/changelogs/pool/main/f/foo/foo_1.0/changelog' foo.changelog +'http://changelogs.ubuntu.com/changelogs/pool/main/libb/libbar/libbar_1.0/changelog' libbar.changelog" aptget changelog foo libbar --print-uris + +releasechanger 'Label' 'Debian' +testsuccessequal "'http://changelogs.ubuntu.com/changelogs/pool/main/f/foo/foo_1.0/changelog' foo.changelog +'http://changelogs.ubuntu.com/changelogs/pool/main/libb/libbar/libbar_1.0/changelog' libbar.changelog" aptget changelog foo libbar --print-uris + +testsuccessequal "'http://localhost:8080/main/f/foo/foo_1.0.changelog' foo.changelog +'http://localhost:8080/main/libb/libbar/libbar_1.0.changelog' libbar.changelog" aptget changelog foo libbar --print-uris -o Acquire::Changelogs::URI::Label::Debian='http://localhost:8080/CHANGEPATH.changelog' + +sed -i '/^Origin: / a\ +Changelogs: http://example.org/CHANGEPATH-changelog' $(find rootdir/var/lib/apt/lists -name '*Release') +rm -f rootdir/var/cache/apt/*.bin -echo 'Apt::Changelogs::Server "http://localhost:8080/";' > rootdir/etc/apt/apt.conf.d/changelog.conf +testsuccessequal "'http://example.org/main/f/foo/foo_1.0-changelog' foo.changelog +'http://example.org/main/libb/libbar/libbar_1.0-changelog' libbar.changelog" aptget changelog foo libbar --print-uris -o Acquire::Changelogs::URI::Label::Debian='http://localhost:8080/CHANGEPATH.changelog' -testsuccessequal "'http://localhost:8080/pool/apt_1.0/changelog'" aptget changelog apt --print-uris +testsuccessequal "'http://localhost:8080/main/f/foo/foo_1.0.changelog' foo.changelog +'http://localhost:8080/main/libb/libbar/libbar_1.0.changelog' libbar.changelog" aptget changelog foo libbar --print-uris -o Acquire::Changelogs::URI::Override::Label::Debian='http://localhost:8080/CHANGEPATH.changelog' -testsuccessequal "'http://localhost:8080/pool/apt_1.0/changelog' -'http://localhost:8080/pool/apt_1.0/changelog'" aptget changelog apt apt --print-uris +releasechanger 'Changelogs' 'no' +testequal 'E: Failed to fetch changelog:/foo.changelog Changelog unavailable for foo=1.0 +' aptget changelog foo -qq -d + +sed -i '/^Changelogs: / d' $(find rootdir/var/lib/apt/lists -name '*Release') +releasechanger 'Label' 'Testcases' + +echo 'Acquire::Changelogs::URI::Label::Testcases "http://localhost:8080/CHANGEPATH/change.txt";' > rootdir/etc/apt/apt.conf.d/changelog.conf +testsuccessequal "'http://localhost:8080/main/f/foo/foo_1.0/change.txt' foo.changelog +'http://localhost:8080/main/libb/libbar/libbar_1.0/change.txt' libbar.changelog" aptget changelog foo libbar --print-uris + +echo 'Acquire::Changelogs::URI::Label::Testcases "http://localhost:8080/pool/CHANGEPATH/changelog";' > rootdir/etc/apt/apt.conf.d/changelog.conf +testsuccessequal "'http://localhost:8080/pool/main/f/foo/foo_1.0/changelog' foo.changelog" aptget changelog foo --print-uris cd downloaded -testsuccess aptget changelog apt -qq -testfileequal '../rootdir/tmp/testsuccess.output' "$(cat ../aptarchive/pool/apt_1.0/changelog)" +testsuccess aptget changelog foo -qq +testfileequal '../rootdir/tmp/testsuccess.output' "$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog)" + +testsuccess aptget changelog foo libbar -qq +testfileequal '../rootdir/tmp/testsuccess.output' "$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog) +$(cat ../aptarchive/pool/main/libb/libbar/libbar_1.0/changelog)" + +testsuccess aptget changelog foo -d +testfilestats 'foo.changelog' '%U:%G:%a' '=' "${TEST_DEFAULT_USER}:${TEST_DEFAULT_GROUP}:644" +testfileequal 'foo.changelog' "$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog)" +rm -f foo.changelog -testsuccess aptget changelog apt -d -testfileequal 'apt.changelog' "$(cat ../aptarchive/pool/apt_1.0/changelog)" -testfilestats 'apt.changelog' '%U:%G:%a' '=' "${TEST_DEFAULT_USER}:${TEST_DEFAULT_GROUP}:644" -rm -f apt.changelog ../aptarchive/pool/apt_1.0/changelog +testsuccess aptget changelog libbar foo -d +testfilestats 'libbar.changelog' '%U:%G:%a' '=' "${TEST_DEFAULT_USER}:${TEST_DEFAULT_GROUP}:644" +testfilestats 'foo.changelog' '%U:%G:%a' '=' "${TEST_DEFAULT_USER}:${TEST_DEFAULT_GROUP}:644" +testfileequal 'libbar.changelog' "$(cat ../aptarchive/pool/main/libb/libbar/libbar_1.0/changelog)" +testfileequal 'foo.changelog' "$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog)" +rm -f libbar.changelog foo.changelog -testequal "$(cat ../aptarchive/pool/apt_1.0.changelog)" aptget changelog apt \ - -qq -o APT::Changelogs::Server='http://not-on-the-main-server:8080/' +# as such bogus, but can happen with multiple binaries from the same source +testsuccessequal "'http://localhost:8080/pool/main/f/foo/foo_1.0/changelog' foo.changelog +'http://localhost:8080/pool/main/f/foo/foo_1.0/changelog' foo.changelog" aptget changelog foo foo --print-uris +testsuccess aptget changelog foo foo -qq +testfileequal '../rootdir/tmp/testsuccess.output' "$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog) +$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog)" +testsuccess aptget changelog foo foo -d +testfilestats 'foo.changelog' '%U:%G:%a' '=' "${TEST_DEFAULT_USER}:${TEST_DEFAULT_GROUP}:644" +testfileequal 'foo.changelog' "$(cat ../aptarchive/pool/main/f/foo/foo_1.0/changelog)" +rm -f foo.changelog -testsuccess aptget changelog apt -d -testfileequal 'apt.changelog' "$(cat ../aptarchive/pool/apt_1.0.changelog)" -testfilestats 'apt.changelog' '%U:%G:%a' '=' "${TEST_DEFAULT_USER}:${TEST_DEFAULT_GROUP}:644" -rm -f apt.changelog ../aptarchive/pool/apt_1.0.changelog +# no CHANGEPATH in the URI +testequal 'E: Failed to fetch changelog:/foo.changelog Changelog unavailable for foo=1.0 +' aptget changelog foo -qq -d -o Acquire::Changelogs::URI::Label::Testcases='http://localhost:8080/change.txt' +testfailure test -e foo.changelog -testequal 'E: changelog download failed' aptget changelog apt -qq -d -o APT::Changelogs::Server='http://not-on-the-main-server:8080/' -testfailure test -e apt.changelog +testequal 'E: Failed to fetch http://localhost:8080/does/not/exist/main/f/foo/foo_1.0/change.txt Changelog unavailable for foo=1.0 (404 Not Found) +' aptget changelog foo -qq -d -o Acquire::Changelogs::URI::Label::Testcases='http://localhost:8080/does/not/exist/CHANGEPATH/change.txt' +testfailure test -e foo.changelog diff --git a/test/integration/test-bug-722207-print-uris-even-if-very-quiet b/test/integration/test-bug-722207-print-uris-even-if-very-quiet index 2cad929cc..e51d72ccd 100755 --- a/test/integration/test-bug-722207-print-uris-even-if-very-quiet +++ b/test/integration/test-bug-722207-print-uris-even-if-very-quiet @@ -12,6 +12,7 @@ insertpackage 'unstable' 'apt' 'all' '2' insertsource 'unstable' 'apt' 'all' '2' insertsource 'unstable' 'apt2' 'all' '1' +getoriginfromsuite() { echo 'Debian'; } setupaptarchive APTARCHIVE=$(readlink -f ./aptarchive) @@ -22,7 +23,7 @@ testsuccessequal "'file://${APTARCHIVE}/pool/main/apt/apt_2_all.deb' apt_2_all.d testsuccessequal "'file://${APTARCHIVE}/pool/main/apt/apt_2_all.deb' apt_2_all.deb 0 " aptget download apt -qq --print-uris testsuccessequal "'file://${APTARCHIVE}/apt_2.dsc' apt_2.dsc 0 MD5Sum:d41d8cd98f00b204e9800998ecf8427e 'file://${APTARCHIVE}/apt_2.tar.gz' apt_2.tar.gz 0 MD5Sum:d41d8cd98f00b204e9800998ecf8427e" aptget source apt -qq --print-uris -testsuccessequal "'http://packages.debian.org/changelogs/pool/main/apt/apt_2/changelog'" aptget changelog apt -qq --print-uris +testsuccessequal "'http://metadata.ftp-master.debian.org/changelogs/main/a/apt/apt_2_changelog' apt.changelog" aptget changelog apt -qq --print-uris testsuccessequal "'file://${APTARCHIVE}/apt_2.dsc' apt_2.dsc 0 MD5Sum:d41d8cd98f00b204e9800998ecf8427e 'file://${APTARCHIVE}/apt_2.tar.gz' apt_2.tar.gz 0 MD5Sum:d41d8cd98f00b204e9800998ecf8427e diff --git a/test/integration/test-bug-738785-switch-protocol b/test/integration/test-bug-738785-switch-protocol index f6336ffe3..539080200 100755 --- a/test/integration/test-bug-738785-switch-protocol +++ b/test/integration/test-bug-738785-switch-protocol @@ -10,6 +10,7 @@ configarchitecture "i386" buildsimplenativepackage 'apt' 'all' '1.0' 'stable' # setup http redirecting to https +getlabelfromsuite() { echo 'Testcases'; } setupaptarchive --no-update changetowebserver -o 'aptwebserver::redirect::replace::/redirectme/=https://localhost:4433/' \ -o 'aptwebserver::redirect::replace::/downgrademe/=http://localhost:8080/' \ @@ -20,10 +21,10 @@ sed -i -e 's#:4433/#:8080/redirectme#' -e 's# https:# http:#' rootdir/etc/apt/so testsuccess aptget update -o Debug::Acquire::http=1 -o Debug::Acquire::https=1 -o Debug::pkgAcquire::Worker=1 msgtest 'Test that the webserver does not answer' 'http requests' -downloadfile 'http://localhost:8080/pool/apt_1.0/changelog' changelog >/dev/null 2>&1 && msgfail || msgpass +downloadfile 'http://localhost:8080/pool/main/a/apt/apt_1.0/changelog' changelog >/dev/null 2>&1 && msgfail || msgpass -echo 'Apt::Changelogs::Server "http://localhost:8080/redirectme";' > rootdir/etc/apt/apt.conf.d/changelog.conf -testsuccessequal "'http://localhost:8080/redirectme/pool/apt_1.0/changelog'" aptget changelog apt --print-uris +echo 'Acquire::Changelogs::URI::Label::Testcases "http://localhost:8080/redirectme/pool/CHANGEPATH/changelog";' > rootdir/etc/apt/apt.conf.d/changelog.conf +testsuccessequal "'http://localhost:8080/redirectme/pool/main/a/apt/apt_1.0/changelog' apt.changelog" aptget changelog apt --print-uris cd downloaded testsuccess aptget changelog apt -d diff --git a/vendor/ubuntu/apt.conf-01-vendor-ubuntu b/vendor/ubuntu/apt.conf-01-vendor-ubuntu index c4092ff44..e69de29bb 100644 --- a/vendor/ubuntu/apt.conf-01-vendor-ubuntu +++ b/vendor/ubuntu/apt.conf-01-vendor-ubuntu @@ -1,6 +0,0 @@ -// Server information for apt-changelog -APT { - Changelogs { - Server "http://changelogs.ubuntu.com/changelogs"; - }; -}; -- cgit v1.2.3-70-g09d2 From 463c8d801595ce5ac94d7c032264820be7434232 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Sun, 21 Jun 2015 16:47:53 +0200 Subject: support lang= and target= sources.list options We support arch= for a while, now we finally add lang= as well and as a first simple way of controlling which targets to acquire also target=. This asked for a redesign of the internal API of parsing and storing information about 'deb' and 'deb-src' lines. As this API isn't visible to the outside no damage done through. Beside being a nice cleanup (= it actually does more in less lines) it also provides us with a predictable order of architectures as provides in the configuration rather than based on string sorting-order, so that now the native architecture is parsed/displayed first. Observeable e.g. in apt-get output. --- apt-pkg/deb/debindexfile.cc | 6 +- apt-pkg/deb/debmetaindex.cc | 430 +++++++++------------ apt-pkg/deb/debmetaindex.h | 38 +- apt-pkg/sourcelist.cc | 40 +- apt-pkg/sourcelist.h | 8 +- cmdline/apt-get.cc | 4 +- doc/sources.list.5.xml | 16 + .../test-bug-632221-cross-dependency-satisfaction | 22 +- .../test-ignore-provides-if-versioned-breaks | 10 +- .../test-ignore-provides-if-versioned-conflicts | 10 +- .../test-sourceslist-lang-plusminus-options | 87 +++++ 11 files changed, 354 insertions(+), 317 deletions(-) create mode 100755 test/integration/test-sourceslist-lang-plusminus-options (limited to 'doc') diff --git a/apt-pkg/deb/debindexfile.cc b/apt-pkg/deb/debindexfile.cc index 29a9a941c..c5086b04b 100644 --- a/apt-pkg/deb/debindexfile.cc +++ b/apt-pkg/deb/debindexfile.cc @@ -510,7 +510,7 @@ class APT_HIDDEN debIFTypeDebPkgFile : public pkgIndexFile::Type { return new debDebFileRecordParser(File.FileName()); }; - debIFTypeDebPkgFile() {Label = "deb Package file";}; + debIFTypeDebPkgFile() {Label = "Debian deb file";}; }; class APT_HIDDEN debIFTypeDscFile : public pkgIndexFile::Type { @@ -519,7 +519,7 @@ class APT_HIDDEN debIFTypeDscFile : public pkgIndexFile::Type { return new debDscRecordParser(DscFile, NULL); }; - debIFTypeDscFile() {Label = "dsc File Source Index";}; + debIFTypeDscFile() {Label = "Debian dsc file";}; }; class APT_HIDDEN debIFTypeDebianSourceDir : public pkgIndexFile::Type { @@ -528,7 +528,7 @@ class APT_HIDDEN debIFTypeDebianSourceDir : public pkgIndexFile::Type { return new debDscRecordParser(SourceDir + string("/debian/control"), NULL); }; - debIFTypeDebianSourceDir() {Label = "debian/control File Source Index";}; + debIFTypeDebianSourceDir() {Label = "Debian control file";}; }; APT_HIDDEN debIFTypeSrc _apt_Src; diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index 5a517b290..f690a8d64 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -29,11 +29,25 @@ #include #include -using namespace std; - -string debReleaseIndex::MetaIndexInfo(const char *Type) const +class APT_HIDDEN debReleaseIndexPrivate /*{{{*/ +{ + public: + struct APT_HIDDEN debSectionEntry + { + std::string Name; + std::vector Targets; + std::vector Architectures; + std::vector Languages; + }; + + std::vector DebEntries; + std::vector DebSrcEntries; +}; + /*}}}*/ +// ReleaseIndex::MetaIndex* - display helpers /*{{{*/ +std::string debReleaseIndex::MetaIndexInfo(const char *Type) const { - string Info = ::URI::ArchiveOnly(URI) + ' '; + std::string Info = ::URI::ArchiveOnly(URI) + ' '; if (Dist[Dist.size() - 1] == '/') { if (Dist != "/") @@ -50,15 +64,15 @@ std::string debReleaseIndex::Describe() const return MetaIndexInfo("Release"); } -string debReleaseIndex::MetaIndexFile(const char *Type) const +std::string debReleaseIndex::MetaIndexFile(const char *Type) const { return _config->FindDir("Dir::State::lists") + URItoFileName(MetaIndexURI(Type)); } -string debReleaseIndex::MetaIndexURI(const char *Type) const +std::string debReleaseIndex::MetaIndexURI(const char *Type) const { - string Res; + std::string Res; if (Dist == "/") Res = URI; @@ -70,8 +84,8 @@ string debReleaseIndex::MetaIndexURI(const char *Type) const Res += Type; return Res; } - -std::string debReleaseIndex::LocalFileName() const + /*}}}*/ +std::string debReleaseIndex::LocalFileName() const /*{{{*/ { // see if we have a InRelease file std::string PathInRelease = MetaIndexFile("InRelease"); @@ -84,28 +98,24 @@ std::string debReleaseIndex::LocalFileName() const return ""; } - -debReleaseIndex::debReleaseIndex(string const &URI, string const &Dist) : - metaIndex(URI, Dist, "deb"), d(NULL), Trusted(CHECK_TRUST) + /*}}}*/ +// ReleaseIndex Con- and Destructors /*{{{*/ +debReleaseIndex::debReleaseIndex(std::string const &URI, std::string const &Dist) : + metaIndex(URI, Dist, "deb"), d(new debReleaseIndexPrivate()), Trusted(CHECK_TRUST) {} - -debReleaseIndex::debReleaseIndex(string const &URI, string const &Dist, bool const Trusted) : - metaIndex(URI, Dist, "deb"), d(NULL) { +debReleaseIndex::debReleaseIndex(std::string const &URI, std::string const &Dist, bool const Trusted) : + metaIndex(URI, Dist, "deb"), d(new debReleaseIndexPrivate()) { SetTrusted(Trusted); } - debReleaseIndex::~debReleaseIndex() { - for (map >::const_iterator A = ArchEntries.begin(); - A != ArchEntries.end(); ++A) - for (vector::const_iterator S = A->second.begin(); - S != A->second.end(); ++S) - delete *S; + if (d != NULL) + delete d; } - -template -void foreachTarget(std::string const &URI, std::string const &Dist, - std::map > const &ArchEntries, - CallC &Call) + /*}}}*/ +// ReleaseIndex::GetIndexTargets /*{{{*/ +static void GetIndexTargetsFor(char const * const Type, std::string const &URI, std::string const &Dist, + std::vector const &entries, + std::vector &IndexTargets) { bool const flatArchive = (Dist[Dist.length() - 1] == '/'); std::string baseURI = URI; @@ -118,148 +128,101 @@ void foreachTarget(std::string const &URI, std::string const &Dist, baseURI += "dists/" + Dist + "/"; std::string const Release = (Dist == "/") ? "" : Dist; std::string const Site = ::URI::ArchiveOnly(URI); - std::vector lang = APT::Configuration::getLanguages(true); - if (lang.empty()) - lang.push_back("none"); - map >::const_iterator const src = ArchEntries.find("source"); - if (src != ArchEntries.end()) + + for (std::vector::const_iterator E = entries.begin(); E != entries.end(); ++E) { - std::vector const targets = _config->FindVector("APT::Acquire::Targets::deb-src", "", true); - for (std::vector::const_iterator T = targets.begin(); T != targets.end(); ++T) + for (std::vector::const_iterator T = E->Targets.begin(); T != E->Targets.end(); ++T) { -#define APT_T_CONFIG(X) _config->Find(std::string("APT::Acquire::Targets::deb-src::") + *T + "::" + (X)) - std::string const MetaKey = APT_T_CONFIG(flatArchive ? "flatMetaKey" : "MetaKey"); - std::string const ShortDesc = APT_T_CONFIG("ShortDescription"); - std::string const LongDesc = APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); +#define APT_T_CONFIG(X) _config->Find(std::string("APT::Acquire::Targets::") + Type + "::" + *T + "::" + (X)) + std::string const tplMetaKey = APT_T_CONFIG(flatArchive ? "flatMetaKey" : "MetaKey"); + std::string const tplShortDesc = APT_T_CONFIG("ShortDescription"); + std::string const tplLongDesc = APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); bool const IsOptional = _config->FindB(std::string("APT::Acquire::Targets::deb-src::") + *T + "::Optional", true); #undef APT_T_CONFIG - if (MetaKey.empty()) + if (tplMetaKey.empty()) continue; - vector const SectionEntries = src->second; - for (vector::const_iterator I = SectionEntries.begin(); - I != SectionEntries.end(); ++I) + for (std::vector::const_iterator L = E->Languages.begin(); L != E->Languages.end(); ++L) { - for (vector::const_iterator l = lang.begin(); l != lang.end(); ++l) + if (*L == "none" && tplMetaKey.find("$(LANGUAGE)") != std::string::npos) + continue; + + for (std::vector::const_iterator A = E->Architectures.begin(); A != E->Architectures.end(); ++A) { - if (*l == "none" && MetaKey.find("$(LANGUAGE)") != std::string::npos) - continue; std::map Options; Options.insert(std::make_pair("SITE", Site)); Options.insert(std::make_pair("RELEASE", Release)); - if (MetaKey.find("$(COMPONENT)") != std::string::npos) - Options.insert(std::make_pair("COMPONENT", (*I)->Section)); - if (MetaKey.find("$(LANGUAGE)") != std::string::npos) - Options.insert(std::make_pair("LANGUAGE", *l)); - Options.insert(std::make_pair("ARCHITECTURE", "source")); + if (tplMetaKey.find("$(COMPONENT)") != std::string::npos) + Options.insert(std::make_pair("COMPONENT", E->Name)); + if (tplMetaKey.find("$(LANGUAGE)") != std::string::npos) + Options.insert(std::make_pair("LANGUAGE", *L)); + if (tplMetaKey.find("$(ARCHITECTURE)") != std::string::npos) + Options.insert(std::make_pair("ARCHITECTURE", *A)); Options.insert(std::make_pair("BASE_URI", baseURI)); Options.insert(std::make_pair("REPO_URI", URI)); Options.insert(std::make_pair("TARGET_OF", "deb-src")); Options.insert(std::make_pair("CREATED_BY", *T)); - Call(MetaKey, ShortDesc, LongDesc, IsOptional, Options); - if (MetaKey.find("$(LANGUAGE)") == std::string::npos) + std::string MetaKey = tplMetaKey; + std::string ShortDesc = tplShortDesc; + std::string LongDesc = tplLongDesc; + for (std::map::const_iterator O = Options.begin(); O != Options.end(); ++O) + { + MetaKey = SubstVar(MetaKey, std::string("$(") + O->first + ")", O->second); + ShortDesc = SubstVar(ShortDesc, std::string("$(") + O->first + ")", O->second); + LongDesc = SubstVar(LongDesc, std::string("$(") + O->first + ")", O->second); + } + IndexTarget Target( + MetaKey, + ShortDesc, + LongDesc, + Options.find("BASE_URI")->second + MetaKey, + IsOptional, + Options + ); + IndexTargets.push_back(Target); + + if (tplMetaKey.find("$(ARCHITECTURE)") == std::string::npos) break; - } - - if (MetaKey.find("$(COMPONENT)") == std::string::npos) - break; - } - } - } - - std::vector const targets = _config->FindVector("APT::Acquire::Targets::deb", "", true); - for (std::vector::const_iterator T = targets.begin(); T != targets.end(); ++T) - { -#define APT_T_CONFIG(X) _config->Find(std::string("APT::Acquire::Targets::deb::") + *T + "::" + (X)) - std::string const MetaKey = APT_T_CONFIG(flatArchive ? "flatMetaKey" : "MetaKey"); - std::string const ShortDesc = APT_T_CONFIG("ShortDescription"); - std::string const LongDesc = APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); - bool const IsOptional = _config->FindB(std::string("APT::Acquire::Targets::deb::") + *T + "::Optional", true); -#undef APT_T_CONFIG - if (MetaKey.empty()) - continue; - - for (map >::const_iterator a = ArchEntries.begin(); - a != ArchEntries.end(); ++a) - { - if (a->first == "source") - continue; - - for (vector ::const_iterator I = a->second.begin(); - I != a->second.end(); ++I) { - - for (vector::const_iterator l = lang.begin(); l != lang.end(); ++l) - { - if (*l == "none" && MetaKey.find("$(LANGUAGE)") != std::string::npos) - continue; - - std::map Options; - Options.insert(std::make_pair("SITE", Site)); - Options.insert(std::make_pair("RELEASE", Release)); - if (MetaKey.find("$(COMPONENT)") != std::string::npos) - Options.insert(std::make_pair("COMPONENT", (*I)->Section)); - if (MetaKey.find("$(LANGUAGE)") != std::string::npos) - Options.insert(std::make_pair("LANGUAGE", *l)); - if (MetaKey.find("$(ARCHITECTURE)") != std::string::npos) - Options.insert(std::make_pair("ARCHITECTURE", a->first)); - Options.insert(std::make_pair("BASE_URI", baseURI)); - Options.insert(std::make_pair("REPO_URI", URI)); - Options.insert(std::make_pair("TARGET_OF", "deb")); - Options.insert(std::make_pair("CREATED_BY", *T)); - Call(MetaKey, ShortDesc, LongDesc, IsOptional, Options); - if (MetaKey.find("$(LANGUAGE)") == std::string::npos) - break; } - if (MetaKey.find("$(COMPONENT)") == std::string::npos) + if (tplMetaKey.find("$(LANGUAGE)") == std::string::npos) break; + } - if (MetaKey.find("$(ARCHITECTURE)") == std::string::npos) - break; } } } - - -struct ComputeIndexTargetsClass -{ - vector IndexTargets; - - void operator()(std::string MetaKey, std::string ShortDesc, std::string LongDesc, - bool const IsOptional, std::map Options) - { - for (std::map::const_iterator O = Options.begin(); O != Options.end(); ++O) - { - MetaKey = SubstVar(MetaKey, std::string("$(") + O->first + ")", O->second); - ShortDesc = SubstVar(ShortDesc, std::string("$(") + O->first + ")", O->second); - LongDesc = SubstVar(LongDesc, std::string("$(") + O->first + ")", O->second); - } - IndexTarget Target( - MetaKey, - ShortDesc, - LongDesc, - Options.find("BASE_URI")->second + MetaKey, - IsOptional, - Options - ); - IndexTargets.push_back(Target); - } -}; - std::vector debReleaseIndex::GetIndexTargets() const { - ComputeIndexTargetsClass comp; - foreachTarget(URI, Dist, ArchEntries, comp); - return comp.IndexTargets; + std::vector IndexTargets; + GetIndexTargetsFor("deb-src", URI, Dist, d->DebSrcEntries, IndexTargets); + GetIndexTargetsFor("deb", URI, Dist, d->DebEntries, IndexTargets); + return IndexTargets; } + /*}}}*/ +void debReleaseIndex::AddComponent(bool const isSrc, std::string const &Name,/*{{{*/ + std::vector const &Targets, + std::vector const &Architectures, + std::vector Languages) +{ + if (Languages.empty() == true) + Languages.push_back("none"); + debReleaseIndexPrivate::debSectionEntry const entry = { + Name, Targets, Architectures, Languages + }; + if (isSrc) + d->DebSrcEntries.push_back(entry); + else + d->DebEntries.push_back(entry); +} + /*}}}*/ - /*}}}*/ -bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const +bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const/*{{{*/ { indexRecords * const iR = new indexRecords(Dist); if (Trusted == ALWAYS_TRUSTED) @@ -282,7 +245,8 @@ bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const return true; } - + /*}}}*/ +// ReleaseIndex::*Trusted setters and checkers /*{{{*/ void debReleaseIndex::SetTrusted(bool const Trusted) { if (Trusted == true) @@ -290,7 +254,6 @@ void debReleaseIndex::SetTrusted(bool const Trusted) else this->Trusted = NEVER_TRUSTED; } - bool debReleaseIndex::IsTrusted() const { if (Trusted == ALWAYS_TRUSTED) @@ -303,19 +266,13 @@ bool debReleaseIndex::IsTrusted() const if(URI.substr(0,strlen("cdrom:")) == "cdrom:") return true; - string VerifiedSigFile = _config->FindDir("Dir::State::lists") + - URItoFileName(MetaIndexURI("Release")) + ".gpg"; - - if (FileExists(VerifiedSigFile)) + if (FileExists(MetaIndexFile("Release.gpg"))) return true; - VerifiedSigFile = _config->FindDir("Dir::State::lists") + - URItoFileName(MetaIndexURI("InRelease")); - - return FileExists(VerifiedSigFile); + return FileExists(MetaIndexFile("InRelease")); } - -std::vector *debReleaseIndex::GetIndexFiles() + /*}}}*/ +std::vector *debReleaseIndex::GetIndexFiles() /*{{{*/ { if (Indexes != NULL) return Indexes; @@ -335,23 +292,9 @@ std::vector *debReleaseIndex::GetIndexFiles() } return Indexes; } + /*}}}*/ -void debReleaseIndex::PushSectionEntry(vector const &Archs, const debSectionEntry *Entry) { - for (vector::const_iterator a = Archs.begin(); - a != Archs.end(); ++a) - ArchEntries[*a].push_back(new debSectionEntry(Entry->Section, Entry->IsSrc)); - delete Entry; -} - -void debReleaseIndex::PushSectionEntry(string const &Arch, const debSectionEntry *Entry) { - ArchEntries[Arch].push_back(Entry); -} - -debReleaseIndex::debSectionEntry::debSectionEntry (string const &Section, - bool const &IsSrc): Section(Section), IsSrc(IsSrc) -{} - -static bool ReleaseFileName(debReleaseIndex const * const That, std::string &ReleaseFile) +static bool ReleaseFileName(debReleaseIndex const * const That, std::string &ReleaseFile)/*{{{*/ { ReleaseFile = That->MetaIndexFile("InRelease"); bool releaseExists = false; @@ -365,7 +308,7 @@ static bool ReleaseFileName(debReleaseIndex const * const That, std::string &Rel } return releaseExists; } - + /*}}}*/ bool debReleaseIndex::Merge(pkgCacheGenerator &Gen,OpProgress * /*Prog*/) const/*{{{*/ { std::string ReleaseFile; @@ -459,46 +402,46 @@ pkgCache::RlsFileIterator debReleaseIndex::FindInCache(pkgCache &Cache, bool con } /*}}}*/ -debDebFileMetaIndex::~debDebFileMetaIndex() {} - -class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type +static std::vector parsePlusMinusOptions(std::string const &Name, /*{{{*/ + std::map const &Options, std::vector const &defaultValues) { - protected: + std::map::const_iterator val = Options.find(Name); + std::vector Values; + if (val != Options.end()) + Values = VectorizeString(val->second, ','); + else + Values = defaultValues; - bool CreateItemInternal(vector &List, string const &URI, - string const &Dist, string const &Section, - bool const &IsSrc, map const &Options) const + if ((val = Options.find(Name + "+")) != Options.end()) { - // parse arch=, arch+= and arch-= settings - map::const_iterator arch = Options.find("arch"); - vector Archs; - if (arch != Options.end()) - Archs = VectorizeString(arch->second, ','); - else - Archs = APT::Configuration::getArchitectures(); - - if ((arch = Options.find("arch+")) != Options.end()) - { - std::vector const plusArch = VectorizeString(arch->second, ','); - for (std::vector::const_iterator plus = plusArch.begin(); plus != plusArch.end(); ++plus) - if (std::find(Archs.begin(), Archs.end(), *plus) == Archs.end()) - Archs.push_back(*plus); - } - if ((arch = Options.find("arch-")) != Options.end()) + std::vector const plusArch = VectorizeString(val->second, ','); + for (std::vector::const_iterator plus = plusArch.begin(); plus != plusArch.end(); ++plus) + if (std::find(Values.begin(), Values.end(), *plus) == Values.end()) + Values.push_back(*plus); + } + if ((val = Options.find(Name + "-")) != Options.end()) + { + std::vector const minusArch = VectorizeString(val->second, ','); + for (std::vector::const_iterator minus = minusArch.begin(); minus != minusArch.end(); ++minus) { - std::vector const minusArch = VectorizeString(arch->second, ','); - for (std::vector::const_iterator minus = minusArch.begin(); minus != minusArch.end(); ++minus) - { - std::vector::iterator kill = std::find(Archs.begin(), Archs.end(), *minus); - if (kill != Archs.end()) - Archs.erase(kill); - } + std::vector::iterator kill = std::find(Values.begin(), Values.end(), *minus); + if (kill != Values.end()) + Values.erase(kill); } + } + return Values; +} + /*}}}*/ +class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type /*{{{*/ +{ + protected: - map::const_iterator const trusted = Options.find("trusted"); - + bool CreateItemInternal(std::vector &List, std::string const &URI, + std::string const &Dist, std::string const &Section, + bool const &IsSrc, std::map const &Options) const + { debReleaseIndex *Deb = NULL; - for (vector::const_iterator I = List.begin(); + for (std::vector::const_iterator I = List.begin(); I != List.end(); ++I) { // We only worry about debian entries here @@ -523,87 +466,86 @@ class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type List.push_back(Deb); } - if (IsSrc == true) - Deb->PushSectionEntry ("source", new debReleaseIndex::debSectionEntry(Section, IsSrc)); - else - { - if (Dist[Dist.size() - 1] == '/') - Deb->PushSectionEntry ("any", new debReleaseIndex::debSectionEntry(Section, IsSrc)); - else - Deb->PushSectionEntry (Archs, new debReleaseIndex::debSectionEntry(Section, IsSrc)); - } + Deb->AddComponent( + IsSrc, + Section, + parsePlusMinusOptions("target", Options, _config->FindVector(std::string("APT::Acquire::Targets::") + Name, "", true)), + parsePlusMinusOptions("arch", Options, APT::Configuration::getArchitectures()), + parsePlusMinusOptions("lang", Options, APT::Configuration::getLanguages(true)) + ); + std::map::const_iterator const trusted = Options.find("trusted"); if (trusted != Options.end()) Deb->SetTrusted(StringToBool(trusted->second, false)); return true; } -}; - -debDebFileMetaIndex::debDebFileMetaIndex(std::string const &DebFile) - : metaIndex(DebFile, "local-uri", "deb-dist"), d(NULL), DebFile(DebFile) -{ - DebIndex = new debDebPkgFileIndex(DebFile); - Indexes = new vector(); - Indexes->push_back(DebIndex); -} - -class APT_HIDDEN debSLTypeDeb : public debSLTypeDebian + debSLTypeDebian(char const * const Name, char const * const Label) : Type(Name, Label) + { + } +}; + /*}}}*/ +class APT_HIDDEN debSLTypeDeb : public debSLTypeDebian /*{{{*/ { public: - bool CreateItem(vector &List, string const &URI, - string const &Dist, string const &Section, - std::map const &Options) const + bool CreateItem(std::vector &List, std::string const &URI, + std::string const &Dist, std::string const &Section, + std::map const &Options) const { return CreateItemInternal(List, URI, Dist, Section, false, Options); } - debSLTypeDeb() + debSLTypeDeb() : debSLTypeDebian("deb", "Debian binary tree") { - Name = "deb"; - Label = "Standard Debian binary tree"; - } + } }; - -class APT_HIDDEN debSLTypeDebSrc : public debSLTypeDebian + /*}}}*/ +class APT_HIDDEN debSLTypeDebSrc : public debSLTypeDebian /*{{{*/ { public: - bool CreateItem(vector &List, string const &URI, - string const &Dist, string const &Section, - std::map const &Options) const + bool CreateItem(std::vector &List, std::string const &URI, + std::string const &Dist, std::string const &Section, + std::map const &Options) const { return CreateItemInternal(List, URI, Dist, Section, true, Options); } - - debSLTypeDebSrc() + + debSLTypeDebSrc() : debSLTypeDebian("deb-src", "Debian source tree") { - Name = "deb-src"; - Label = "Standard Debian source tree"; - } + } }; + /*}}}*/ -class APT_HIDDEN debSLTypeDebFile : public pkgSourceList::Type +debDebFileMetaIndex::debDebFileMetaIndex(std::string const &DebFile) /*{{{*/ + : metaIndex(DebFile, "local-uri", "deb-dist"), d(NULL), DebFile(DebFile) +{ + DebIndex = new debDebPkgFileIndex(DebFile); + Indexes = new std::vector(); + Indexes->push_back(DebIndex); +} +debDebFileMetaIndex::~debDebFileMetaIndex() {} + /*}}}*/ +class APT_HIDDEN debSLTypeDebFile : public pkgSourceList::Type /*{{{*/ { public: - bool CreateItem(vector &List, string const &URI, - string const &/*Dist*/, string const &/*Section*/, - std::map const &/*Options*/) const + bool CreateItem(std::vector &List, std::string const &URI, + std::string const &/*Dist*/, std::string const &/*Section*/, + std::map const &/*Options*/) const { metaIndex *mi = new debDebFileMetaIndex(URI); List.push_back(mi); return true; } - - debSLTypeDebFile() + + debSLTypeDebFile() : Type("deb-file", "Debian local deb file") { - Name = "deb-file"; - Label = "Debian Deb File"; - } + } }; + /*}}}*/ APT_HIDDEN debSLTypeDeb _apt_DebType; APT_HIDDEN debSLTypeDebSrc _apt_DebSrcType; diff --git a/apt-pkg/deb/debmetaindex.h b/apt-pkg/deb/debmetaindex.h index 648c22436..9b60b6137 100644 --- a/apt-pkg/deb/debmetaindex.h +++ b/apt-pkg/deb/debmetaindex.h @@ -1,4 +1,3 @@ -// ijones, walters #ifndef PKGLIB_DEBMETAINDEX_H #define PKGLIB_DEBMETAINDEX_H @@ -22,26 +21,20 @@ class debDebPkgFileIndex; class IndexTarget; class pkgCacheGenerator; class OpProgress; +class debReleaseIndexPrivate; -class APT_HIDDEN debReleaseIndex : public metaIndex { - public: +class APT_HIDDEN debReleaseIndex : public metaIndex +{ + debReleaseIndexPrivate * const d; - class debSectionEntry - { - public: - debSectionEntry (std::string const &Section, bool const &IsSrc); - std::string const Section; - bool const IsSrc; - }; - - private: - /** \brief dpointer placeholder (for later in case we need it) */ - void * const d; - std::map > ArchEntries; enum APT_HIDDEN { ALWAYS_TRUSTED, NEVER_TRUSTED, CHECK_TRUST } Trusted; public: + APT_HIDDEN std::string MetaIndexInfo(const char *Type) const; + APT_HIDDEN std::string MetaIndexFile(const char *Types) const; + APT_HIDDEN std::string MetaIndexURI(const char *Type) const; + debReleaseIndex(std::string const &URI, std::string const &Dist); debReleaseIndex(std::string const &URI, std::string const &Dist, bool const Trusted); virtual ~debReleaseIndex(); @@ -54,22 +47,17 @@ class APT_HIDDEN debReleaseIndex : public metaIndex { virtual pkgCache::RlsFileIterator FindInCache(pkgCache &Cache, bool const ModifyCheck) const; virtual bool Merge(pkgCacheGenerator &Gen,OpProgress *Prog) const; - std::string MetaIndexInfo(const char *Type) const; - std::string MetaIndexFile(const char *Types) const; - std::string MetaIndexURI(const char *Type) const; - -#if APT_PKG_ABI >= 413 - virtual -#endif - std::string LocalFileName() const; + virtual std::string LocalFileName() const; virtual std::vector *GetIndexFiles(); void SetTrusted(bool const Trusted); virtual bool IsTrusted() const; - void PushSectionEntry(std::vector const &Archs, const debSectionEntry *Entry); - void PushSectionEntry(std::string const &Arch, const debSectionEntry *Entry); + void AddComponent(bool const isSrc, std::string const &Name, + std::vector const &Targets, + std::vector const &Architectures, + std::vector Languages); }; class APT_HIDDEN debDebFileMetaIndex : public metaIndex diff --git a/apt-pkg/sourcelist.cc b/apt-pkg/sourcelist.cc index 99b646513..6ef99863d 100644 --- a/apt-pkg/sourcelist.cc +++ b/apt-pkg/sourcelist.cc @@ -37,25 +37,26 @@ using namespace std; // Global list of Items supported -static pkgSourceList::Type *ItmList[10]; +static pkgSourceList::Type *ItmList[10]; pkgSourceList::Type **pkgSourceList::Type::GlobalList = ItmList; unsigned long pkgSourceList::Type::GlobalListLen = 0; // Type::Type - Constructor /*{{{*/ // --------------------------------------------------------------------- /* Link this to the global list of items*/ -pkgSourceList::Type::Type() : Name(NULL), Label(NULL) +pkgSourceList::Type::Type(char const * const pName, char const * const pLabel) : Name(pName), Label(pLabel) { ItmList[GlobalListLen] = this; - GlobalListLen++; + ++GlobalListLen; } +pkgSourceList::Type::~Type() {} /*}}}*/ // Type::GetType - Get a specific meta for a given type /*{{{*/ // --------------------------------------------------------------------- /* */ pkgSourceList::Type *pkgSourceList::Type::GetType(const char *Type) { - for (unsigned I = 0; I != GlobalListLen; I++) + for (unsigned I = 0; I != GlobalListLen; ++I) if (strcmp(GlobalList[I]->Name,Type) == 0) return GlobalList[I]; return 0; @@ -91,23 +92,26 @@ bool pkgSourceList::Type::ParseStanza(vector &List, string Enabled = Tags.FindS("Enabled"); if (Enabled.size() > 0 && StringToBool(Enabled) == false) return true; - - // Define external/internal options - const char* option_deb822[] = { - "Architectures", "Architectures-Add", "Architectures-Remove", "Trusted", - }; - const char* option_internal[] = { - "arch", "arch+", "arch-", "trusted", - }; - for (unsigned int j=0; j < sizeof(option_deb822)/sizeof(char*); j++) - if (Tags.Exists(option_deb822[j])) + + std::map mapping; +#define APT_PLUSMINUS(X, Y) \ + mapping.insert(std::make_pair(X, Y)); \ + mapping.insert(std::make_pair(X "Add", Y "+")); \ + mapping.insert(std::make_pair(X "Remove", Y "-")) + APT_PLUSMINUS("Architectures", "arch"); + APT_PLUSMINUS("Languages", "lang"); + APT_PLUSMINUS("Targets", "target"); +#undef APT_PLUSMINUS + mapping.insert(std::make_pair("Trusted", "trusted")); + for (std::map::const_iterator m = mapping.begin(); m != mapping.end(); ++m) + if (Tags.Exists(m->first)) { // for deb822 the " " is the delimiter, but the backend expects "," - std::string option = Tags.FindS(option_deb822[j]); + std::string option = Tags.FindS(m->first); std::replace(option.begin(), option.end(), ' ', ','); - Options[option_internal[j]] = option; + Options[m->second] = option; } - + // now create one item per suite/section string Suite = Tags.FindS("Suites"); Suite = SubstVar(Suite,"$(ARCH)",_config->Find("APT::Architecture")); @@ -209,7 +213,7 @@ bool pkgSourceList::Type::ParseLine(vector &List, if (FixupURI(URI) == false) return _error->Error(_("Malformed line %lu in source list %s (URI parse)"),CurLine,File.c_str()); - + // Check for an absolute dists specification. if (Dist.empty() == false && Dist[Dist.size() - 1] == '/') { diff --git a/apt-pkg/sourcelist.h b/apt-pkg/sourcelist.h index e17ad6a9a..4f42b3e91 100644 --- a/apt-pkg/sourcelist.h +++ b/apt-pkg/sourcelist.h @@ -66,8 +66,8 @@ class pkgSourceList static unsigned long GlobalListLen; static Type *GetType(const char *Type) APT_PURE; - const char *Name; - const char *Label; + char const * const Name; + char const * const Label; bool FixupURI(std::string &URI) const; virtual bool ParseStanza(std::vector &List, @@ -80,8 +80,8 @@ class pkgSourceList virtual bool CreateItem(std::vector &List,std::string const &URI, std::string const &Dist,std::string const &Section, std::map const &Options) const = 0; - Type(); - virtual ~Type() {}; + Type(char const * const Name, char const * const Label); + virtual ~Type(); }; typedef std::vector::const_iterator const_iterator; diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc index 632c7cfea..500a0a3c5 100644 --- a/cmdline/apt-get.cc +++ b/cmdline/apt-get.cc @@ -1045,7 +1045,7 @@ static bool DoBuildDep(CommandLine &CmdL) { ioprintf(c1out, _("Note, using directory '%s' to get the build dependencies\n"), *I); // FIXME: how can we make this more elegant? - std::string TypeName = "debian/control File Source Index"; + std::string TypeName = "Debian control file"; pkgIndexFile::Type *Type = pkgIndexFile::Type::GetType(TypeName.c_str()); if(Type != NULL) LastOwner = Last = Type->CreateSrcPkgParser(*I); @@ -1056,7 +1056,7 @@ static bool DoBuildDep(CommandLine &CmdL) ioprintf(c1out, _("Note, using file '%s' to get the build dependencies\n"), *I); // see if we can get a parser for this pkgIndexFile type - string TypeName = flExtension(*I) + " File Source Index"; + string TypeName = "Debian " + flExtension(*I) + " file"; pkgIndexFile::Type *Type = pkgIndexFile::Type::GetType(TypeName.c_str()); if(Type != NULL) LastOwner = Last = Type->CreateSrcPkgParser(*I); diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index da4f571b5..6ebba3528 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -137,9 +137,25 @@ can be used to specify for which architectures information should be downloaded. If this option is not set all architectures defined by the APT::Architectures option will be downloaded. + arch+=arch1,arch2,… and arch-=arch1,arch2,… which can be used to add/remove architectures from the set which will be downloaded. + + lang=lang1,lang2,…, + lang+=lang1,lang2,… and + lang-=lang1,lang2,… functioning in + the same way as the arch-options described before. They can be used to specify for + which languages apt will acquire metadata, like translated package descriptions, for. If not specified, the + default set is defined by the Acquire::Languages config option. + + target=target1,target2,…, + target+=target1,target2,… and + target-=target1,target2,… again functioning in + the same way as the arch-options described before. They can be used to specify which + targets apt will try to acquire from this source. If not specified, the default set is defined by + the APT::Acquire::Targets configuration scope. + trusted=yes can be set to indicate that packages from this source are always authenticated even if the Release file is not signed or the signature can't be checked. This disables parts of &apt-secure; diff --git a/test/integration/test-bug-632221-cross-dependency-satisfaction b/test/integration/test-bug-632221-cross-dependency-satisfaction index 563821173..12704e5de 100755 --- a/test/integration/test-bug-632221-cross-dependency-satisfaction +++ b/test/integration/test-bug-632221-cross-dependency-satisfaction @@ -167,17 +167,17 @@ Conf libfwibble-dev (1.0 unstable [armel])' aptget build-dep apt -s testsuccessequal 'Reading package lists... Building dependency tree... The following NEW packages will be installed: - amdboot:amd64 cool doxygen foreigner libc6:amd64 libc6 libc6-dev:amd64 - libc6-dev libfwibble-dev:amd64 libfwibble1:amd64 linux-stuff:amd64 + amdboot:amd64 cool doxygen foreigner libc6 libc6:amd64 libc6-dev + libc6-dev:amd64 libfwibble-dev:amd64 libfwibble1:amd64 linux-stuff:amd64 0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded. Inst amdboot:amd64 (1.0 unstable [amd64]) Inst cool (1.0 unstable [armel]) Inst doxygen (1.0 unstable [armel]) Inst foreigner (1.0 unstable [armel]) -Inst libc6:amd64 (1.0 unstable [amd64]) Inst libc6 (1.0 unstable [armel]) -Inst libc6-dev:amd64 (1.0 unstable [amd64]) +Inst libc6:amd64 (1.0 unstable [amd64]) Inst libc6-dev (1.0 unstable [armel]) +Inst libc6-dev:amd64 (1.0 unstable [amd64]) Inst libfwibble1:amd64 (1.0 unstable [amd64]) Inst libfwibble-dev:amd64 (1.0 unstable [amd64]) Inst linux-stuff:amd64 (1.0 unstable [amd64]) @@ -185,10 +185,10 @@ Conf amdboot:amd64 (1.0 unstable [amd64]) Conf cool (1.0 unstable [armel]) Conf doxygen (1.0 unstable [armel]) Conf foreigner (1.0 unstable [armel]) -Conf libc6:amd64 (1.0 unstable [amd64]) Conf libc6 (1.0 unstable [armel]) -Conf libc6-dev:amd64 (1.0 unstable [amd64]) +Conf libc6:amd64 (1.0 unstable [amd64]) Conf libc6-dev (1.0 unstable [armel]) +Conf libc6-dev:amd64 (1.0 unstable [amd64]) Conf libfwibble1:amd64 (1.0 unstable [amd64]) Conf libfwibble-dev:amd64 (1.0 unstable [amd64]) Conf linux-stuff:amd64 (1.0 unstable [amd64])' aptget build-dep apt -s -a amd64 @@ -275,24 +275,24 @@ Conf libfwibble-dev (1.0 unstable [armel])' aptget build-dep apt -s testsuccessequal 'Reading package lists... Building dependency tree... The following NEW packages will be installed: - amdboot:amd64 doxygen libc6:amd64 libc6 libc6-dev:amd64 libc6-dev + amdboot:amd64 doxygen libc6 libc6:amd64 libc6-dev libc6-dev:amd64 libfwibble-dev:amd64 libfwibble1:amd64 linux-stuff:amd64 0 upgraded, 9 newly installed, 0 to remove and 2 not upgraded. Inst amdboot:amd64 (1.0 unstable [amd64]) Inst doxygen (1.0 unstable [armel]) -Inst libc6:amd64 (1.0 unstable [amd64]) Inst libc6 (1.0 unstable [armel]) -Inst libc6-dev:amd64 (1.0 unstable [amd64]) +Inst libc6:amd64 (1.0 unstable [amd64]) Inst libc6-dev (1.0 unstable [armel]) +Inst libc6-dev:amd64 (1.0 unstable [amd64]) Inst libfwibble1:amd64 (1.0 unstable [amd64]) Inst libfwibble-dev:amd64 (1.0 unstable [amd64]) Inst linux-stuff:amd64 (1.0 unstable [amd64]) Conf amdboot:amd64 (1.0 unstable [amd64]) Conf doxygen (1.0 unstable [armel]) -Conf libc6:amd64 (1.0 unstable [amd64]) Conf libc6 (1.0 unstable [armel]) -Conf libc6-dev:amd64 (1.0 unstable [amd64]) +Conf libc6:amd64 (1.0 unstable [amd64]) Conf libc6-dev (1.0 unstable [armel]) +Conf libc6-dev:amd64 (1.0 unstable [amd64]) Conf libfwibble1:amd64 (1.0 unstable [amd64]) Conf libfwibble-dev:amd64 (1.0 unstable [amd64]) Conf linux-stuff:amd64 (1.0 unstable [amd64])' aptget build-dep apt -s -a amd64 diff --git a/test/integration/test-ignore-provides-if-versioned-breaks b/test/integration/test-ignore-provides-if-versioned-breaks index 20424b942..4d6114637 100755 --- a/test/integration/test-ignore-provides-if-versioned-breaks +++ b/test/integration/test-ignore-provides-if-versioned-breaks @@ -134,17 +134,17 @@ Conf foo-same-provider (1.0 unstable [i386])' aptget install foo-same-provider f testsuccessequal 'Reading package lists... Building dependency tree... The following extra packages will be installed: - foo-same:amd64 foo-same + foo-same foo-same:amd64 The following NEW packages will be installed: foo-same-breaker-3 foo-same-provider The following packages will be upgraded: - foo-same:amd64 foo-same + foo-same foo-same:amd64 2 upgraded, 2 newly installed, 0 to remove and 2 not upgraded. -Inst foo-same:amd64 [2.0] (4.0 unstable [amd64]) [foo-same:amd64 on foo-same:i386] [foo-same:i386 on foo-same:amd64] [foo-same:i386 ] -Inst foo-same [2.0] (4.0 unstable [i386]) +Inst foo-same [2.0] (4.0 unstable [i386]) [foo-same:i386 on foo-same:amd64] [foo-same:amd64 on foo-same:i386] [foo-same:amd64 ] +Inst foo-same:amd64 [2.0] (4.0 unstable [amd64]) Inst foo-same-breaker-3 (1.0 unstable [i386]) Inst foo-same-provider (1.0 unstable [i386]) -Conf foo-same (4.0 unstable [i386]) Conf foo-same:amd64 (4.0 unstable [amd64]) +Conf foo-same (4.0 unstable [i386]) Conf foo-same-breaker-3 (1.0 unstable [i386]) Conf foo-same-provider (1.0 unstable [i386])' aptget install foo-same-provider foo-same-breaker-3 -s diff --git a/test/integration/test-ignore-provides-if-versioned-conflicts b/test/integration/test-ignore-provides-if-versioned-conflicts index a781d8e44..6a0c924e2 100755 --- a/test/integration/test-ignore-provides-if-versioned-conflicts +++ b/test/integration/test-ignore-provides-if-versioned-conflicts @@ -134,17 +134,17 @@ Conf foo-same-provider (1.0 unstable [i386])' aptget install foo-same-provider f testsuccessequal 'Reading package lists... Building dependency tree... The following extra packages will be installed: - foo-same:amd64 foo-same + foo-same foo-same:amd64 The following NEW packages will be installed: foo-same-breaker-3 foo-same-provider The following packages will be upgraded: - foo-same:amd64 foo-same + foo-same foo-same:amd64 2 upgraded, 2 newly installed, 0 to remove and 2 not upgraded. -Inst foo-same:amd64 [2.0] (4.0 unstable [amd64]) [foo-same:amd64 on foo-same:i386] [foo-same:i386 on foo-same:amd64] [foo-same:i386 ] -Inst foo-same [2.0] (4.0 unstable [i386]) +Inst foo-same [2.0] (4.0 unstable [i386]) [foo-same:i386 on foo-same:amd64] [foo-same:amd64 on foo-same:i386] [foo-same:amd64 ] +Inst foo-same:amd64 [2.0] (4.0 unstable [amd64]) Inst foo-same-breaker-3 (1.0 unstable [i386]) Inst foo-same-provider (1.0 unstable [i386]) -Conf foo-same (4.0 unstable [i386]) Conf foo-same:amd64 (4.0 unstable [amd64]) +Conf foo-same (4.0 unstable [i386]) Conf foo-same-breaker-3 (1.0 unstable [i386]) Conf foo-same-provider (1.0 unstable [i386])' aptget install foo-same-provider foo-same-breaker-3 -s diff --git a/test/integration/test-sourceslist-lang-plusminus-options b/test/integration/test-sourceslist-lang-plusminus-options new file mode 100755 index 000000000..28d5f9e06 --- /dev/null +++ b/test/integration/test-sourceslist-lang-plusminus-options @@ -0,0 +1,87 @@ +#!/bin/sh +set -e + +TESTDIR=$(readlink -f $(dirname $0)) +. $TESTDIR/framework +setupenvironment +configarchitecture 'native' + +testlangs() { + msgtest 'Test acquired languages for' "$1" + local LANGS="$2" + shift 2 + rm -f gotlangs.list + aptget files --no-release-info 'Created-By: Translations' "$@" --format '$(LANGUAGE)' | sort -u > gotlangs.list + if [ -z "$LANGS" ]; then + echo -n | tr ',' '\n' | sort | checkdiff - gotlangs.list && msgpass || msgfail + else + echo -n "$LANGS" | tr ',' '\n' | sort | checkdiff - gotlangs.list && msgpass || msgfail + fi +} +echo 'deb http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'default' 'en' + +echo 'Acquire::Languages "environment,en";' > rootdir/etc/apt/apt.conf.d/langs.conf +testlangs 'default config' 'en' + +echo 'Acquire::Languages "en,en,en";' > rootdir/etc/apt/apt.conf.d/langs.conf +testlangs 'duplicated config' 'en' + +echo 'Acquire::Languages "none";' > rootdir/etc/apt/apt.conf.d/langs.conf +testlangs 'none config' '' + +echo 'Acquire::Languages "en,none,de,de_DE";' > rootdir/etc/apt/apt.conf.d/langs.conf +testlangs 'english + german config' 'en,de,de_DE' + +echo 'deb [lang=pt] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang=pt' 'pt' + +echo 'deb [lang=en] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang=en' 'en' + +echo 'deb [lang=de_DE] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang=de_DE' 'de_DE' + +echo 'deb [lang=none] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang=none' '' +testequal 'amd64' aptget files --no-release-info 'Created-By: Packages' --format '$(ARCHITECTURE)' + +echo 'deb [lang+=pt] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang+=pt' 'en,de,de_DE,pt' + +echo 'deb [lang+=en] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang+=en' 'en,de,de_DE' + +echo 'deb [lang+=de_DE] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang+=de_DE' 'en,de,de_DE' + +echo 'deb [lang-=pt] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang-=pt' 'en,de,de_DE' + +echo 'deb [lang-=en] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang-=en' 'de,de_DE' + +echo 'deb [lang-=de_DE] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list +testlangs 'lang-=de_DE' 'en,de' + +echo 'deb http://example.org/debian stable rocks +deb http://example.org/debian stable solid' > rootdir/etc/apt/sources.list +testlangs 'english + german config multicomponent' 'en,de,de_DE' + +echo 'deb http://example.org/debian stable rocks +deb [lang=pt] http://example.org/debian stable solid' > rootdir/etc/apt/sources.list +testlangs 'multicomponent one lang= combined' 'en,de,de_DE,pt' +testlangs 'multicomponent one lang= rocks' 'en,de,de_DE' 'Component: rocks' +testlangs 'multicomponent one lang= solid' 'pt' 'Component: solid' + +echo 'deb [lang=pt] http://example.org/debian stable rocks +deb [lang=de] http://example.org/debian stable solid' > rootdir/etc/apt/sources.list +testlangs 'multicomponent different lang= combined' 'de,pt' +testlangs 'multicomponent different lang= rocks' 'pt' 'Component: rocks' +testlangs 'multicomponent different lang= solid' 'de' 'Component: solid' + +echo 'deb [lang+=pt] http://example.org/debian stable rocks +deb [lang-=de] http://example.org/debian stable solid' > rootdir/etc/apt/sources.list +testlangs 'multicomponent different lang+-= combined' 'en,de,de_DE,pt' +testlangs 'multicomponent different lang+-= rocks' 'en,de,de_DE,pt' 'Component: rocks' +testlangs 'multicomponent different lang+-= solid' 'en,de_DE' 'Component: solid' -- cgit v1.2.3-70-g09d2 From 81460e32961bb0b9922bf8a1a27d87705d8c3e51 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Sun, 21 Jun 2015 23:12:24 +0200 Subject: bring back deb822 sources.list entries as .sources Having two different formats in the same file is very dirty and causes external tools to fail hard trying to parse them. It is probably not a good idea for them to parse them in the first place, but they do and we shouldn't break them if there is a better way. So we solve this issue for now by giving our deb822 format a new filename extension ".sources" which unsupporting applications are likely to ignore an can begin gradually moving forward rather than waiting for the unknown applications to catch up. Currently and for the forseeable future apt is going to support both with the same feature set as documented in the manpage, with the longtime plan of adopting the 'new' format as default, but that is a long way to go and might get going more from having an easier time setting options than from us pushing it explicitely. --- apt-pkg/policy.cc | 19 +- apt-pkg/sourcelist.cc | 228 ++++++++++----------- apt-pkg/sourcelist.h | 16 +- apt-pkg/tagfile.cc | 8 + apt-pkg/tagfile.h | 8 + doc/sources.list.5.xml | 337 +++++++++++++++++++++---------- test/integration/test-apt-sources-deb822 | 75 +++++-- test/libapt/file-helpers.cc | 11 +- test/libapt/sourcelist_test.cc | 15 +- vendor/README | 8 +- vendor/blankon/apt-vendor.ent | 9 + vendor/debian/apt-vendor.ent | 12 ++ vendor/debian/sources.list.in | 3 +- vendor/raspbian/apt-vendor.ent | 6 + vendor/steamos/apt-vendor.ent | 7 + vendor/steamos/sources.list.in | 4 +- vendor/tanglu/apt-vendor.ent | 6 + vendor/ubuntu/apt-vendor.ent | 13 ++ 18 files changed, 500 insertions(+), 285 deletions(-) (limited to 'doc') diff --git a/apt-pkg/policy.cc b/apt-pkg/policy.cc index cd48e040c..170da7c63 100644 --- a/apt-pkg/policy.cc +++ b/apt-pkg/policy.cc @@ -396,21 +396,6 @@ APT_PURE signed short pkgPolicy::GetPriority(pkgCache::PkgFileIterator const &Fi return PFPriority[File->ID]; } /*}}}*/ -// PreferenceSection class - Overriding the default TrimRecord method /*{{{*/ -// --------------------------------------------------------------------- -/* The preference file is a user generated file so the parser should - therefore be a bit more friendly by allowing comments and new lines - all over the place rather than forcing a special format */ -class PreferenceSection : public pkgTagSection -{ - void TrimRecord(bool /*BeforeRecord*/, const char* &End) - { - for (; Stop < End && (Stop[0] == '\n' || Stop[0] == '\r' || Stop[0] == '#'); Stop++) - if (Stop[0] == '#') - Stop = (const char*) memchr(Stop,'\n',End-Stop); - } -}; - /*}}}*/ // ReadPinDir - Load the pin files from this dir into a Policy /*{{{*/ // --------------------------------------------------------------------- /* This will load each pin file in the given dir into a Policy. If the @@ -455,8 +440,8 @@ bool ReadPinFile(pkgPolicy &Plcy,string File) pkgTagFile TF(&Fd); if (_error->PendingError() == true) return false; - - PreferenceSection Tags; + + pkgUserTagSection Tags; while (TF.Step(Tags) == true) { // can happen when there are only comments in a record diff --git a/apt-pkg/sourcelist.cc b/apt-pkg/sourcelist.cc index 6ef99863d..69f7ac043 100644 --- a/apt-pkg/sourcelist.cc +++ b/apt-pkg/sourcelist.cc @@ -82,15 +82,15 @@ bool pkgSourceList::Type::FixupURI(string &URI) const return true; } /*}}}*/ -bool pkgSourceList::Type::ParseStanza(vector &List, +bool pkgSourceList::Type::ParseStanza(vector &List, /*{{{*/ pkgTagSection &Tags, - int i, + unsigned int const i, FileFd &Fd) { map Options; string Enabled = Tags.FindS("Enabled"); - if (Enabled.size() > 0 && StringToBool(Enabled) == false) + if (Enabled.empty() == false && StringToBool(Enabled) == false) return true; std::map mapping; @@ -115,46 +115,63 @@ bool pkgSourceList::Type::ParseStanza(vector &List, // now create one item per suite/section string Suite = Tags.FindS("Suites"); Suite = SubstVar(Suite,"$(ARCH)",_config->Find("APT::Architecture")); - string const Section = Tags.FindS("Sections"); - string URIS = Tags.FindS("URIs"); + string const Component = Tags.FindS("Components"); + string const URIS = Tags.FindS("URIs"); + + std::vector const list_uris = VectorizeString(URIS, ' '); + std::vector const list_suite = VectorizeString(Suite, ' '); + std::vector const list_comp = VectorizeString(Component, ' '); + + if (list_uris.empty()) + // TRANSLATOR: %u is a line number, the first %s is a filename of a file with the extension "second %s" and the third %s is a unique identifier for bugreports + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), i, "sources", Fd.Name().c_str(), "URI"); - std::vector list_uris = StringSplit(URIS, " "); - std::vector list_dist = StringSplit(Suite, " "); - std::vector list_section = StringSplit(Section, " "); - for (std::vector::const_iterator U = list_uris.begin(); U != list_uris.end(); ++U) { - std::string URI = (*U); - if (!FixupURI(URI)) - { - _error->Error(_("Malformed stanza %u in source list %s (URI parse)"),i,Fd.Name().c_str()); - return false; - } + std::string URI = *U; + if (U->empty() || FixupURI(URI) == false) + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), i, "sources", Fd.Name().c_str(), "URI parse"); + + if (list_suite.empty()) + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), i, "sources", Fd.Name().c_str(), "Suite"); - for (std::vector::const_iterator I = list_dist.begin(); - I != list_dist.end(); ++I) + for (std::vector::const_iterator S = list_suite.begin(); + S != list_suite.end(); ++S) { - for (std::vector::const_iterator J = list_section.begin(); - J != list_section.end(); ++J) - { - if (CreateItem(List, URI, (*I), (*J), Options) == false) - { - return false; - } - } + if (S->empty() == false && (*S)[S->size() - 1] == '/') + { + if (list_comp.empty() == false) + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), i, "sources", Fd.Name().c_str(), "absolute Suite Component"); + if (CreateItem(List, URI, *S, "", Options) == false) + return false; + } + else + { + if (list_comp.empty()) + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), i, "sources", Fd.Name().c_str(), "Component"); + + for (std::vector::const_iterator C = list_comp.begin(); + C != list_comp.end(); ++C) + { + if (CreateItem(List, URI, *S, *C, Options) == false) + { + return false; + } + } + } } } return true; } - + /*}}}*/ // Type::ParseLine - Parse a single line /*{{{*/ // --------------------------------------------------------------------- /* This is a generic one that is the 'usual' format for sources.list Weird types may override this. */ bool pkgSourceList::Type::ParseLine(vector &List, const char *Buffer, - unsigned long const &CurLine, + unsigned int const CurLine, string const &File) const { for (;Buffer != 0 && isspace(*Buffer); ++Buffer); // Skip whitespaces @@ -171,10 +188,10 @@ bool pkgSourceList::Type::ParseLine(vector &List, // get one option, e.g. option1=value1 string option; if (ParseQuoteWord(Buffer,option) == false) - return _error->Error(_("Malformed line %lu in source list %s ([option] unparseable)"),CurLine,File.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "[option] unparseable"); if (option.length() < 3) - return _error->Error(_("Malformed line %lu in source list %s ([option] too short)"),CurLine,File.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "[option] too short"); // accept options even if the last has no space before the ]-end marker if (option.at(option.length()-1) == ']') @@ -185,16 +202,16 @@ bool pkgSourceList::Type::ParseLine(vector &List, size_t const needle = option.find('='); if (needle == string::npos) - return _error->Error(_("Malformed line %lu in source list %s ([%s] is not an assignment)"),CurLine,File.c_str(), option.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "[option] not assignment"); string const key = string(option, 0, needle); string const value = string(option, needle + 1, option.length()); if (key.empty() == true) - return _error->Error(_("Malformed line %lu in source list %s ([%s] has no key)"),CurLine,File.c_str(), option.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "[option] no key"); if (value.empty() == true) - return _error->Error(_("Malformed line %lu in source list %s ([%s] key %s has no value)"),CurLine,File.c_str(),option.c_str(),key.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "[option] no value"); Options[key] = value; } @@ -207,33 +224,33 @@ bool pkgSourceList::Type::ParseLine(vector &List, string Section; if (ParseQuoteWord(Buffer,URI) == false) - return _error->Error(_("Malformed line %lu in source list %s (URI)"),CurLine,File.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "URI"); if (ParseQuoteWord(Buffer,Dist) == false) - return _error->Error(_("Malformed line %lu in source list %s (dist)"),CurLine,File.c_str()); - + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "Suite"); + if (FixupURI(URI) == false) - return _error->Error(_("Malformed line %lu in source list %s (URI parse)"),CurLine,File.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "URI parse"); // Check for an absolute dists specification. if (Dist.empty() == false && Dist[Dist.size() - 1] == '/') { if (ParseQuoteWord(Buffer,Section) == true) - return _error->Error(_("Malformed line %lu in source list %s (absolute dist)"),CurLine,File.c_str()); + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "absolute Suite Component"); Dist = SubstVar(Dist,"$(ARCH)",_config->Find("APT::Architecture")); return CreateItem(List, URI, Dist, Section, Options); } - + // Grab the rest of the dists if (ParseQuoteWord(Buffer,Section) == false) - return _error->Error(_("Malformed line %lu in source list %s (dist parse)"),CurLine,File.c_str()); - + return _error->Error(_("Malformed entry %u in %s file %s (%s)"), CurLine, "list", File.c_str(), "Component"); + do { if (CreateItem(List, URI, Dist, Section, Options) == false) return false; } while (ParseQuoteWord(Buffer,Section) == true); - + return true; } /*}}}*/ @@ -242,11 +259,6 @@ bool pkgSourceList::Type::ParseLine(vector &List, /* */ pkgSourceList::pkgSourceList() : d(NULL) { -} - -pkgSourceList::pkgSourceList(string File) : d(NULL) -{ - Read(File); } /*}}}*/ // SourceList::~pkgSourceList - Destructor /*{{{*/ @@ -305,7 +317,7 @@ void pkgSourceList::Reset() // SourceList::Read - Parse the sourcelist file /*{{{*/ // --------------------------------------------------------------------- /* */ -bool pkgSourceList::Read(string File) +bool pkgSourceList::Read(string const &File) { Reset(); return ReadAppend(File); @@ -314,71 +326,63 @@ bool pkgSourceList::Read(string File) // SourceList::ReadAppend - Parse a sourcelist file /*{{{*/ // --------------------------------------------------------------------- /* */ -bool pkgSourceList::ReadAppend(string File) +bool pkgSourceList::ReadAppend(string const &File) { - if (_config->FindB("APT::Sources::Use-Deb822", false) == true) - { - int lines_parsed =ParseFileDeb822(File); - if (lines_parsed < 0) - return false; - else if (lines_parsed > 0) - return true; - // no lines parsed ... fall through and use old style parser - } - return ParseFileOldStyle(File); + if (flExtension(File) == "sources") + return ParseFileDeb822(File); + else + return ParseFileOldStyle(File); } // SourceList::ReadFileOldStyle - Read Traditional style sources.list /*{{{*/ // --------------------------------------------------------------------- /* */ -bool pkgSourceList::ParseFileOldStyle(string File) +bool pkgSourceList::ParseFileOldStyle(std::string const &File) { // Open the stream for reading ifstream F(File.c_str(),ios::in /*| ios::nocreate*/); if (F.fail() == true) return _error->Errno("ifstream::ifstream",_("Opening %s"),File.c_str()); - // CNC:2003-12-10 - 300 is too short. - char Buffer[1024]; - - int CurLine = 0; - while (F.eof() == false) + std::string Buffer; + for (unsigned int CurLine = 1; std::getline(F, Buffer); ++CurLine) { - F.getline(Buffer,sizeof(Buffer)); - CurLine++; - _strtabexpand(Buffer,sizeof(Buffer)); - if (F.fail() && !F.eof()) - return _error->Error(_("Line %u too long in source list %s."), - CurLine,File.c_str()); - - - char *I; - // CNC:2003-02-20 - Do not break if '#' is inside []. - for (I = Buffer; *I != 0 && *I != '#'; I++) - if (*I == '[') - { - char *b_end = strchr(I + 1, ']'); - if (b_end != NULL) - I = b_end; - } - *I = 0; - - const char *C = _strstrip(Buffer); - - // Comment or blank - if (C[0] == '#' || C[0] == 0) + // remove comments + size_t curpos = 0; + while ((curpos = Buffer.find('#', curpos)) != std::string::npos) + { + size_t const openbrackets = std::count(Buffer.begin(), Buffer.begin() + curpos, '['); + size_t const closedbrackets = std::count(Buffer.begin(), Buffer.begin() + curpos, ']'); + if (openbrackets > closedbrackets) + { + // a # in an option, unlikely, but oh well, it was supported so stick to it + ++curpos; + continue; + } + Buffer.erase(curpos); + break; + } + // remove spaces before/after + curpos = Buffer.find_first_not_of(" \t\r"); + if (curpos != 0) + Buffer.erase(0, curpos); + curpos = Buffer.find_last_not_of(" \t\r"); + if (curpos != std::string::npos) + Buffer.erase(curpos + 1); + + if (Buffer.empty()) continue; - + // Grok it - string LineType; - if (ParseQuoteWord(C,LineType) == false) + std::string const LineType = Buffer.substr(0, Buffer.find(' ')); + if (LineType.empty() || LineType == Buffer) return _error->Error(_("Malformed line %u in source list %s (type)"),CurLine,File.c_str()); Type *Parse = Type::GetType(LineType.c_str()); if (Parse == 0) return _error->Error(_("Type '%s' is not known on line %u in source list %s"),LineType.c_str(),CurLine,File.c_str()); - - if (Parse->ParseLine(SrcList, C, CurLine, File) == false) + + if (Parse->ParseLine(SrcList, Buffer.c_str() + LineType.length(), CurLine, File) == false) return false; } return true; @@ -387,30 +391,25 @@ bool pkgSourceList::ParseFileOldStyle(string File) // SourceList::ParseFileDeb822 - Parse deb822 style sources.list /*{{{*/ // --------------------------------------------------------------------- /* Returns: the number of stanzas parsed*/ -int pkgSourceList::ParseFileDeb822(string File) +bool pkgSourceList::ParseFileDeb822(string const &File) { - pkgTagSection Tags; - unsigned int i=0; + pkgUserTagSection Tags; + unsigned int i = 1; // see if we can read the file - _error->PushToStack(); FileFd Fd(File, FileFd::ReadOnly); pkgTagFile Sources(&Fd); if (_error->PendingError() == true) - { - _error->RevertToStack(); - return 0; - } - _error->MergeWithStack(); - + return _error->Error(_("Malformed stanza %u in source list %s (type)"),i,File.c_str()); + // read step by step while (Sources.Step(Tags) == true) { - if(!Tags.Exists("Types")) - continue; + if(Tags.Exists("Types") == false) + return _error->Error(_("Malformed stanza %u in source list %s (type)"),i,File.c_str()); string const types = Tags.FindS("Types"); - std::vector list_types = StringSplit(types, " "); + std::vector const list_types = VectorizeString(types, ' '); for (std::vector::const_iterator I = list_types.begin(); I != list_types.end(); ++I) { @@ -418,18 +417,16 @@ int pkgSourceList::ParseFileDeb822(string File) if (Parse == 0) { _error->Error(_("Type '%s' is not known on stanza %u in source list %s"), (*I).c_str(),i,Fd.Name().c_str()); - return -1; + return false; } - + if (!Parse->ParseStanza(SrcList, Tags, i, Fd)) - return -1; + return false; - i++; + ++i; } } - - // we are done, return the number of stanzas read - return i; + return true; } /*}}}*/ // SourceList::FindIndex - Get the index associated with a file /*{{{*/ @@ -471,9 +468,12 @@ bool pkgSourceList::GetIndexes(pkgAcquire *Owner, bool GetAll) const // Based on ReadConfigDir() /*{{{*/ // --------------------------------------------------------------------- /* */ -bool pkgSourceList::ReadSourceDir(string Dir) +bool pkgSourceList::ReadSourceDir(string const &Dir) { - vector const List = GetListOfFilesInDir(Dir, "list", true); + std::vector ext; + ext.push_back("list"); + ext.push_back("sources"); + std::vector const List = GetListOfFilesInDir(Dir, ext, true); // Read the files for (vector::const_iterator I = List.begin(); I != List.end(); ++I) diff --git a/apt-pkg/sourcelist.h b/apt-pkg/sourcelist.h index 4f42b3e91..079f3cb3d 100644 --- a/apt-pkg/sourcelist.h +++ b/apt-pkg/sourcelist.h @@ -72,11 +72,11 @@ class pkgSourceList bool FixupURI(std::string &URI) const; virtual bool ParseStanza(std::vector &List, pkgTagSection &Tags, - int stanza_n, + unsigned int const stanza_n, FileFd &Fd); virtual bool ParseLine(std::vector &List, const char *Buffer, - unsigned long const &CurLine,std::string const &File) const; + unsigned int const CurLine,std::string const &File) const; virtual bool CreateItem(std::vector &List,std::string const &URI, std::string const &Dist,std::string const &Section, std::map const &Options) const = 0; @@ -90,18 +90,19 @@ class pkgSourceList std::vector SrcList; - int ParseFileDeb822(std::string File); - bool ParseFileOldStyle(std::string File); + private: + APT_HIDDEN bool ParseFileDeb822(std::string const &File); + APT_HIDDEN bool ParseFileOldStyle(std::string const &File); public: bool ReadMainList(); - bool Read(std::string File); + bool Read(std::string const &File); // CNC:2003-03-03 void Reset(); - bool ReadAppend(std::string File); - bool ReadSourceDir(std::string Dir); + bool ReadAppend(std::string const &File); + bool ReadSourceDir(std::string const &Dir); // List accessors inline const_iterator begin() const {return SrcList.begin();}; @@ -117,7 +118,6 @@ class pkgSourceList time_t GetLastModifiedTime(); pkgSourceList(); - explicit pkgSourceList(std::string File); virtual ~pkgSourceList(); }; diff --git a/apt-pkg/tagfile.cc b/apt-pkg/tagfile.cc index 6d7d8185b..cc63b213f 100644 --- a/apt-pkg/tagfile.cc +++ b/apt-pkg/tagfile.cc @@ -775,6 +775,14 @@ bool pkgTagSection::Write(FileFd &File, char const * const * const Order, std::v } /*}}}*/ +void pkgUserTagSection::TrimRecord(bool /*BeforeRecord*/, const char* &End)/*{{{*/ +{ + for (; Stop < End && (Stop[0] == '\n' || Stop[0] == '\r' || Stop[0] == '#'); Stop++) + if (Stop[0] == '#') + Stop = (const char*) memchr(Stop,'\n',End-Stop); +} + /*}}}*/ + #include "tagfile-order.c" // TFRewrite - Rewrite a control record /*{{{*/ diff --git a/apt-pkg/tagfile.h b/apt-pkg/tagfile.h index d0d0c7a84..81fff89f0 100644 --- a/apt-pkg/tagfile.h +++ b/apt-pkg/tagfile.h @@ -142,6 +142,14 @@ class pkgTagSection bool Write(FileFd &File, char const * const * const Order = NULL, std::vector const &Rewrite = std::vector()) const; }; + +/* For user generated file the parser should be a bit more relaxed in exchange + for being a bit slower to allow comments and new lines all over the place */ +class pkgUserTagSection : public pkgTagSection +{ + virtual void TrimRecord(bool BeforeRecord, const char* &End); +}; + class pkgTagFilePrivate; class pkgTagFile { diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index 6ebba3528..8506017ad 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -31,37 +31,99 @@ Description - The source list /etc/apt/sources.list is designed to support - any number of active sources and a variety of source media. The file lists one - source per line, with the most preferred source listed first. The information available - from the configured sources is acquired by apt-get update - (or by an equivalent command from another APT front-end). - - - Each line specifying a source starts with type (e.g. deb-src) - followed by options and arguments for this type. - Individual entries cannot be continued onto a following line. Empty lines - are ignored, and a # character anywhere on a line marks - the remainder of that line as a comment. + The source list /etc/apt/sources.list and the the + files contained in /etc/apt/sources.list.d/ are + designed to support any number of active sources and a variety of source + media. The files list one source per line (one line style) or contain multiline + stanzas defining one or more sources per stanza (deb822 style), with the + most preferred source listed first. The information available from the + configured sources is acquired by apt-get update (or + by an equivalent command from another APT front-end). sources.list.d - The /etc/apt/sources.list.d directory provides - a way to add sources.list entries in separate files. - The format is the same as for the regular sources.list file. - File names need to end with - .list and may only contain letters (a-z and A-Z), - digits (0-9), underscore (_), hyphen (-) and period (.) characters. - Otherwise APT will print a notice that it has ignored a file, unless that - file matches a pattern in the Dir::Ignore-Files-Silently - configuration list - in which case it will be silently ignored. + The /etc/apt/sources.list.d directory provides + a way to add sources.list entries in separate files. + Two different file formats are allowed as described in the next two sections. + Filenames need to have either the extension .list or + .sources depending on the contained format. + The filenames may only contain letters (a-z and A-Z), + digits (0-9), underscore (_), hyphen (-) and period (.) characters. + Otherwise APT will print a notice that it has ignored a file, unless that + file matches a pattern in the Dir::Ignore-Files-Silently + configuration list - in which case it will be silently ignored. - The deb and deb-src types + one line style format + + Files in this format have the extension .list. + Each line specifying a source starts with a type (e.g. deb-src) + followed by options and arguments for this type. + + Individual entries cannot be continued onto a following line. Empty lines + are ignored, and a # character anywhere on a line marks + the remainder of that line as a comment. Consequently an entry can be + disabled by commenting out the entire line. + + If options should be provided they are separated by spaces and all of + them together are enclosed by square brackets ([]) + included in the line after the type separated from it with a space. + If an option allows multiple values these are separated from each other + with a comma (,). An option name is separated from its + value(s) by a equal sign (=). Multivalue options have + also -= and += as separator which + instead of replacing the default with the given value(s) modify the default + value(s) to remove or include the given values. + + This is the traditional format and supported by all apt versions. + Note that not all options as described below are supported by all apt versions. + Note also that some older applications parsing this format on its own might not + expect to encounter options as they were uncommon before the introduction of + multi-architecture support. + + + + deb822 style format + + Files in this format have the extension .sources. + The format is similar in syntax to other files used by Debian and its + derivatives, like the metadata itself apt will download from the configured + sources or the debian/control file in a Debian source package. + + Individual entries are separated by an empty line, additional empty + lines are ignored, and a # character at the start of + the line marks the entire line as a comment. An entry can hence be + disabled by commenting out each line belonging to the stanza, but it is + usually easier to add the field "Enabled: no" to the stanza to disable + the entry. Removing the field or setting it to yes reenables it. + + Options have the same syntax as every other field: A fieldname separated by + a colon (:) and optionally spaces from its value(s). + Note especially that multiple values are separated by spaces, not by + commas as in the one line format. Multivalue fields like Architectures + also have Architectures-Add and Architectures-Remove + to modify the default value rather than replacing it. + + This is a new format supported by apt itself since version 1.1. Previous + versions ignore such files with a notice message as described earlier. + It is intended to make this format gradually the default format and + deprecating the previously described one line style format as it is + easier to create, extend and modify by humans and machines alike + especially if a lot of sources and/or options are involved. + + Developers who are working with and/or parsing apt sources are highly + encouraged to add support for this format and to contact the APT team + to coordinate and share this work. Users can freely adopt this format + already, but could encounter problems with software not supporting + the format yet. + + + + The deb and deb-src types: General Format The deb type references a typical two-level Debian archive, distribution/component. The - distribution is generally an archive name like + distribution is generally a suite name like stable or testing or a codename like &stable-codename; or &testing-codename; while component is one of main, contrib or @@ -70,42 +132,33 @@ code in the same form as the deb type. A deb-src line is required to fetch source indexes. - The format for a sources.list entry using the + The format for two one line style entries using the deb and deb-src types is: - deb [ options ] uri suite [component1] [component2] [...] + deb [ option1=value1 option2=value2 ] uri suite [component1] [component2] [...] +deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [...] - Alternatively a rfc822 style format is also supported: + Alternatively the equivalent entry in deb822 style looks like this: Types: deb deb-src - URIs: http://example.com - Suites: stable testing - Sections: component1 component2 - Description: short - long long long - [option1]: [option1-value] - - Types: deb - URIs: http://another.example.com - Suites: experimental - Sections: component1 component2 - Enabled: no - Description: short - long long long - [option1]: [option1-value] + URIs: uri + Suites: suite + Components: [component1] [component2] [...] + option1: value1 + option2: value2 The URI for the deb type must specify the base of the - Debian distribution, from which APT will find the information it needs. - suite can specify an exact path, in which case the + Debian distribution, from which APT will find the information it needs. + suite can specify an exact path, in which case the components must be omitted and suite must end with a slash (/). This is useful for the case when only a - particular sub-section of the archive denoted by the URI is of interest. + particular sub-directory of the archive denoted by the URI is of interest. If suite does not specify an exact path, at least one component must be present. - suite may also contain a variable, + suite may also contain a variable, $(ARCH) which expands to the Debian architecture (such as amd64 or armel) used on the system. This permits architecture-independent @@ -113,67 +166,80 @@ of interest when specifying an exact path, APT will automatically generate a URI with the current architecture otherwise. - In the traditional style sources.list format since only one - distribution can be specified per line it may be necessary to have - multiple lines for the same URI, if a subset of all available - distributions or components at that location is desired. APT will - sort the URI list after it has generated a complete set internally, - and will collapse multiple references to the same Internet host, - for instance, into a single connection, so that it does not - inefficiently establish an FTP connection, close it, do something - else, and then re-establish a connection to that same host. This - feature is useful for accessing busy FTP sites with limits on the - number of simultaneous anonymous users. APT also parallelizes - connections to different hosts to more effectively deal with sites - with low bandwidth. - - options is always optional and needs to be surrounded by - square brackets. It can consist of multiple settings in the form - setting=value. - Multiple settings are separated by spaces. The following settings are supported by APT - (note however that unsupported settings will be ignored silently): - - arch=arch1,arch2,… - can be used to specify for which architectures information should - be downloaded. If this option is not set all architectures defined by the - APT::Architectures option will be downloaded. - - arch+=arch1,arch2,… - and arch-=arch1,arch2,… - which can be used to add/remove architectures from the set which will be downloaded. - - lang=lang1,lang2,…, - lang+=lang1,lang2,… and - lang-=lang1,lang2,… functioning in - the same way as the arch-options described before. They can be used to specify for - which languages apt will acquire metadata, like translated package descriptions, for. If not specified, the - default set is defined by the Acquire::Languages config option. - - target=target1,target2,…, - target+=target1,target2,… and - target-=target1,target2,… again functioning in - the same way as the arch-options described before. They can be used to specify which - targets apt will try to acquire from this source. If not specified, the default set is defined by - the APT::Acquire::Targets configuration scope. - - trusted=yes can be set to indicate that packages - from this source are always authenticated even if the Release file - is not signed or the signature can't be checked. This disables parts of &apt-secure; - and should therefore only be used in a local and trusted context. trusted=no - is the opposite which handles even correctly authenticated sources as not authenticated. - + Especially in the one line style format since only one distribution + can be specified per line it may be necessary to have multiple lines for + the same URI, if a subset of all available distributions or components at + that location is desired. APT will sort the URI list after it has + generated a complete set internally, and will collapse multiple + references to the same Internet host, for instance, into a single + connection, so that it does not inefficiently establish a + connection, close it, do something else, and then re-establish a + connection to that same host. APT also parallelizes connections to + different hosts to more effectively deal with sites with low + bandwidth. It is important to list sources in order of preference, with the most preferred source listed first. Typically this will result in sorting by speed from fastest to slowest (CD-ROM followed by hosts on a local network, followed by distant Internet hosts, for example). - Some examples: - -deb http://ftp.debian.org/debian &stable-codename; main contrib non-free -deb http://security.debian.org/ &stable-codename;/updates main contrib non-free - + As an example, the sources for your distribution could look like this + in one line style format: + &sourceslist-list-format; or like this in + deb822 style format: + &sourceslist-sources-format; + + The deb and deb-src types: Options + Each source entry can have options specified modifying which and how + the source is accessed and data acquired from it. Format, syntax and names + of the options varies between the two formats one line and deb822 style + as described, but they have both the same options available. For simplicity + we list the deb822 fieldname and provide the one line name in brackets. + Remember that beside setting multivalue options explicitly, there is also + the option to modify them based on the default, but we aren't listing those + names explicitly here. Unsupported options are silently ignored by all + APT versions. + + + Architectures + (arch) is a multivalue option defining for + which architectures information should be downloaded. If this + option isn't set the default is all architectures as defined by + the APT::Architectures config option. + + + Languages + (lang) is a multivalue option defining for + which languages information like translated package + descriptions should be downloaded. If this option isn't set + the default is all languages as defined by the + Acquire::Languages config option. + + + Targets + (target) is a multivalue option defining + which download targets apt will try to acquire from this + source. If not specified, the default set is defined by the + APT::Acquire::Targets configuration scope. + + + Trusted (trusted) + is a tri-state value which defaults to APT deciding if a source + is considered trusted or if warnings should be raised before e.g. + packages are installed from this source. This option can be used + to override this decision either with the value yes, + which lets APT consider this source always as a trusted source + even if it has no or fails authentication checks by disabling parts + of &apt-secure; and should therefore only be used in a local and trusted + context (if at all) as otherwise security is breached. The opposite + can be achieved with the value no, which causes the source to be handled + as untrusted even if the authentication checks passed successfully. + The default value can't be set explicitly. + + + + URI specification @@ -247,34 +313,70 @@ deb http://security.debian.org/ &stable-codename;/updates main contrib non-free Examples - Uses the archive stored locally (or NFS mounted) at /home/jason/debian + Uses the archive stored locally (or NFS mounted) at /home/apt/debian for stable/main, stable/contrib, and stable/non-free. - deb file:/home/jason/debian stable main contrib non-free + deb file:/home/apt/debian stable main contrib non-free + Types: deb +URIs: file:/home/apt/debian +Suites: stable +Components: main contrib non-free As above, except this uses the unstable (development) distribution. - deb file:/home/jason/debian unstable main contrib non-free + deb file:/home/apt/debian unstable main contrib non-free + Types: deb +URIs: file:/home/apt/debian +Suites: unstable +Components: main contrib non-free Source line for the above - deb-src file:/home/jason/debian unstable main contrib non-free + deb-src file:/home/apt/debian unstable main contrib non-free + Types: deb-src +URIs: file:/home/apt/debian +Suites: unstable +Components: main contrib non-free + The first line gets package information for the architectures in APT::Architectures while the second always retrieves amd64 and armel. - deb http://ftp.debian.org/debian &stable-codename; main -deb [ arch=amd64,armel ] http://ftp.debian.org/debian &stable-codename; main + deb http://httpredir.debian.org/debian &stable-codename; main +deb [ arch=amd64,armel ] http://httpredir.debian.org/debian &stable-codename; main + Types: deb +URIs: http://httpredir.debian.org/debian +Suites: &stable-codename; +Components: main + +Types: deb +URIs: http://httpredir.debian.org/debian +Suites: &stable-codename; +Components: main +Architectures: amd64 armel + Uses HTTP to access the archive at archive.debian.org, and uses only the hamm/main area. deb http://archive.debian.org/debian-archive hamm main + Types: deb +URIs: http://archive.debian.org/debian-archive +Suites: hamm +Components: main Uses FTP to access the archive at ftp.debian.org, under the debian directory, and uses only the &stable-codename;/contrib area. deb ftp://ftp.debian.org/debian &stable-codename; contrib + Types: deb +URIs: ftp://ftp.debian.org/debian +Suites: &stable-codename; +Components: contrib Uses FTP to access the archive at ftp.debian.org, under the debian directory, and uses only the unstable/contrib area. If this line appears as well as the one in the previous example in sources.list a single FTP session will be used for both resource lines. deb ftp://ftp.debian.org/debian unstable contrib + Types: deb +URIs: ftp://ftp.debian.org/debian +Suites: unstable +Components: contrib Uses HTTP to access the archive at ftp.tlh.debian.org, under the universe directory, and uses only files found under @@ -284,15 +386,32 @@ deb [ arch=amd64,armel ] http://ftp.debian.org/debian &stable-codename; maindeb http://ftp.tlh.debian.org/universe unstable/binary-$(ARCH)/ + Types: deb +URIs: http://ftp.tlh.debian.org/universe +Suites: unstable/binary-$(ARCH)/ + + Uses HTTP to get binary packages as well as sources from the stable, testing and unstable + suites and the components main and contrib. + deb http://httpredir.debian.org/debian stable main contrib +deb-src http://httpredir.debian.org/debian stable main contrib +deb http://httpredir.debian.org/debian testing main contrib +deb-src http://httpredir.debian.org/debian testing main contrib +deb http://httpredir.debian.org/debian unstable main contrib +deb-src http://httpredir.debian.org/debian unstable main contrib + Types: deb deb-src +URIs: http://httpredir.debian.org/debian +Suites: stable testing unstable +Components: main contrib + + - + See Also - &apt-cache; &apt-conf; + &apt-get;, &apt-conf; &manbugs; - - + diff --git a/test/integration/test-apt-sources-deb822 b/test/integration/test-apt-sources-deb822 index 51fe7bcfe..adfe0e003 100755 --- a/test/integration/test-apt-sources-deb822 +++ b/test/integration/test-apt-sources-deb822 @@ -7,9 +7,8 @@ TESTDIR=$(readlink -f $(dirname $0)) setupenvironment configarchitecture 'i386' -echo 'APT::Sources::Use-Deb822 "true";' > rootdir/etc/apt/apt.conf.d/00enabledeb822 - -SOURCES='rootdir/etc/apt/sources.list' +LISTS='rootdir/etc/apt/sources.list.d/test.list' +SOURCES='rootdir/etc/apt/sources.list.d/test.sources' BASE='# some comment # that contains a : as well #Types: meep @@ -17,23 +16,48 @@ BASE='# some comment Types: deb URIs: http://ftp.debian.org/debian Suites: stable -Sections: main +Components: main Description: summay and the long part' -msgtest 'Test sources.list' 'old style' -echo "deb http://ftp.debian.org/debian stable main" > $SOURCES +msgcleantest() { + rm -f $LISTS $SOURCES + msgtest "$@" +} + +msgcleantest 'Test sources.list' 'old style' +echo "deb http://ftp.debian.org/debian stable main" > $LISTS +testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 +'http://ftp.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-i386_Packages 0 +'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris + +msgcleantest 'Test sources.list' 'old style with options' +echo "deb [trusted=yes arch+=armel,powerpc] http://ftp.debian.org/debian stable main" > $LISTS +testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 +'http://ftp.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-i386_Packages 0 +'http://ftp.debian.org/debian/dists/stable/main/binary-armel/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-armel_Packages 0 +'http://ftp.debian.org/debian/dists/stable/main/binary-powerpc/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-powerpc_Packages 0 +'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris + +msgcleantest 'Test sources.list' 'old style with comments' +echo "deb http://ftp.debian.org/debian stable main # non-free" > $LISTS +testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 +'http://ftp.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-i386_Packages 0 +'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris + +msgcleantest 'Test sources.list' 'old style with option comments' +echo "deb [trusted=yes#yeahreally] http://ftp.debian.org/debian stable main # non-free" > $LISTS testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 'http://ftp.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-i386_Packages 0 'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris -msgtest 'Test sources.list' 'simple deb822' +msgcleantest 'Test sources.list' 'simple deb822' echo "$BASE" > $SOURCES testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 'http://ftp.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-i386_Packages 0 'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris -msgtest 'Test deb822 with' 'two entries' +msgcleantest 'Test deb822 with' 'two entries' # Two entries echo "$BASE" > $SOURCES echo "" >> $SOURCES @@ -46,7 +70,7 @@ testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.deb 'http://ftp.debian.org/debian/dists/unstable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_unstable_main_i18n_Translation-en 0 " aptget update --print-uris # two suite entries -msgtest 'Test deb822 with' 'two Suite entries' +msgcleantest 'Test deb822 with' 'two Suite entries' echo "$BASE" | sed -e "s/stable/stable unstable/" > $SOURCES testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 'http://ftp.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-i386_Packages 0 @@ -55,7 +79,7 @@ testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.deb 'http://ftp.debian.org/debian/dists/unstable/main/binary-i386/Packages.bz2' ftp.debian.org_debian_dists_unstable_main_binary-i386_Packages 0 'http://ftp.debian.org/debian/dists/unstable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_unstable_main_i18n_Translation-en 0 " aptget update --print-uris -msgtest 'Test deb822' 'architecture option' +msgcleantest 'Test deb822' 'architecture option' echo "$BASE" > $SOURCES echo "Architectures: amd64 armel" >> $SOURCES testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 @@ -63,17 +87,30 @@ testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.deb 'http://ftp.debian.org/debian/dists/stable/main/binary-armel/Packages.bz2' ftp.debian.org_debian_dists_stable_main_binary-armel_Packages 0 'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris - -msgtest 'Test old-style sources.list file which has' 'malformed dist' -echo "deb http://ftp.debian.org" > $SOURCES -testequal --nomsg "E: Malformed line 1 in source list $TMPWORKINGDIRECTORY/rootdir/etc/apt/sources.list (dist) +msgcleantest 'Test old-style' 'suite arch variable' +echo 'deb http://ftp.tlh.debian.org/universe unstable/binary-$(ARCH)/' > $LISTS +testequal --nomsg "'http://ftp.tlh.debian.org/universe/unstable/binary-i386/InRelease' ftp.tlh.debian.org_universe_unstable_binary-i386_InRelease 0 +'http://ftp.tlh.debian.org/universe/unstable/binary-i386/Packages.bz2' ftp.tlh.debian.org_universe_unstable_binary-i386_Packages 0 +'http://ftp.tlh.debian.org/universe/unstable/binary-i386/en.bz2' ftp.tlh.debian.org_universe_unstable_binary-i386_en 0 " aptget update --print-uris + +msgcleantest 'Test deb822' 'suite arch variable' +echo 'Types: deb +URIs: http://ftp.tlh.debian.org/universe +Suites: stable/binary-$(ARCH)/' > $SOURCES +testequal --nomsg "'http://ftp.tlh.debian.org/universe/stable/binary-i386/InRelease' ftp.tlh.debian.org_universe_stable_binary-i386_InRelease 0 +'http://ftp.tlh.debian.org/universe/stable/binary-i386/Packages.bz2' ftp.tlh.debian.org_universe_stable_binary-i386_Packages 0 +'http://ftp.tlh.debian.org/universe/stable/binary-i386/en.bz2' ftp.tlh.debian.org_universe_stable_binary-i386_en 0 " aptget update --print-uris + +msgcleantest 'Test old-style sources.list file which has' 'malformed dist' +echo "deb http://ftp.debian.org" > $LISTS +testequal --nomsg "E: Malformed entry 1 in list file $TMPWORKINGDIRECTORY/$LISTS (Suite) E: The list of sources could not be read." aptget update --print-uris -msgtest 'Test deb822 sources.list file which has' 'malformed URI' +msgcleantest 'Test deb822 sources.list file which has' 'malformed URI' echo "Types: deb Suites: stable " > $SOURCES -testequal --nomsg "E: Malformed stanza 0 in source list $TMPWORKINGDIRECTORY/rootdir/etc/apt/sources.list (URI parse) +testequal --nomsg "E: Malformed entry 1 in sources file $TMPWORKINGDIRECTORY/$SOURCES (URI) E: The list of sources could not be read." aptget update --print-uris # with Enabled: false @@ -82,7 +119,7 @@ echo "Enabled: no" >> $SOURCES testempty aptget update --print-uris # multiple URIs -msgtest 'Test deb822 sources.list file which has' 'Multiple URIs work' +msgcleantest 'Test deb822 sources.list file which has' 'Multiple URIs work' echo "$BASE" | sed -e 's#http://ftp.debian.org/debian#http://ftp.debian.org/debian http://ftp.de.debian.org/debian#' > $SOURCES testequal --nomsg "'http://ftp.de.debian.org/debian/dists/stable/InRelease' ftp.de.debian.org_debian_dists_stable_InRelease 0 'http://ftp.de.debian.org/debian/dists/stable/main/binary-i386/Packages.bz2' ftp.de.debian.org_debian_dists_stable_main_binary-i386_Packages 0 @@ -92,7 +129,7 @@ testequal --nomsg "'http://ftp.de.debian.org/debian/dists/stable/InRelease' ftp 'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris # multiple Type in one field -msgtest 'Test deb822 sources.list file which has' 'Multiple Types work' +msgcleantest 'Test deb822 sources.list file which has' 'Multiple Types work' echo "$BASE" | sed -e 's#Types: deb#Types: deb deb-src#' > $SOURCES testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.debian.org_debian_dists_stable_InRelease 0 'http://ftp.debian.org/debian/dists/stable/main/source/Sources.bz2' ftp.debian.org_debian_dists_stable_main_source_Sources 0 @@ -100,7 +137,7 @@ testequal --nomsg "'http://ftp.debian.org/debian/dists/stable/InRelease' ftp.deb 'http://ftp.debian.org/debian/dists/stable/main/i18n/Translation-en.bz2' ftp.debian.org_debian_dists_stable_main_i18n_Translation-en 0 " aptget update --print-uris # a Suite -msgtest 'Test deb822 sources.list file which has' 'a exact path and no sections' +msgcleantest 'Test deb822 sources.list file which has' 'an exact path and no sections' cat > $SOURCES < + + + + diff --git a/vendor/debian/apt-vendor.ent b/vendor/debian/apt-vendor.ent index 6cda5995c..8b26da385 100644 --- a/vendor/debian/apt-vendor.ent +++ b/vendor/debian/apt-vendor.ent @@ -5,3 +5,15 @@ /usr/share/keyrings/debian-archive-removed-keys.gpg"> + + + diff --git a/vendor/debian/sources.list.in b/vendor/debian/sources.list.in index 2e430296a..c36fb16a7 100644 --- a/vendor/debian/sources.list.in +++ b/vendor/debian/sources.list.in @@ -1,7 +1,6 @@ # See sources.list(5) manpage for more information # Remember that CD-ROMs, DVDs and such are managed through the apt-cdrom tool. -deb http://ftp.us.debian.org/debian &debian-stable-codename; main contrib non-free -deb http://security.debian.org &debian-stable-codename;/updates main contrib non-free +&sourceslist-list-format; # Uncomment if you want the apt-get source function to work #deb-src http://ftp.us.debian.org/debian &debian-stable-codename; main contrib non-free diff --git a/vendor/raspbian/apt-vendor.ent b/vendor/raspbian/apt-vendor.ent index e359d2001..dc69f3c03 100644 --- a/vendor/raspbian/apt-vendor.ent +++ b/vendor/raspbian/apt-vendor.ent @@ -5,3 +5,9 @@ /usr/share/keyrings/raspbian-archive-removed-keys.gpg"> + + + diff --git a/vendor/steamos/apt-vendor.ent b/vendor/steamos/apt-vendor.ent index dc1b798e6..7cf100fc4 100644 --- a/vendor/steamos/apt-vendor.ent +++ b/vendor/steamos/apt-vendor.ent @@ -6,3 +6,10 @@ + + + diff --git a/vendor/steamos/sources.list.in b/vendor/steamos/sources.list.in index 69174d090..246d2e336 100644 --- a/vendor/steamos/sources.list.in +++ b/vendor/steamos/sources.list.in @@ -1,5 +1,3 @@ # See sources.list(5) manpage for more information # Remember that CD-ROMs, DVDs and such are managed through the apt-cdrom tool. - -deb http://repo.steampowered.com/steamos ¤t-codename; main contrib non-free -deb-src http://repo.steampowered.com/steamos ¤t-codename; main contrib non-free +&sourceslist-list-format; diff --git a/vendor/tanglu/apt-vendor.ent b/vendor/tanglu/apt-vendor.ent index d2442209c..f5cd813bf 100644 --- a/vendor/tanglu/apt-vendor.ent +++ b/vendor/tanglu/apt-vendor.ent @@ -6,3 +6,9 @@ + + + diff --git a/vendor/ubuntu/apt-vendor.ent b/vendor/ubuntu/apt-vendor.ent index caa532699..dcebc9209 100644 --- a/vendor/ubuntu/apt-vendor.ent +++ b/vendor/ubuntu/apt-vendor.ent @@ -5,3 +5,16 @@ /usr/share/keyrings/ubuntu-archive-removed-keys.gpg"> + + + -- cgit v1.2.3-70-g09d2 From 268ffcebb9ae4278b1e3c3f89f8167f229164dbd Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Mon, 22 Jun 2015 12:34:11 +0200 Subject: detect and error out on conflicting Trusted settings A specific trust state can be enforced via a sources.list option, but it effects all entries handled by the same Release file, not just the entry it was given on so we enforce acknowledgement of this by requiring the same value to be (not) set on all such entries. --- apt-pkg/deb/debmetaindex.cc | 43 ++++++++++++++-------- apt-pkg/deb/debmetaindex.h | 8 ++-- apt-pkg/metaindex.cc | 2 +- apt-pkg/metaindex.h | 1 - doc/sources.list.5.xml | 8 ++++ .../test-bug-596498-trusted-unsigned-repo | 8 ++-- test/integration/test-sourceslist-trusted-options | 17 +++++++++ 7 files changed, 63 insertions(+), 24 deletions(-) (limited to 'doc') diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index f690a8d64..1f725ba05 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -29,6 +29,8 @@ #include #include +#include + class APT_HIDDEN debReleaseIndexPrivate /*{{{*/ { public: @@ -42,6 +44,11 @@ class APT_HIDDEN debReleaseIndexPrivate /*{{{*/ std::vector DebEntries; std::vector DebSrcEntries; + + debReleaseIndex::TriState Trusted; + + debReleaseIndexPrivate() : Trusted(debReleaseIndex::TRI_UNSET) {} + debReleaseIndexPrivate(bool const pTrusted) : Trusted(pTrusted ? debReleaseIndex::TRI_YES : debReleaseIndex::TRI_NO) {} }; /*}}}*/ // ReleaseIndex::MetaIndex* - display helpers /*{{{*/ @@ -101,12 +108,11 @@ std::string debReleaseIndex::LocalFileName() const /*{{{*/ /*}}}*/ // ReleaseIndex Con- and Destructors /*{{{*/ debReleaseIndex::debReleaseIndex(std::string const &URI, std::string const &Dist) : - metaIndex(URI, Dist, "deb"), d(new debReleaseIndexPrivate()), Trusted(CHECK_TRUST) + metaIndex(URI, Dist, "deb"), d(new debReleaseIndexPrivate()) {} debReleaseIndex::debReleaseIndex(std::string const &URI, std::string const &Dist, bool const Trusted) : - metaIndex(URI, Dist, "deb"), d(new debReleaseIndexPrivate()) { - SetTrusted(Trusted); -} + metaIndex(URI, Dist, "deb"), d(new debReleaseIndexPrivate(Trusted)) +{} debReleaseIndex::~debReleaseIndex() { if (d != NULL) delete d; @@ -225,9 +231,9 @@ void debReleaseIndex::AddComponent(bool const isSrc, std::string const &Name,/*{ bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const/*{{{*/ { indexRecords * const iR = new indexRecords(Dist); - if (Trusted == ALWAYS_TRUSTED) + if (d->Trusted == TRI_YES) iR->SetTrusted(true); - else if (Trusted == NEVER_TRUSTED) + else if (d->Trusted == TRI_NO) iR->SetTrusted(false); // special case for --print-uris @@ -246,19 +252,21 @@ bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll) const/*{ return true; } /*}}}*/ -// ReleaseIndex::*Trusted setters and checkers /*{{{*/ -void debReleaseIndex::SetTrusted(bool const Trusted) +// ReleaseIndex::IsTrusted /*{{{*/ +bool debReleaseIndex::SetTrusted(TriState const Trusted) { - if (Trusted == true) - this->Trusted = ALWAYS_TRUSTED; - else - this->Trusted = NEVER_TRUSTED; + if (d->Trusted == TRI_UNSET) + d->Trusted = Trusted; + else if (d->Trusted != Trusted) + // TRANSLATOR: The first is an option name from sources.list manpage, the other two URI and Suite + return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Trusted", URI.c_str(), Dist.c_str()); + return true; } bool debReleaseIndex::IsTrusted() const { - if (Trusted == ALWAYS_TRUSTED) + if (d->Trusted == TRI_YES) return true; - else if (Trusted == NEVER_TRUSTED) + else if (d->Trusted == TRI_NO) return false; @@ -476,7 +484,12 @@ class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type /*{{{*/ std::map::const_iterator const trusted = Options.find("trusted"); if (trusted != Options.end()) - Deb->SetTrusted(StringToBool(trusted->second, false)); + { + if (Deb->SetTrusted(StringToBool(trusted->second, false) ? debReleaseIndex::TRI_YES : debReleaseIndex::TRI_NO) == false) + return false; + } + else if (Deb->SetTrusted(debReleaseIndex::TRI_DONTCARE) == false) + return false; return true; } diff --git a/apt-pkg/deb/debmetaindex.h b/apt-pkg/deb/debmetaindex.h index 9b60b6137..a6db4e287 100644 --- a/apt-pkg/deb/debmetaindex.h +++ b/apt-pkg/deb/debmetaindex.h @@ -27,8 +27,6 @@ class APT_HIDDEN debReleaseIndex : public metaIndex { debReleaseIndexPrivate * const d; - enum APT_HIDDEN { ALWAYS_TRUSTED, NEVER_TRUSTED, CHECK_TRUST } Trusted; - public: APT_HIDDEN std::string MetaIndexInfo(const char *Type) const; @@ -51,7 +49,11 @@ class APT_HIDDEN debReleaseIndex : public metaIndex virtual std::vector *GetIndexFiles(); - void SetTrusted(bool const Trusted); + enum APT_HIDDEN TriState { + TRI_YES, TRI_DONTCARE, TRI_NO, TRI_UNSET + }; + bool SetTrusted(TriState const Trusted); + virtual bool IsTrusted() const; void AddComponent(bool const isSrc, std::string const &Name, diff --git a/apt-pkg/metaindex.cc b/apt-pkg/metaindex.cc index d96349974..0c88ee9cd 100644 --- a/apt-pkg/metaindex.cc +++ b/apt-pkg/metaindex.cc @@ -41,7 +41,7 @@ bool metaIndex::Merge(pkgCacheGenerator &Gen,OpProgress *) const metaIndex::metaIndex(std::string const &URI, std::string const &Dist, char const * const Type) -: d(NULL), Indexes(NULL), Type(Type), URI(URI), Dist(Dist), Trusted(false) +: d(NULL), Indexes(NULL), Type(Type), URI(URI), Dist(Dist) { /* nothing */ } diff --git a/apt-pkg/metaindex.h b/apt-pkg/metaindex.h index 1bcec1c4a..9667e1c92 100644 --- a/apt-pkg/metaindex.h +++ b/apt-pkg/metaindex.h @@ -34,7 +34,6 @@ class metaIndex const char *Type; std::string URI; std::string Dist; - bool Trusted; public: diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index 8506017ad..f87dcda23 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -223,7 +223,15 @@ deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [. source. If not specified, the default set is defined by the APT::Acquire::Targets configuration scope. + + + Further more, there are options which if set effect + all sources with the same URI and Suite, so they + have to be set on all such entries and can not be varied between + different components. APT will try to detect and error out on such + anomalies. + Trusted (trusted) is a tri-state value which defaults to APT deciding if a source is considered trusted or if warnings should be raised before e.g. diff --git a/test/integration/test-bug-596498-trusted-unsigned-repo b/test/integration/test-bug-596498-trusted-unsigned-repo index 1ff0f1d8d..c515837a3 100755 --- a/test/integration/test-bug-596498-trusted-unsigned-repo +++ b/test/integration/test-bug-596498-trusted-unsigned-repo @@ -18,7 +18,7 @@ aptgetupdate() { PKGTEXT="$(aptget install cool --assume-no -d | head -n 8)" DOWNLOG="$(echo "$PKGTEXT" | tail -n 1)" PKGTEXT="$(echo "$PKGTEXT" | head -n 7)" -DEBFILE='rootdir/etc/apt/sources.list.d/apt-test-unstable-deb.list' +DEBFILE='rootdir/etc/apt/sources.list.d/apt-test-unstable-*.list' testsuccessequal "$PKGTEXT $DOWNLOG @@ -28,7 +28,7 @@ testsuccessequal "$PKGTEXT $DOWNLOG Download complete and in download only mode" aptget install cool --assume-no -d --allow-unauthenticated -sed -i -e 's#deb#deb [trusted=no]#' $DEBFILE +sed -i -e 's#\(deb\(-src\)\?\) #\1 [trusted=no] #' $DEBFILE aptgetupdate 'testsuccess' testfailureequal "$PKGTEXT @@ -38,7 +38,7 @@ Install these packages without verification? [y/N] N E: Some packages could not be authenticated" aptget install cool --assume-no -d find aptarchive/ \( -name 'Release.gpg' -o -name 'InRelease' \) -delete -sed -i -e 's#deb \[trusted=no\]#deb#' $DEBFILE +sed -i -e 's#\(deb\(-src\)\?\) \[trusted=no\] #\1 #' $DEBFILE aptgetupdate testfailureequal "$PKGTEXT @@ -54,7 +54,7 @@ Authentication warning overridden. $DOWNLOG Download complete and in download only mode" aptget install cool --assume-no -d --allow-unauthenticated -sed -i -e 's#deb#deb [trusted=yes]#' $DEBFILE +sed -i -e 's#\(deb\(-src\)\?\) #\1 [trusted=yes] #' $DEBFILE aptgetupdate testsuccessequal "$PKGTEXT diff --git a/test/integration/test-sourceslist-trusted-options b/test/integration/test-sourceslist-trusted-options index 5fe4933ce..86036e242 100755 --- a/test/integration/test-sourceslist-trusted-options +++ b/test/integration/test-sourceslist-trusted-options @@ -199,3 +199,20 @@ insecureaptgetupdate everythingfails everythingfails -t stable everythingfails -t testing + +msgmsg 'Test conflicting trusted options are refused' +testsource() { + echo "$@" > rootdir/etc/apt/sources.list.d/example.list + testfailuremsg 'E: Conflicting values set for option Trusted concerning source http://example.org/bad/ unstable +E: The list of sources could not be read.' aptget update --print-uris +} +for VAL in 'yes' 'no'; do + testsource "deb http://example.org/bad unstable main +deb [trusted=${VAL}] http://example.org/bad unstable non-free" + testsource "deb [trusted=${VAL}] http://example.org/bad unstable main +deb http://example.org/bad unstable non-free" +done +testsource 'deb [trusted=yes] http://example.org/bad unstable main +deb [trusted=no] http://example.org/bad unstable non-free' +testsource 'deb [trusted=no] http://example.org/bad unstable main +deb [trusted=yes] http://example.org/bad unstable non-free' -- cgit v1.2.3-70-g09d2 From 0741daeb7ab870b4dd62a93fa12a1cf6330f9a72 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Tue, 23 Jun 2015 17:26:57 +0200 Subject: add sources.list Check-Valid-Until and Valid-Until-{Max,Min} options MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit These options could be set via configuration before, but the connection to the actual sources is so strong that they should really be set in the sources.list instead – especially as this can be done a lot more specific rather than e.g. disabling Valid-Until for all sources at once. Valid-Until-* names are chosen instead of the Min/Max-ValidTime as this seems like a better name and their use in the wild is probably low enough that this isn't going to confuse anyone if we have to names for the same thing in different areas. In the longrun, the config options should be removed, but for now documentation hinting at the new options is good enough as these are the kind of options you set once across many systems with different apt versions, so the new way should work everywhere first before we deprecate the old way. --- apt-pkg/acquire-item.cc | 4 +- apt-pkg/deb/debmetaindex.cc | 134 ++++++++++++++++++-------- apt-pkg/deb/debmetaindex.h | 3 + apt-pkg/sourcelist.cc | 3 + doc/apt-get.8.xml | 10 +- doc/apt.conf.5.xml | 8 +- doc/sources.list.5.xml | 55 +++++++++-- test/integration/test-releasefile-valid-until | 9 ++ 8 files changed, 170 insertions(+), 56 deletions(-) (limited to 'doc') diff --git a/apt-pkg/acquire-item.cc b/apt-pkg/acquire-item.cc index 100199bc1..a30a5d154 100644 --- a/apt-pkg/acquire-item.cc +++ b/apt-pkg/acquire-item.cc @@ -1041,8 +1041,8 @@ bool pkgAcqMetaBase::VerifyVendor(string const &Message) /*{{{*/ Transformed = ""; } - if (_config->FindB("Acquire::Check-Valid-Until", true) == true && - TransactionManager->MetaIndexParser->GetValidUntil() > 0) { + if (TransactionManager->MetaIndexParser->GetValidUntil() > 0) + { time_t const invalid_since = time(NULL) - TransactionManager->MetaIndexParser->GetValidUntil(); if (invalid_since > 0) { diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index f0b859eb4..5d7e539c7 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -44,7 +44,11 @@ class APT_HIDDEN debReleaseIndexPrivate /*{{{*/ std::vector DebEntries; std::vector DebSrcEntries; - debReleaseIndexPrivate() {} + metaIndex::TriState CheckValidUntil; + time_t ValidUntilMin; + time_t ValidUntilMax; + + debReleaseIndexPrivate() : CheckValidUntil(metaIndex::TRI_UNSET), ValidUntilMin(0), ValidUntilMax(0) {} }; /*}}}*/ // ReleaseIndex::MetaIndex* - display helpers /*{{{*/ @@ -283,43 +287,56 @@ bool debReleaseIndex::Load(std::string const &Filename, std::string * const Erro return false; } - std::string const Label = Section.FindS("Label"); - std::string const StrValidUntil = Section.FindS("Valid-Until"); + bool CheckValidUntil = _config->FindB("Acquire::Check-Valid-Until", true); + if (d->CheckValidUntil == metaIndex::TRI_NO) + CheckValidUntil = false; + else if (d->CheckValidUntil == metaIndex::TRI_YES) + CheckValidUntil = true; - // if we have a Valid-Until header in the Release file, use it as default - if (StrValidUntil.empty() == false) + if (CheckValidUntil == true) { - if(RFC1123StrToTime(StrValidUntil.c_str(), ValidUntil) == false) + std::string const Label = Section.FindS("Label"); + std::string const StrValidUntil = Section.FindS("Valid-Until"); + + // if we have a Valid-Until header in the Release file, use it as default + if (StrValidUntil.empty() == false) { - if (ErrorText != NULL) - strprintf(*ErrorText, _("Invalid 'Valid-Until' entry in Release file %s"), Filename.c_str()); - return false; + if(RFC1123StrToTime(StrValidUntil.c_str(), ValidUntil) == false) + { + if (ErrorText != NULL) + strprintf(*ErrorText, _("Invalid 'Valid-Until' entry in Release file %s"), Filename.c_str()); + return false; + } + } + // get the user settings for this archive and use what expires earlier + time_t MaxAge = d->ValidUntilMax; + if (MaxAge == 0) + { + MaxAge = _config->FindI("Acquire::Max-ValidTime", 0); + if (Label.empty() == false) + MaxAge = _config->FindI(("Acquire::Max-ValidTime::" + Label).c_str(), MaxAge); + } + time_t MinAge = d->ValidUntilMin; + if (MinAge == 0) + { + MinAge = _config->FindI("Acquire::Min-ValidTime", 0); + if (Label.empty() == false) + MinAge = _config->FindI(("Acquire::Min-ValidTime::" + Label).c_str(), MinAge); } - } - // get the user settings for this archive and use what expires earlier - int MaxAge = _config->FindI("Acquire::Max-ValidTime", 0); - if (Label.empty() == false) - MaxAge = _config->FindI(("Acquire::Max-ValidTime::" + Label).c_str(), MaxAge); - int MinAge = _config->FindI("Acquire::Min-ValidTime", 0); - if (Label.empty() == false) - MinAge = _config->FindI(("Acquire::Min-ValidTime::" + Label).c_str(), MinAge); - - LoadedSuccessfully = TRI_YES; - if(MaxAge == 0 && - (MinAge == 0 || ValidUntil == 0)) // No user settings, use the one from the Release file - return true; - if (MinAge != 0 && ValidUntil != 0) { - time_t const min_date = Date + MinAge; - if (ValidUntil < min_date) - ValidUntil = min_date; - } - if (MaxAge != 0) { - time_t const max_date = Date + MaxAge; - if (ValidUntil == 0 || ValidUntil > max_date) - ValidUntil = max_date; + if (MinAge != 0 && ValidUntil != 0) { + time_t const min_date = Date + MinAge; + if (ValidUntil < min_date) + ValidUntil = min_date; + } + if (MaxAge != 0) { + time_t const max_date = Date + MaxAge; + if (ValidUntil == 0 || ValidUntil > max_date) + ValidUntil = max_date; + } } + LoadedSuccessfully = TRI_YES; return true; } /*}}}*/ @@ -411,7 +428,7 @@ bool debReleaseIndex::GetIndexes(pkgAcquire *Owner, bool const &GetAll)/*{{{*/ return true; } /*}}}*/ -// ReleaseIndex::IsTrusted /*{{{*/ +// ReleaseIndex::Set* TriState options /*{{{*/ bool debReleaseIndex::SetTrusted(TriState const pTrusted) { if (Trusted == TRI_UNSET) @@ -421,6 +438,32 @@ bool debReleaseIndex::SetTrusted(TriState const pTrusted) return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Trusted", URI.c_str(), Dist.c_str()); return true; } +bool debReleaseIndex::SetCheckValidUntil(TriState const pCheckValidUntil) +{ + if (d->CheckValidUntil == TRI_UNSET) + d->CheckValidUntil = pCheckValidUntil; + else if (d->CheckValidUntil != pCheckValidUntil) + return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Check-Valid-Until", URI.c_str(), Dist.c_str()); + return true; +} +bool debReleaseIndex::SetValidUntilMin(time_t const Valid) +{ + if (d->ValidUntilMin == 0) + d->ValidUntilMin = Valid; + else if (d->ValidUntilMin != Valid) + return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Min-ValidTime", URI.c_str(), Dist.c_str()); + return true; +} +bool debReleaseIndex::SetValidUntilMax(time_t const Valid) +{ + if (d->ValidUntilMax == 0) + d->ValidUntilMax = Valid; + else if (d->ValidUntilMax != Valid) + return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Max-ValidTime", URI.c_str(), Dist.c_str()); + return true; +} + /*}}}*/ +// ReleaseIndex::IsTrusted /*{{{*/ bool debReleaseIndex::IsTrusted() const { if (Trusted == TRI_YES) @@ -601,6 +644,22 @@ static std::vector parsePlusMinusOptions(std::string const &Name, / /*}}}*/ class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type /*{{{*/ { + metaIndex::TriState GetTriStateOption(std::mapconst &Options, char const * const name) const + { + std::map::const_iterator const opt = Options.find(name); + if (opt != Options.end()) + return StringToBool(opt->second, false) ? metaIndex::TRI_YES : metaIndex::TRI_NO; + return metaIndex::TRI_DONTCARE; + } + + time_t GetTimeOption(std::mapconst &Options, char const * const name) const + { + std::map::const_iterator const opt = Options.find(name); + if (opt == Options.end()) + return 0; + return strtoull(opt->second.c_str(), NULL, 10); + } + protected: bool CreateItemInternal(std::vector &List, std::string const &URI, @@ -641,13 +700,10 @@ class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type /*{{{*/ parsePlusMinusOptions("lang", Options, APT::Configuration::getLanguages(true)) ); - std::map::const_iterator const trusted = Options.find("trusted"); - if (trusted != Options.end()) - { - if (Deb->SetTrusted(StringToBool(trusted->second, false) ? debReleaseIndex::TRI_YES : debReleaseIndex::TRI_NO) == false) - return false; - } - else if (Deb->SetTrusted(debReleaseIndex::TRI_DONTCARE) == false) + if (Deb->SetTrusted(GetTriStateOption(Options, "trusted")) == false || + Deb->SetCheckValidUntil(GetTriStateOption(Options, "check-valid-until")) == false || + Deb->SetValidUntilMax(GetTimeOption(Options, "valid-until-max")) == false || + Deb->SetValidUntilMin(GetTimeOption(Options, "valid-until-min")) == false) return false; return true; diff --git a/apt-pkg/deb/debmetaindex.h b/apt-pkg/deb/debmetaindex.h index 19fe6806c..879eb3bfc 100644 --- a/apt-pkg/deb/debmetaindex.h +++ b/apt-pkg/deb/debmetaindex.h @@ -53,6 +53,9 @@ class APT_HIDDEN debReleaseIndex : public metaIndex virtual std::vector *GetIndexFiles(); bool SetTrusted(TriState const Trusted); + bool SetCheckValidUntil(TriState const Trusted); + bool SetValidUntilMin(time_t const Valid); + bool SetValidUntilMax(time_t const Valid); virtual bool IsTrusted() const; diff --git a/apt-pkg/sourcelist.cc b/apt-pkg/sourcelist.cc index 0502f0e1d..0d65558ed 100644 --- a/apt-pkg/sourcelist.cc +++ b/apt-pkg/sourcelist.cc @@ -103,6 +103,9 @@ bool pkgSourceList::Type::ParseStanza(vector &List, /*{{{*/ APT_PLUSMINUS("Targets", "target"); #undef APT_PLUSMINUS mapping.insert(std::make_pair("Trusted", "trusted")); + mapping.insert(std::make_pair("Check-Valid-Until", "check-valid-until")); + mapping.insert(std::make_pair("Valid-Until-Min", "valid-until-min")); + mapping.insert(std::make_pair("Valid-Until-Max", "valid-until-max")); for (std::map::const_iterator m = mapping.begin(); m != mapping.end(); ++m) if (Tags.Exists(m->first)) diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml index 5b6788ed4..b0fe390df 100644 --- a/doc/apt-get.8.xml +++ b/doc/apt-get.8.xml @@ -531,9 +531,13 @@ - Ignore if packages can't be authenticated and don't prompt about it. - This is useful for tools like pbuilder. - Configuration Item: APT::Get::AllowUnauthenticated. + Ignore if packages can't be authenticated and don't prompt + about it. This can be useful while working with local repositories, + but is a huge security risk if data authenticity isn't ensured in + another way by the user itself. The usage of the + option for &sources-list; entries should + usually be preferred over this global override. Configuration Item: + APT::Get::AllowUnauthenticated. diff --git a/doc/apt.conf.5.xml b/doc/apt.conf.5.xml index 7d5f7e9b3..103d0622c 100644 --- a/doc/apt.conf.5.xml +++ b/doc/apt.conf.5.xml @@ -301,6 +301,8 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; Valid-Until header, but if they don't or a stricter value is desired the Max-ValidTime option below can be used. + The option of &sources-list; entries should be + preferred to disable the check selectively instead of using this global override. @@ -312,7 +314,8 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; the earlier date of the two is used as the expiration date. The default value is 0 which stands for "valid forever". Archive specific settings can be made by appending the label of the archive - to the option name. + to the option name. Preferably, the same can be achieved for specific + &sources-list; entries by using the option there. @@ -324,7 +327,8 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; frequently updated archive with a Valid-Until header instead of completely disabling the expiration date checking. Archive specific settings can and should be used by appending the label of - the archive to the option name. + the archive to the option name. Preferably, the same can be achieved for specific + &sources-list; entries by using the option there. diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index f87dcda23..aded8ecef 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -202,26 +202,26 @@ deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [. APT versions. - Architectures - (arch) is a multivalue option defining for + + () is a multivalue option defining for which architectures information should be downloaded. If this option isn't set the default is all architectures as defined by - the APT::Architectures config option. + the config option. - Languages - (lang) is a multivalue option defining for + + () is a multivalue option defining for which languages information like translated package descriptions should be downloaded. If this option isn't set the default is all languages as defined by the - Acquire::Languages config option. + config option. - Targets - (target) is a multivalue option defining + + () is a multivalue option defining which download targets apt will try to acquire from this source. If not specified, the default set is defined by the - APT::Acquire::Targets configuration scope. + configuration scope. @@ -232,7 +232,7 @@ deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [. anomalies. - Trusted (trusted) + () is a tri-state value which defaults to APT deciding if a source is considered trusted or if warnings should be raised before e.g. packages are installed from this source. This option can be used @@ -245,6 +245,41 @@ deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [. as untrusted even if the authentication checks passed successfully. The default value can't be set explicitly. + + () + is a yes/no value which controls if APT should try to detect + replay attacks. A repository creator can declare until then the + data provided in the repository should be considered valid and + if this time is reached, but no new data is provided the data + is considered expired and an error is raised. Beside + increasing security as a malicious attacker can't sent old data + forever denying a user to be able to upgrade to a new version, + this also helps users identify mirrors which are no longer + updated. Some repositories like historic archives aren't + updated anymore by design through, so this check can be + disabled by setting this option to no. + Defaults to the value of configuration option + which itself + defaults to yes. + + + + () and + + () can be used to raise or + lower the time period in seconds in which the data from this + repository is considered valid. -Max can be especially useful + if the repository provides no Valid-Until field on its Release + file to set your own value, while -Min can be used to increase + the valid time on seldomly updated (local) mirrors of a more + frequently updated but less accessible archive (which is in the + sources.list as well) instead of disabling the check entirely. + Default to the value of the configuration options + and + which are both unset by + default. + + diff --git a/test/integration/test-releasefile-valid-until b/test/integration/test-releasefile-valid-until index e000abf5d..43574ec3e 100755 --- a/test/integration/test-releasefile-valid-until +++ b/test/integration/test-releasefile-valid-until @@ -46,3 +46,12 @@ runtest 'accepted' 'good Min-Valid (bad Until, good Max-Valid) <' 'now - 7 days' runtest 'rejected' 'bad Max-Valid (bad Until, good Min-Valid) >' 'now - 7 days' 'now - 2 days' -o Acquire::Max-ValidTime=12096 -o Acquire::Min-ValidTime=2419200 runtest 'rejected' 'bad Max-Valid (bad Until, bad Min-Valid) <' 'now - 7 days' 'now - 2 days' -o Acquire::Min-ValidTime=12096 -o Acquire::Max-ValidTime=241920 runtest 'rejected' 'bad Max-Valid (bad Until, bad Min-Valid) >' 'now - 7 days' 'now - 2 days' -o Acquire::Max-ValidTime=12096 -o Acquire::Min-ValidTime=241920 + +sed -i -e 's#\(deb\(-src\)\?\) #\1 [check-valid-until=no] #' rootdir/etc/apt/sources.list.d/* +runtest 'accepted' 'bad Until but overriden by sources option' 'now - 7 days' 'now - 4 days' + +sed -i -e 's#\(deb\(-src\)\?\) \[check-valid-until=no\] #\1 [valid-until-max=86400] #' rootdir/etc/apt/sources.list.d/* +runtest 'rejected' 'bad Max-Valid (good Until) via sources option' 'now - 7 days' 'now + 4 days' + +sed -i -e 's#\(deb\(-src\)\?\) \[valid-until-max=86400\] #\1 [valid-until-min=1209600] #' rootdir/etc/apt/sources.list.d/* +runtest 'accepted' 'good Min-Valid (bad Until) via sources option' 'now - 7 days' 'now - 4 days' -- cgit v1.2.3-70-g09d2 From b0d408547734100bf86781615f546487ecf390d9 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Wed, 24 Jun 2015 19:31:22 +0200 Subject: implement Signed-By option for sources.list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Limits which key(s) can be used to sign a repository. Not immensely useful from a security perspective all by itself, but if the user has additional measures in place to confine a repository (like pinning) an attacker who gets the key for such a repository is limited to its potential and can't use the key to sign its attacks for an other (maybe less limited) repository… (yes, this is as weak as it sounds, but having the capability might come in handy for implementing other stuff later). --- apt-pkg/acquire-item.cc | 15 +++++++- apt-pkg/acquire-item.h | 1 + apt-pkg/contrib/gpgv.cc | 17 +++++++++- apt-pkg/contrib/gpgv.h | 5 ++- apt-pkg/deb/debmetaindex.cc | 35 +++++++++++++++++++ apt-pkg/deb/debmetaindex.h | 1 + apt-pkg/metaindex.cc | 4 +-- apt-pkg/metaindex.h | 4 ++- apt-pkg/sourcelist.cc | 28 ++++++++------- cmdline/apt-key.in | 22 ++++++++++-- doc/sources.list.5.xml | 24 ++++++------- methods/gpgv.cc | 18 +++++----- test/integration/framework | 23 +++++++++---- test/integration/test-apt-key | 37 ++++++++++++++++++-- test/integration/test-releasefile-verification | 47 +++++++++++++++++++++++--- 15 files changed, 225 insertions(+), 56 deletions(-) (limited to 'doc') diff --git a/apt-pkg/acquire-item.cc b/apt-pkg/acquire-item.cc index a30a5d154..01a679fe0 100644 --- a/apt-pkg/acquire-item.cc +++ b/apt-pkg/acquire-item.cc @@ -808,7 +808,6 @@ string pkgAcqMetaBase::Custom600Headers() const Header += MaximumSize; string const FinalFile = GetFinalFilename(); - struct stat Buf; if (stat(FinalFile.c_str(),&Buf) == 0) Header += "\nLast-Modified: " + TimeRFC1123(Buf.st_mtime); @@ -1132,6 +1131,10 @@ string pkgAcqMetaClearSig::Custom600Headers() const { string Header = pkgAcqMetaBase::Custom600Headers(); Header += "\nFail-Ignore: true"; + std::string const key = TransactionManager->MetaIndexParser->GetSignedBy(); + if (key.empty() == false) + Header += "\nSigned-By: " + key; + return Header; } /*}}}*/ @@ -1372,6 +1375,16 @@ pkgAcqMetaSig::pkgAcqMetaSig(pkgAcquire * const Owner, /*}}}*/ pkgAcqMetaSig::~pkgAcqMetaSig() /*{{{*/ { +} + /*}}}*/ +// pkgAcqMetaSig::Custom600Headers - Insert custom request headers /*{{{*/ +std::string pkgAcqMetaSig::Custom600Headers() const +{ + std::string Header = pkgAcqTransactionItem::Custom600Headers(); + std::string const key = TransactionManager->MetaIndexParser->GetSignedBy(); + if (key.empty() == false) + Header += "\nSigned-By: " + key; + return Header; } /*}}}*/ // AcqMetaSig::Done - The signature was downloaded/verified /*{{{*/ diff --git a/apt-pkg/acquire-item.h b/apt-pkg/acquire-item.h index 10ece76c9..1cd2a6d03 100644 --- a/apt-pkg/acquire-item.h +++ b/apt-pkg/acquire-item.h @@ -541,6 +541,7 @@ class APT_HIDDEN pkgAcqMetaSig : public pkgAcqTransactionItem virtual void Failed(std::string const &Message,pkgAcquire::MethodConfig const * const Cnf); virtual void Done(std::string const &Message, HashStringList const &Hashes, pkgAcquire::MethodConfig const * const Cnf); + virtual std::string Custom600Headers() const; /** \brief Create a new pkgAcqMetaSig. */ pkgAcqMetaSig(pkgAcquire * const Owner, pkgAcqMetaClearSig * const TransactionManager, diff --git a/apt-pkg/contrib/gpgv.cc b/apt-pkg/contrib/gpgv.cc index a01e319eb..ef84da0d8 100644 --- a/apt-pkg/contrib/gpgv.cc +++ b/apt-pkg/contrib/gpgv.cc @@ -16,6 +16,8 @@ #include #include #include + +#include #include #include #include @@ -42,7 +44,7 @@ static char * GenerateTemporaryFileTemplate(const char *basename) /*{{{*/ of the lifting in regards to merging keyrings. Fun for the whole family. */ void ExecGPGV(std::string const &File, std::string const &FileGPG, - int const &statusfd, int fd[2]) + int const &statusfd, int fd[2], std::string const &key) { #define EINTERNAL 111 std::string const aptkey = _config->FindFile("Dir::Bin::apt-key", "/usr/bin/apt-key"); @@ -55,6 +57,19 @@ void ExecGPGV(std::string const &File, std::string const &FileGPG, Args.push_back(aptkey.c_str()); Args.push_back("--quiet"); Args.push_back("--readonly"); + if (key.empty() == false) + { + if (key[0] == '/') + { + Args.push_back("--keyring"); + Args.push_back(key.c_str()); + } + else + { + Args.push_back("--keyid"); + Args.push_back(key.c_str()); + } + } Args.push_back("verify"); char statusfdstr[10]; diff --git a/apt-pkg/contrib/gpgv.h b/apt-pkg/contrib/gpgv.h index f018893fd..2a4cdad72 100644 --- a/apt-pkg/contrib/gpgv.h +++ b/apt-pkg/contrib/gpgv.h @@ -38,9 +38,12 @@ class FileFd; * * @param File is the message (unsigned or clear-signed) * @param FileSig is the signature (detached or clear-signed) + * @param statusfd is the fd given to gpgv as --status-fd + * @param fd is used as a pipe for the standard output of gpgv + * @param key is the specific one to be used instead of using all */ void ExecGPGV(std::string const &File, std::string const &FileSig, - int const &statusfd, int fd[2]) APT_NORETURN; + int const &statusfd, int fd[2], std::string const &Key = "") APT_NORETURN; inline APT_NORETURN void ExecGPGV(std::string const &File, std::string const &FileSig, int const &statusfd = -1) { int fd[2]; diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index 5d7e539c7..4bb03a942 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -461,6 +461,29 @@ bool debReleaseIndex::SetValidUntilMax(time_t const Valid) else if (d->ValidUntilMax != Valid) return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Max-ValidTime", URI.c_str(), Dist.c_str()); return true; +} +bool debReleaseIndex::SetSignedBy(std::string const &pSignedBy) +{ + if (SignedBy.empty() == true && pSignedBy.empty() == false) + { + if (pSignedBy[0] == '/') // no check for existence as we could be chrooting later or such things + ; // absolute path to a keyring file + else + { + // we could go all fancy and allow short/long/string matches as gpgv/apt-key does, + // but fingerprints are harder to fake than the others and this option is set once, + // not interactively all the time so easy to type is not really a concern. + std::string finger = pSignedBy; + finger.erase(std::remove(finger.begin(), finger.end(), ' '), finger.end()); + std::transform(finger.begin(), finger.end(), finger.begin(), ::toupper); + if (finger.length() != 40 || finger.find_first_not_of("0123456789ABCDEF") != std::string::npos) + return _error->Error(_("Invalid value set for option %s concerning source %s %s (%s)"), "Signed-By", URI.c_str(), Dist.c_str(), "not a fingerprint"); + } + SignedBy = pSignedBy; + } + else if (SignedBy != pSignedBy) + return _error->Error(_("Conflicting values set for option %s concerning source %s %s"), "Signed-By", URI.c_str(), Dist.c_str()); + return true; } /*}}}*/ // ReleaseIndex::IsTrusted /*{{{*/ @@ -706,6 +729,18 @@ class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type /*{{{*/ Deb->SetValidUntilMin(GetTimeOption(Options, "valid-until-min")) == false) return false; + std::map::const_iterator const signedby = Options.find("signed-by"); + if (signedby == Options.end()) + { + if (Deb->SetSignedBy("") == false) + return false; + } + else + { + if (Deb->SetSignedBy(signedby->second) == false) + return false; + } + return true; } diff --git a/apt-pkg/deb/debmetaindex.h b/apt-pkg/deb/debmetaindex.h index 879eb3bfc..bf5b7c1ce 100644 --- a/apt-pkg/deb/debmetaindex.h +++ b/apt-pkg/deb/debmetaindex.h @@ -56,6 +56,7 @@ class APT_HIDDEN debReleaseIndex : public metaIndex bool SetCheckValidUntil(TriState const Trusted); bool SetValidUntilMin(time_t const Valid); bool SetValidUntilMax(time_t const Valid); + bool SetSignedBy(std::string const &SignedBy); virtual bool IsTrusted() const; diff --git a/apt-pkg/metaindex.cc b/apt-pkg/metaindex.cc index 8bd13bb18..baf695f16 100644 --- a/apt-pkg/metaindex.cc +++ b/apt-pkg/metaindex.cc @@ -27,8 +27,7 @@ bool metaIndex::Merge(pkgCacheGenerator &Gen,OpProgress *) const metaIndex::metaIndex(std::string const &URI, std::string const &Dist, char const * const Type) : d(NULL), Indexes(NULL), Type(Type), URI(URI), Dist(Dist), Trusted(TRI_UNSET), - LoadedSuccessfully(TRI_UNSET), - Date(0), ValidUntil(0), SupportsAcquireByHash(false) + Date(0), ValidUntil(0), SupportsAcquireByHash(false), LoadedSuccessfully(TRI_UNSET) { /* nothing */ } @@ -48,6 +47,7 @@ APT_PURE std::string metaIndex::GetURI() const { return URI; } APT_PURE std::string metaIndex::GetDist() const { return Dist; } APT_PURE const char* metaIndex::GetType() const { return Type; } APT_PURE metaIndex::TriState metaIndex::GetTrusted() const { return Trusted; } +APT_PURE std::string metaIndex::GetSignedBy() const { return SignedBy; } APT_PURE std::string metaIndex::GetCodename() const { return Codename; } APT_PURE std::string metaIndex::GetSuite() const { return Suite; } APT_PURE bool metaIndex::GetSupportsAcquireByHash() const { return SupportsAcquireByHash; } diff --git a/apt-pkg/metaindex.h b/apt-pkg/metaindex.h index 5be7397ae..d284655bf 100644 --- a/apt-pkg/metaindex.h +++ b/apt-pkg/metaindex.h @@ -52,7 +52,7 @@ protected: std::string URI; std::string Dist; TriState Trusted; - TriState LoadedSuccessfully; + std::string SignedBy; // parsed from a file std::string Suite; @@ -61,6 +61,7 @@ protected: time_t ValidUntil; bool SupportsAcquireByHash; std::map Entries; + TriState LoadedSuccessfully; public: // Various accessors @@ -68,6 +69,7 @@ public: std::string GetDist() const; const char* GetType() const; TriState GetTrusted() const; + std::string GetSignedBy() const; std::string GetCodename() const; std::string GetSuite() const; diff --git a/apt-pkg/sourcelist.cc b/apt-pkg/sourcelist.cc index 0d65558ed..eef0ee709 100644 --- a/apt-pkg/sourcelist.cc +++ b/apt-pkg/sourcelist.cc @@ -93,27 +93,29 @@ bool pkgSourceList::Type::ParseStanza(vector &List, /*{{{*/ if (Enabled.empty() == false && StringToBool(Enabled) == false) return true; - std::map mapping; + std::map > mapping; #define APT_PLUSMINUS(X, Y) \ - mapping.insert(std::make_pair(X, Y)); \ - mapping.insert(std::make_pair(X "Add", Y "+")); \ - mapping.insert(std::make_pair(X "Remove", Y "-")) + mapping.insert(std::make_pair(X, std::make_pair(Y, true))); \ + mapping.insert(std::make_pair(X "Add", std::make_pair(Y "+", true))); \ + mapping.insert(std::make_pair(X "Remove", std::make_pair(Y "-", true))) APT_PLUSMINUS("Architectures", "arch"); APT_PLUSMINUS("Languages", "lang"); APT_PLUSMINUS("Targets", "target"); #undef APT_PLUSMINUS - mapping.insert(std::make_pair("Trusted", "trusted")); - mapping.insert(std::make_pair("Check-Valid-Until", "check-valid-until")); - mapping.insert(std::make_pair("Valid-Until-Min", "valid-until-min")); - mapping.insert(std::make_pair("Valid-Until-Max", "valid-until-max")); + mapping.insert(std::make_pair("Trusted", std::make_pair("trusted", false))); + mapping.insert(std::make_pair("Check-Valid-Until", std::make_pair("check-valid-until", false))); + mapping.insert(std::make_pair("Valid-Until-Min", std::make_pair("valid-until-min", false))); + mapping.insert(std::make_pair("Valid-Until-Max", std::make_pair("valid-until-max", false))); + mapping.insert(std::make_pair("Signed-By", std::make_pair("signed-by", false))); - for (std::map::const_iterator m = mapping.begin(); m != mapping.end(); ++m) + for (std::map >::const_iterator m = mapping.begin(); m != mapping.end(); ++m) if (Tags.Exists(m->first)) { - // for deb822 the " " is the delimiter, but the backend expects "," - std::string option = Tags.FindS(m->first); - std::replace(option.begin(), option.end(), ' ', ','); - Options[m->second] = option; + std::string option = Tags.FindS(m->first); + // for deb822 the " " is the delimiter, but the backend expects "," + if (m->second.second == true) + std::replace(option.begin(), option.end(), ' ', ','); + Options[m->second.first] = option; } // now create one item per suite/section diff --git a/cmdline/apt-key.in b/cmdline/apt-key.in index 2a66ad74d..16887bd50 100644 --- a/cmdline/apt-key.in +++ b/cmdline/apt-key.in @@ -199,7 +199,7 @@ remove_key_from_keyring() { foreach_keyring_do() { local ACTION="$1" shift - # if a --keyring was given, just remove from there + # if a --keyring was given, just work on this one if [ -n "$FORCED_KEYRING" ]; then $ACTION "$FORCED_KEYRING" "$@" else @@ -279,7 +279,14 @@ merge_back_changes() { } setup_merged_keyring() { - if [ -z "$FORCED_KEYRING" ]; then + if [ -n "$FORCED_KEYID" ]; then + foreach_keyring_do 'import_keys_from_keyring' "${GPGHOMEDIR}/allrings.gpg" + FORCED_KEYRING="${GPGHOMEDIR}/forcedkeyid.gpg" + TRUSTEDFILE="${FORCED_KEYRING}" + GPG="$GPG --keyring $TRUSTEDFILE" + # ignore error as this "just" means we haven't found the forced keyid and the keyring will be empty + $GPG_CMD --batch --yes --keyring "${GPGHOMEDIR}/allrings.gpg" --export "$FORCED_KEYID" | $GPG --batch --yes --import || true + elif [ -z "$FORCED_KEYRING" ]; then foreach_keyring_do 'import_keys_from_keyring' "${GPGHOMEDIR}/pubring.gpg" if [ -r "${GPGHOMEDIR}/pubring.gpg" ]; then cp -a "${GPGHOMEDIR}/pubring.gpg" "${GPGHOMEDIR}/pubring.orig.gpg" @@ -328,12 +335,17 @@ while [ -n "$1" ]; do TRUSTEDFILE="$1" FORCED_KEYRING="$1" ;; + --keyid) + shift + FORCED_KEYID="$1" + ;; --secret-keyring) shift FORCED_SECRET_KEYRING="$1" ;; --readonly) merge_back_changes() { true; } + create_new_keyring() { true; } ;; --fakeroot) requires_root() { true; } @@ -460,7 +472,11 @@ case "$command" in verify) setup_merged_keyring if which gpgv >/dev/null 2>&1; then - gpgv --homedir "${GPGHOMEDIR}" --keyring "${GPGHOMEDIR}/pubring.gpg" --ignore-time-conflict "$@" + if [ -n "$FORCED_KEYRING" ]; then + gpgv --homedir "${GPGHOMEDIR}" --keyring "${FORCED_KEYRING}" --ignore-time-conflict "$@" + else + gpgv --homedir "${GPGHOMEDIR}" --keyring "${GPGHOMEDIR}/pubring.gpg" --ignore-time-conflict "$@" + fi else $GPG --verify "$@" fi diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index aded8ecef..12a7773f5 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -232,18 +232,18 @@ deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [. anomalies. - () - is a tri-state value which defaults to APT deciding if a source - is considered trusted or if warnings should be raised before e.g. - packages are installed from this source. This option can be used - to override this decision either with the value yes, - which lets APT consider this source always as a trusted source - even if it has no or fails authentication checks by disabling parts - of &apt-secure; and should therefore only be used in a local and trusted - context (if at all) as otherwise security is breached. The opposite - can be achieved with the value no, which causes the source to be handled - as untrusted even if the authentication checks passed successfully. - The default value can't be set explicitly. + () + is either an absolute path to a keyring file (has to be + accessible and readable for the _apt user, + so ensure everyone has read-permissions on the file) or a + fingerprint of a key in either the + trusted.gpg keyring or in one of the + keyrings in the trusted.gpg.d/ directory + (see apt-key fingerprint). If the option is + set only the key(s) in this keyring or only the key with this + fingerprint is used for the &apt-secure; verification of this + repository. Otherwise all keys in the trusted keyrings are + considered valid signers for this repository. () diff --git a/methods/gpgv.cc b/methods/gpgv.cc index 41f138be6..014430041 100644 --- a/methods/gpgv.cc +++ b/methods/gpgv.cc @@ -37,13 +37,14 @@ class GPGVMethod : public pkgAcqMethod { private: string VerifyGetSigners(const char *file, const char *outfile, - vector &GoodSigners, + std::string const &key, + vector &GoodSigners, vector &BadSigners, vector &WorthlessSigners, vector &NoPubKeySigners); protected: - virtual bool Fetch(FetchItem *Itm); + virtual bool URIAcquire(std::string const &Message, FetchItem *Itm); virtual bool Configuration(string Message); public: @@ -61,6 +62,7 @@ bool GPGVMethod::Configuration(string Message) } string GPGVMethod::VerifyGetSigners(const char *file, const char *outfile, + std::string const &key, vector &GoodSigners, vector &BadSigners, vector &WorthlessSigners, @@ -80,7 +82,7 @@ string GPGVMethod::VerifyGetSigners(const char *file, const char *outfile, if (pid < 0) return string("Couldn't spawn new process") + strerror(errno); else if (pid == 0) - ExecGPGV(outfile, file, 3, fd); + ExecGPGV(outfile, file, 3, fd, key); close(fd[1]); FILE *pipein = fdopen(fd[0], "r"); @@ -174,11 +176,11 @@ string GPGVMethod::VerifyGetSigners(const char *file, const char *outfile, return _("Unknown error executing apt-key"); } -bool GPGVMethod::Fetch(FetchItem *Itm) +bool GPGVMethod::URIAcquire(std::string const &Message, FetchItem *Itm) { - URI Get = Itm->Uri; - string Path = Get.Host + Get.Path; // To account for relative paths - string keyID; + URI const Get = Itm->Uri; + string const Path = Get.Host + Get.Path; // To account for relative paths + std::string const key = LookupTag(Message, "Signed-By"); vector GoodSigners; vector BadSigners; // a worthless signature is a expired or revoked one @@ -190,7 +192,7 @@ bool GPGVMethod::Fetch(FetchItem *Itm) URIStart(Res); // Run apt-key on file, extract contents and get the key ID of the signer - string msg = VerifyGetSigners(Path.c_str(), Itm->DestFile.c_str(), + string msg = VerifyGetSigners(Path.c_str(), Itm->DestFile.c_str(), key, GoodSigners, BadSigners, WorthlessSigners, NoPubKeySigners); if (GoodSigners.empty() || !BadSigners.empty() || !NoPubKeySigners.empty()) diff --git a/test/integration/framework b/test/integration/framework index 059cba9fb..6ae5003f7 100644 --- a/test/integration/framework +++ b/test/integration/framework @@ -1414,14 +1414,23 @@ testfailure() { else local EXITCODE=$? if expr match "$1" '^apt.*' >/dev/null; then - if grep -q -E ' runtime error: ' "$OUTPUT"; then - msgfailoutput 'compiler detected undefined behavior' "$OUTPUT" "$@" - elif grep -q -E '==ERROR' "$OUTPUT"; then - msgfailoutput 'compiler sanitizers reported errors' "$OUTPUT" "$@" - elif ! grep -q -E '^E: ' "$OUTPUT"; then - msgfailoutput "run failed with exitcode ${EXITCODE}, but with no errors" "$OUTPUT" "$@" + if [ "$1" = 'aptkey' ]; then + if grep -q -E " Can't check signature: " "$OUTPUT" || \ + grep -q -E " BAD signature from " "$OUTPUT"; then + msgpass + else + msgfailoutput "run failed with exitcode ${EXITCODE}, but no signature error" "$OUTPUT" "$@" + fi else - msgpass + if grep -q -E ' runtime error: ' "$OUTPUT"; then + msgfailoutput 'compiler detected undefined behavior' "$OUTPUT" "$@" + elif grep -q -E '==ERROR' "$OUTPUT"; then + msgfailoutput 'compiler sanitizers reported errors' "$OUTPUT" "$@" + elif ! grep -q -E '^E: ' "$OUTPUT"; then + msgfailoutput "run failed with exitcode ${EXITCODE}, but with no errors" "$OUTPUT" "$@" + else + msgpass + fi fi else msgpass diff --git a/test/integration/test-apt-key b/test/integration/test-apt-key index 486acccc8..e1be08c65 100755 --- a/test/integration/test-apt-key +++ b/test/integration/test-apt-key @@ -73,7 +73,7 @@ pub 2048R/DBAC8DAE 2010-08-18' testsuccess aptkey --fakeroot del DBAC8DAE testempty aptkey list - msgtest 'Test key removal with' 'lowercase key ID' #keylength somewher between 8byte and short + msgtest 'Test key removal with' 'lowercase key ID' #keylength somewhere between 8byte and short cleanplate cp -a keys/joesixpack.pub rootdir/etc/apt/trusted.gpg.d/joesixpack.gpg testsuccess --nomsg aptkey --fakeroot del d141dbac8dae @@ -166,6 +166,40 @@ pub 2048R/528144E2 2011-01-16' msgtest 'Test merge-back of' 'removed duplicate keys' testsuccess --nomsg aptkey adv --batch --yes --delete-keys DBAC8DAE testaptkeys 'pub 2048R/528144E2 2011-01-16' + + cleanplate + cp -a keys/joesixpack.pub rootdir/etc/apt/trusted.gpg.d/joesixpack.gpg + cp -a keys/testcase-multikey.pub rootdir/etc/apt/trusted.gpg.d/multikey.gpg + msgtest 'Test signing a file' 'with a key' + echo 'Verify me. This is my signature.' > signature + testsuccess --nomsg aptkey --quiet --keyring keys/marvinparanoid.pub --secret-keyring keys/marvinparanoid.sec --readonly \ + adv --batch --yes --default-key 'Marvin' --armor --detach-sign --sign --output signature.gpg signature + + msgtest 'Test verify a file' 'with all keys' + testsuccess --nomsg aptkey --quiet --readonly verify signature.gpg signature + + msgtest 'Test verify a file' 'with good keyring' + testsuccess --nomsg aptkey --quiet --readonly --keyring keys/testcase-multikey.pub verify signature.gpg signature + + msgtest 'Test fail verify a file' 'with bad keyring' + testfailure --nomsg aptkey --quiet --readonly --keyring keys/joesixpack.pub verify signature.gpg signature + + msgtest 'Test fail verify a file' 'with non-existing keyring' + testfailure --nomsg aptkey --quiet --readonly --keyring keys/does-not-exist.pub verify signature.gpg signature + testfailure test -e keys/does-not-exist.pub + + msgtest 'Test verify a file' 'with good keyid' + testsuccess --nomsg aptkey --quiet --readonly --keyid 'Paranoid' verify signature.gpg signature + + msgtest 'Test fail verify a file' 'with bad keyid' + testfailure --nomsg aptkey --quiet --readonly --keyid 'Sixpack' verify signature.gpg signature + + msgtest 'Test fail verify a file' 'with non-existing keyid' + testfailure --nomsg aptkey --quiet --readonly --keyid 'Kalnischkies' verify signature.gpg signature + + msgtest 'Test verify fails on' 'bad file' + echo 'lalalalala' > signature + testfailure --nomsg aptkey --quiet --readonly verify signature.gpg signature } setupgpgcommand() { @@ -187,4 +221,3 @@ setupgpgcommand 'gpg' testrun setupgpgcommand 'gpg2' testrun - diff --git a/test/integration/test-releasefile-verification b/test/integration/test-releasefile-verification index e8419524c..1c3953c8b 100755 --- a/test/integration/test-releasefile-verification +++ b/test/integration/test-releasefile-verification @@ -139,11 +139,6 @@ runtest() { failaptold prepare ${PKGFILE}-new - # weborf doesn't support If-Range - for release in $(find rootdir/var/lib/apt/lists/partial/ -name '*Release'); do - rm $release - touch $release - done signreleasefiles 'Joe Sixpack' find aptarchive/ -name "$DELETEFILE" -delete msgmsg 'Bad warm archive signed by' 'Joe Sixpack' @@ -191,6 +186,48 @@ runtest() { testsuccessequal "$(cat ${PKGFILE}-new) " aptcache show apt installaptnew + + prepare ${PKGFILE} + rm -rf rootdir/var/lib/apt/lists + signreleasefiles 'Marvin Paranoid' + find aptarchive/ -name "$DELETEFILE" -delete + msgmsg 'Cold archive signed by good keyring' 'Marvin Paranoid' + local MARVIN="$(readlink -f keys/marvinparanoid.pub)" + sed -i "s#^\(deb\(-src\)\?\) #\1 [signed-by=$MARVIN] #" rootdir/etc/apt/sources.list.d/* + testsuccess aptget update -o Debug::pkgAcquire::Worker=1 + testsuccessequal "$(cat ${PKGFILE}) +" aptcache show apt + installaptold + + rm -rf rootdir/var/lib/apt/lists + signreleasefiles 'Joe Sixpack' + find aptarchive/ -name "$DELETEFILE" -delete + msgmsg 'Cold archive signed by bad keyring' 'Joe Sixpack' + updatewithwarnings '^W: .* NO_PUBKEY' + + sed -i "s#^\(deb\(-src\)\?\) \[signed-by=$MARVIN\] #\1 #" rootdir/etc/apt/sources.list.d/* + local MARVIN="$(aptkey --keyring $MARVIN finger | grep 'Key fingerprint' | cut -d'=' -f 2 | tr -d ' ')" + + prepare ${PKGFILE} + rm -rf rootdir/var/lib/apt/lists + signreleasefiles 'Marvin Paranoid' + find aptarchive/ -name "$DELETEFILE" -delete + msgmsg 'Cold archive signed by good keyid' 'Marvin Paranoid' + sed -i "s#^\(deb\(-src\)\?\) #\1 [signed-by=$MARVIN] #" rootdir/etc/apt/sources.list.d/* + cp keys/marvinparanoid.pub rootdir/etc/apt/trusted.gpg.d/marvinparanoid.gpg + testsuccess aptget update -o Debug::pkgAcquire::Worker=1 -o Debug::Acquire::gpgv=1 + testsuccessequal "$(cat ${PKGFILE}) +" aptcache show apt + installaptold + rm -f rootdir/etc/apt/trusted.gpg.d/marvinparanoid.gpg + + rm -rf rootdir/var/lib/apt/lists + signreleasefiles 'Joe Sixpack' + find aptarchive/ -name "$DELETEFILE" -delete + msgmsg 'Cold archive signed by bad keyid' 'Joe Sixpack' + updatewithwarnings '^W: .* NO_PUBKEY' + + sed -i "s#^\(deb\(-src\)\?\) \[signed-by=$MARVIN\] #\1 #" rootdir/etc/apt/sources.list.d/* } runtest2() { -- cgit v1.2.3-70-g09d2 From c2a4a8dded2dfb56dbcab9689b6cb4b96c9999b6 Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Fri, 10 Jul 2015 00:07:37 +0200 Subject: rename 'apt-get files' to 'apt-get indextargets' 'files' is a bit too generic as a name for a command usually only used programmatically (if at all) by developers, so instead of "wasting" this generic name for this we use "indextargets" which is actually the name of the datastructure the displayed data is stored in. Along with this rename the config options are renamed accordingly. --- apt-pkg/deb/debmetaindex.cc | 10 +++---- apt-pkg/init.cc | 34 +++++++++++----------- apt-private/private-cmndline.cc | 6 ++-- cmdline/apt-get.cc | 10 +++---- doc/acquire-additional-files.txt | 18 +++++++----- doc/apt-get.8.xml | 24 +++++++-------- doc/sources.list.5.xml | 2 +- .../test-acquire-same-repository-multiple-times | 2 +- test/integration/test-apt-acquire-additional-files | 26 ++++++++--------- .../integration/test-apt-get-update-unauth-warning | 2 +- .../test-sourceslist-lang-plusminus-options | 4 +-- 11 files changed, 70 insertions(+), 68 deletions(-) (limited to 'doc') diff --git a/apt-pkg/deb/debmetaindex.cc b/apt-pkg/deb/debmetaindex.cc index 46a7181fc..480317db3 100644 --- a/apt-pkg/deb/debmetaindex.cc +++ b/apt-pkg/deb/debmetaindex.cc @@ -128,12 +128,12 @@ static void GetIndexTargetsFor(char const * const Type, std::string const &URI, { for (std::vector::const_iterator T = E->Targets.begin(); T != E->Targets.end(); ++T) { -#define APT_T_CONFIG(X) _config->Find(std::string("APT::Acquire::Targets::") + Type + "::" + *T + "::" + (X)) +#define APT_T_CONFIG(X) _config->Find(std::string("Acquire::IndexTargets::") + Type + "::" + *T + "::" + (X)) std::string const tplMetaKey = APT_T_CONFIG(flatArchive ? "flatMetaKey" : "MetaKey"); std::string const tplShortDesc = APT_T_CONFIG("ShortDescription"); - std::string const tplLongDesc = APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); - bool const IsOptional = _config->FindB(std::string("APT::Acquire::Targets::") + Type + "::" + *T + "::Optional", true); - bool const KeepCompressed = _config->FindB(std::string("APT::Acquire::Targets::") + Type + "::" + *T + "::KeepCompressed", GzipIndex); + std::string const tplLongDesc = "$(SITE) " + APT_T_CONFIG(flatArchive ? "flatDescription" : "Description"); + bool const IsOptional = _config->FindB(std::string("Acquire::IndexTargets::") + Type + "::" + *T + "::Optional", true); + bool const KeepCompressed = _config->FindB(std::string("Acquire::IndexTargets::") + Type + "::" + *T + "::KeepCompressed", GzipIndex); #undef APT_T_CONFIG if (tplMetaKey.empty()) continue; @@ -721,7 +721,7 @@ class APT_HIDDEN debSLTypeDebian : public pkgSourceList::Type /*{{{*/ Deb->AddComponent( IsSrc, Section, - parsePlusMinusOptions("target", Options, _config->FindVector(std::string("APT::Acquire::Targets::") + Name, "", true)), + parsePlusMinusOptions("target", Options, _config->FindVector(std::string("Acquire::IndexTargets::") + Name, "", true)), parsePlusMinusOptions("arch", Options, APT::Configuration::getArchitectures()), parsePlusMinusOptions("lang", Options, APT::Configuration::getLanguages(true)) ); diff --git a/apt-pkg/init.cc b/apt-pkg/init.cc index 1fbb035f7..858bcec43 100644 --- a/apt-pkg/init.cc +++ b/apt-pkg/init.cc @@ -99,23 +99,23 @@ bool pkgInitConfig(Configuration &Cnf) // The default user we drop to in the methods Cnf.CndSet("APT::Sandbox::User", "_apt"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::MetaKey", "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::flatMetaKey", "Packages"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::ShortDescription", "Packages"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::Description", "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::flatDescription", "$(SITE) $(RELEASE) Packages"); - Cnf.CndSet("APT::Acquire::Targets::deb::Packages::Optional", false); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::MetaKey", "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::flatMetaKey", "$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::ShortDescription", "Translation-$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::Description", "$(SITE) $(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb::Translations::flatDescription", "$(SITE) $(RELEASE) Translation-$(LANGUAGE)"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::MetaKey", "$(COMPONENT)/source/Sources"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::flatMetaKey", "Sources"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::ShortDescription", "Sources"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::Description", "$(SITE) $(RELEASE)/$(COMPONENT) Sources"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::flatDescription", "$(SITE) $(RELEASE) Sources"); - Cnf.CndSet("APT::Acquire::Targets::deb-src::Sources::Optional", false); + Cnf.CndSet("Acquire::IndexTargets::deb::Packages::MetaKey", "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"); + Cnf.CndSet("Acquire::IndexTargets::deb::Packages::flatMetaKey", "Packages"); + Cnf.CndSet("Acquire::IndexTargets::deb::Packages::ShortDescription", "Packages"); + Cnf.CndSet("Acquire::IndexTargets::deb::Packages::Description", "$(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages"); + Cnf.CndSet("Acquire::IndexTargets::deb::Packages::flatDescription", "$(RELEASE) Packages"); + Cnf.CndSet("Acquire::IndexTargets::deb::Packages::Optional", false); + Cnf.CndSet("Acquire::IndexTargets::deb::Translations::MetaKey", "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"); + Cnf.CndSet("Acquire::IndexTargets::deb::Translations::flatMetaKey", "$(LANGUAGE)"); + Cnf.CndSet("Acquire::IndexTargets::deb::Translations::ShortDescription", "Translation-$(LANGUAGE)"); + Cnf.CndSet("Acquire::IndexTargets::deb::Translations::Description", "$(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)"); + Cnf.CndSet("Acquire::IndexTargets::deb::Translations::flatDescription", "$(RELEASE) Translation-$(LANGUAGE)"); + Cnf.CndSet("Acquire::IndexTargets::deb-src::Sources::MetaKey", "$(COMPONENT)/source/Sources"); + Cnf.CndSet("Acquire::IndexTargets::deb-src::Sources::flatMetaKey", "Sources"); + Cnf.CndSet("Acquire::IndexTargets::deb-src::Sources::ShortDescription", "Sources"); + Cnf.CndSet("Acquire::IndexTargets::deb-src::Sources::Description", "$(RELEASE)/$(COMPONENT) Sources"); + Cnf.CndSet("Acquire::IndexTargets::deb-src::Sources::flatDescription", "$(RELEASE) Sources"); + Cnf.CndSet("Acquire::IndexTargets::deb-src::Sources::Optional", false); Cnf.CndSet("Acquire::Changelogs::URI::Origin::Debian", "http://metadata.ftp-master.debian.org/changelogs/CHANGEPATH_changelog"); Cnf.CndSet("Acquire::Changelogs::URI::Origin::Ubuntu", "http://changelogs.ubuntu.com/changelogs/pool/CHANGEPATH/changelog"); diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index 11e88b1e7..71dceb559 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -163,10 +163,10 @@ static bool addArgumentsAPTGet(std::vector &Args, char const // once sbuild is fixed, this option can be removed addArg('f', "fix-broken", "APT::Get::Fix-Broken", 0); } - else if (CmdMatches("files")) + else if (CmdMatches("indextargets")) { - addArg(0,"format","APT::Get::Files::Format", CommandLine::HasArg); - addArg(0,"release-info","APT::Get::Files::ReleaseInfo", 0); + addArg(0,"format","APT::Get::IndexTargets::Format", CommandLine::HasArg); + addArg(0,"release-info","APT::Get::IndexTargets::ReleaseInfo", 0); } else if (CmdMatches("clean", "autoclean", "check", "download", "changelog") || CmdMatches("markauto", "unmarkauto")) // deprecated commands diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc index 9232e1505..a69f5418a 100644 --- a/cmdline/apt-get.cc +++ b/cmdline/apt-get.cc @@ -1468,7 +1468,7 @@ static bool DoChangelog(CommandLine &CmdL) return true; } /*}}}*/ -// DoFiles - Lists all IndexTargets /*{{{*/ +// DoIndexTargets - Lists all IndexTargets /*{{{*/ static std::string format_key(std::string key) { // deb822 is case-insensitive, but the human eye prefers candy @@ -1484,7 +1484,7 @@ static std::string format_key(std::string key) } return key; } -static bool DoFiles(CommandLine &CmdL) +static bool DoIndexTargets(CommandLine &CmdL) { pkgCacheFile CacheFile; pkgSourceList *SrcList = CacheFile.GetSourceList(); @@ -1492,8 +1492,8 @@ static bool DoFiles(CommandLine &CmdL) if (SrcList == NULL) return false; - std::string const Format = _config->Find("APT::Get::Files::Format"); - bool const ReleaseInfo = _config->FindB("APT::Get::Files::ReleaseInfo", true); + std::string const Format = _config->Find("APT::Get::IndexTargets::Format"); + bool const ReleaseInfo = _config->FindB("APT::Get::IndexTargets::ReleaseInfo", true); bool Filtered = CmdL.FileSize() > 1; for (pkgSourceList::const_iterator S = SrcList->begin(); S != SrcList->end(); ++S) { @@ -1687,7 +1687,7 @@ int main(int argc,const char *argv[]) /*{{{*/ {"source",&DoSource}, {"download",&DoDownload}, {"changelog",&DoChangelog}, - {"files",&DoFiles}, + {"indextargets",&DoIndexTargets}, {"moo",&DoMoo}, {"help",&ShowHelp}, {0,0}}; diff --git a/doc/acquire-additional-files.txt b/doc/acquire-additional-files.txt index f9a16318d..71ce7b0cb 100644 --- a/doc/acquire-additional-files.txt +++ b/doc/acquire-additional-files.txt @@ -26,7 +26,7 @@ they would be written in a configuration file the configuration instructing the Acquire system to download the Packages files would look like this (see also apt.conf(5) manpage for configuration file syntax): - APT::Acquire::Targets::deb::Packages { + Acquire::IndexTargets::deb::Packages { MetaKey "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages"; ShortDescription "Packages"; Description "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages"; @@ -38,7 +38,7 @@ like this (see also apt.conf(5) manpage for configuration file syntax): }; All files which should be downloaded (nicknamed 'Targets') are mentioned -below the APT::Acquire::Targets scope. 'deb' is here the type of the +below the Acquire::IndexTargets scope. 'deb' is here the type of the sources.list entry the file should be acquired for. The only other supported value is hence 'deb-src'. Beware: You can't specify multiple types here and you can't download the same (evaluated) MetaKey from @@ -47,7 +47,7 @@ multiple types! After the type you can pick any valid and unique string which preferable refers to the file it downloads (In the example we picked 'Packages'). This string is used as identifier for the target class and accessible as -'Created-By' e.g. in the "apt-get files" output as detailed below. +'Created-By' e.g. in the "apt-get indextargets" output as detailed below. All targets have three main properties you can define: * MetaKey: The identifier of the file to be downloaded as used in the @@ -92,7 +92,7 @@ NO properties have to be set to enable this. The stanzas for Translation-* files as well as for Sources files would look like this: -APT::Acquire::Targets { +Acquire::IndexTargets { deb::Translations { MetaKey "$(COMPONENT)/i18n/Translation-$(LANGUAGE)"; ShortDescription "Translation-$(LANGUAGE)"; @@ -152,7 +152,7 @@ design so multiple applications can download and use the same file rather than each and every one of them potentially downloads and uses its own copy somewhere on disk. -"apt-get files" can be used to get the location as well as other +"apt-get indextargets" can be used to get the location as well as other information about all files downloaded (aka: you will see Packages, Sources and Translation-* files here as well). Provide a line of the default output format as parameter to filter out all entries which do @@ -161,12 +161,16 @@ own output style. The variables are what you see in the output, just all uppercase and wrapped in $(), as in the configuration file. To get all the filenames of all Translation-en files you can e.g. call: - apt-get files --format '$(FILENAME)' "Created-By: Translations" "Language: en" + apt-get indextargets --format '$(FILENAME)' "Created-By: Translations" "Language: en" + +The line-based filtering and the formating is rather crude and feature- +less by design, so it is recommend to use dedicated and more powerful +tools like 'grep-dctrl'. Accessing this information via libapt is done by reading the sources.lists (pkgSourceList), iterating over the metaIndex objects this creates and calling GetIndexTargets() on them. See the sourcecode of -"apt-get files" for a complete example. +"apt-get indextargets" for a complete example. Note that by default targets are not listed if they weren't downloaded. If you want to see all targets, you can use the --no-release-info, which diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml index b0fe390df..81a9036c4 100644 --- a/doc/apt-get.8.xml +++ b/doc/apt-get.8.xml @@ -239,9 +239,9 @@ - + Displays by default a deb822 formatted listing of - information about all data files apt-get + information about all data files (aka index targets) apt-get update would download. Supports a option to modify the output format as well as accepts lines of the default output to filter the records @@ -327,17 +327,15 @@ - No action; perform a simulation of events that would occur but do not - actually change the system. - Configuration Item: APT::Get::Simulate. - - Simulated runs performed as a user will automatically deactivate locking - (Debug::NoLocking), and if the option - APT::Get::Show-User-Simulation-Note is set - (as it is by default) a notice will also be displayed indicating that - this is only a simulation. Runs performed as root do not trigger either - NoLocking or the notice - superusers should know what they are doing - without further warnings from apt-get. + No action; perform a simulation of events that would occur + based on the current system state but do not actually change the + system. Locking will be disabled () + so the system state could change while apt-get is + running. Simulations can also be executed by non-root users which might + not have read access to all apt configuration distorting the simulation. + A notice expressing this warning is also shown by default for non-root + users (). + Configuration Item: APT::Get::Simulate. Simulated runs print out a series of lines, each representing a dpkg operation: configure (Conf), remove (Remv) diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index 12a7773f5..3bc8a94ac 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -221,7 +221,7 @@ deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [. () is a multivalue option defining which download targets apt will try to acquire from this source. If not specified, the default set is defined by the - configuration scope. + configuration scope. diff --git a/test/integration/test-acquire-same-repository-multiple-times b/test/integration/test-acquire-same-repository-multiple-times index ad9cd6d7e..d3cb46c14 100755 --- a/test/integration/test-acquire-same-repository-multiple-times +++ b/test/integration/test-acquire-same-repository-multiple-times @@ -43,7 +43,7 @@ tworepos() { testsuccess --nomsg aptget update -o Debug::pkgAcquire::Worker=1 cp rootdir/tmp/testsuccess.output download.log #cat download.log - aptget files --format '$(FILENAME)' --no-release-info | sort > file.lst + aptget indextargets --format '$(FILENAME)' --no-release-info | sort > file.lst testequal "$(find $(readlink -f ./rootdir/var/lib/apt/lists) -name '*_dists_*' \( ! -name '*InRelease' \) -type f | sort)" cat file.lst testsuccess aptcache policy testequal "foo: diff --git a/test/integration/test-apt-acquire-additional-files b/test/integration/test-apt-acquire-additional-files index fecdf30bf..ee7908a2d 100755 --- a/test/integration/test-apt-acquire-additional-files +++ b/test/integration/test-apt-acquire-additional-files @@ -30,18 +30,18 @@ Reading package lists..." aptget update testempty find rootdir/var/lib/apt/lists -name '*Contents*' cat > rootdir/etc/apt/apt.conf.d/content-target.conf <> rootdir/etc/apt/apt.conf.d/content-target.conf +echo 'Acquire::IndexTargets::deb::Contents::KeepCompressed "true";' >> rootdir/etc/apt/apt.conf.d/content-target.conf testsuccessequal "Hit:1 http://localhost:8080 unstable InRelease Get:2 http://localhost:8080 unstable/main amd64 Contents [$(stat -c%s aptarchive/dists/unstable/main/Contents-amd64.gz) B] Reading package lists..." aptget update testequal 'rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz' find rootdir/var/lib/apt/lists -name '*Contents*' -testequal "$(readlink -f ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz)" aptget files --format '$(FILENAME)' 'Created-By: Contents' +testequal "$(readlink -f ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz)" aptget indextargets --format '$(FILENAME)' 'Created-By: Contents' testsuccess cmp 'rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz' 'aptarchive/dists/unstable/main/Contents-amd64.gz' rm ./rootdir/var/lib/apt/lists/localhost:8080_dists_unstable_main_Contents-amd64.gz -testempty aptget files --format '$(FILENAME)' 'Created-By: Contents' +testempty aptget indextargets --format '$(FILENAME)' 'Created-By: Contents' # and no automatic uncompress based on the name please, # only if we downloaded a compressed file, but target was uncompressed cat > rootdir/etc/apt/apt.conf.d/content-target.conf < gotlangs.list + aptget indextargets --no-release-info 'Created-By: Translations' "$@" --format '$(LANGUAGE)' | sort -u > gotlangs.list if [ -z "$LANGS" ]; then echo -n | tr ',' '\n' | sort | checkdiff - gotlangs.list && msgpass || msgfail else @@ -44,7 +44,7 @@ testlangs 'lang=de_DE' 'de_DE' echo 'deb [lang=none] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list testlangs 'lang=none' '' -testequal 'amd64' aptget files --no-release-info 'Created-By: Packages' --format '$(ARCHITECTURE)' +testequal 'amd64' aptget indextargets --no-release-info 'Created-By: Packages' --format '$(ARCHITECTURE)' echo 'deb [lang+=pt] http://example.org/debian stable rocks' > rootdir/etc/apt/sources.list testlangs 'lang+=pt' 'en,de,de_DE,pt' -- cgit v1.2.3-70-g09d2 From 8c7af4d4c95d0423fbd0f3baa979792504f4f45f Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Thu, 16 Jul 2015 11:15:25 +0200 Subject: hide implicit deps in apt-cache again by default Before MultiArch implicits weren't a thing, so they were hidden by default by definition. Adding them for MultiArch solved many problems, but having no reliable way of detecting which dependency (and provides) is implicit or not causes problems everytime we want to output dependencies without confusing our observers with unneeded implementation details. The really notworthy point here is actually that we keep now a better record of how a dependency came to be so that we can later reason about it more easily, but that is hidden so deep down in the library internals that change is more the problems it solves than the change itself. --- apt-pkg/cacheiterators.h | 18 ++++- apt-pkg/deb/deblistparser.cc | 8 +-- apt-pkg/edsp.cc | 8 +-- apt-pkg/pkgcache.cc | 61 +++++------------ apt-pkg/pkgcache.h | 16 +++-- apt-pkg/pkgcachegen.cc | 32 ++++----- apt-pkg/pkgcachegen.h | 15 +++-- apt-private/private-cmndline.cc | 1 + cmdline/apt-cache.cc | 11 ++- doc/apt-cache.8.xml | 15 ++++- test/integration/test-apt-cache | 78 ++++++++++++++++++++-- .../test-ordering-ignore-not-matching-breaks | 56 ---------------- 12 files changed, 169 insertions(+), 150 deletions(-) delete mode 100755 test/integration/test-ordering-ignore-not-matching-breaks (limited to 'doc') diff --git a/apt-pkg/cacheiterators.h b/apt-pkg/cacheiterators.h index 7e6adb92f..1063d6f9e 100644 --- a/apt-pkg/cacheiterators.h +++ b/apt-pkg/cacheiterators.h @@ -295,7 +295,16 @@ class pkgCache::DepIterator : public Iterator { bool IsNegative() const APT_PURE; bool IsIgnorable(PrvIterator const &Prv) const APT_PURE; bool IsIgnorable(PkgIterator const &Pkg) const APT_PURE; - bool IsMultiArchImplicit() const APT_PURE; + /* MultiArch can be translated to SingleArch for an resolver and we did so, + by adding dependencies to help the resolver understand the problem, but + sometimes it is needed to identify these to ignore them… */ + inline bool IsMultiArchImplicit() const APT_PURE { + return (S2->CompareOp & pkgCache::Dep::MultiArchImplicit) == pkgCache::Dep::MultiArchImplicit; + } + /* This covers additionally negative dependencies, which aren't arch-specific, + but change architecture nontheless as a Conflicts: foo does applies for all archs */ + bool IsImplicit() const APT_PURE; + bool IsSatisfied(VerIterator const &Ver) const APT_PURE; bool IsSatisfied(PrvIterator const &Prv) const APT_PURE; void GlobOr(DepIterator &Start,DepIterator &End); @@ -367,7 +376,12 @@ class pkgCache::PrvIterator : public Iterator { inline VerIterator OwnerVer() const {return VerIterator(*Owner,Owner->VerP + S->Version);} inline PkgIterator OwnerPkg() const {return PkgIterator(*Owner,Owner->PkgP + Owner->VerP[S->Version].ParentPkg);} - bool IsMultiArchImplicit() const APT_PURE; + /* MultiArch can be translated to SingleArch for an resolver and we did so, + by adding provides to help the resolver understand the problem, but + sometimes it is needed to identify these to ignore them… */ + bool IsMultiArchImplicit() const APT_PURE + { return (S->Flags & pkgCache::Flag::MultiArchImplicit) == pkgCache::Flag::MultiArchImplicit; } + inline PrvIterator() : Iterator(), Type(PrvVer) {} inline PrvIterator(pkgCache &Owner, Provides *Trg, Version*) : diff --git a/apt-pkg/deb/deblistparser.cc b/apt-pkg/deb/deblistparser.cc index e57a3524f..df0879641 100644 --- a/apt-pkg/deb/deblistparser.cc +++ b/apt-pkg/deb/deblistparser.cc @@ -812,7 +812,7 @@ bool debListParser::ParseDepends(pkgCache::VerIterator &Ver, // … but this is probably the best thing to do. if (Arch == "native") Arch = _config->Find("APT::Architecture"); - if (NewDepends(Ver,Package,Arch,Version,Op,Type) == false) + if (NewDepends(Ver,Package,Arch,Version,Op | pkgCache::Dep::ArchSpecific,Type) == false) return false; } else @@ -858,13 +858,13 @@ bool debListParser::ParseProvides(pkgCache::VerIterator &Ver) } else if (archfound != string::npos) { string OtherArch = Package.substr(archfound+1, string::npos); Package = Package.substr(0, archfound); - if (NewProvides(Ver, Package, OtherArch, Version) == false) + if (NewProvides(Ver, Package, OtherArch, Version, pkgCache::Flag::ArchSpecific) == false) return false; } else if ((Ver->MultiArch & pkgCache::Version::Foreign) == pkgCache::Version::Foreign) { if (NewProvidesAllArch(Ver, Package, Version) == false) return false; } else { - if (NewProvides(Ver, Package, Arch, Version) == false) + if (NewProvides(Ver, Package, Arch, Version, 0) == false) return false; } @@ -890,7 +890,7 @@ bool debListParser::NewProvidesAllArch(pkgCache::VerIterator &Ver, string const for (std::vector::const_iterator a = Architectures.begin(); a != Architectures.end(); ++a) { - if (NewProvides(Ver, Package, *a, Version) == false) + if (NewProvides(Ver, Package, *a, Version, pkgCache::Flag::MultiArchImplicit) == false) return false; } return true; diff --git a/apt-pkg/edsp.cc b/apt-pkg/edsp.cc index 5c53581fe..34b0b0cc7 100644 --- a/apt-pkg/edsp.cc +++ b/apt-pkg/edsp.cc @@ -102,11 +102,11 @@ static void WriteScenarioDependency( FILE* output, pkgCache::VerIterator const & bool orGroup = false; for (pkgCache::DepIterator Dep = Ver.DependsList(); Dep.end() == false; ++Dep) { - if (Dep.IsMultiArchImplicit() == true) + if (Dep.IsImplicit() == true) continue; if (orGroup == false) dependencies[Dep->Type].append(", "); - dependencies[Dep->Type].append(Dep.TargetPkg().Name()); + dependencies[Dep->Type].append(Dep.TargetPkg().FullName((Dep->CompareOp & pkgCache::Dep::ArchSpecific) != pkgCache::Dep::ArchSpecific)); if (Dep->Version != 0) dependencies[Dep->Type].append(" (").append(pkgCache::CompTypeDeb(Dep->CompareOp)).append(" ").append(Dep.TargetVer()).append(")"); if ((Dep->CompareOp & pkgCache::Dep::Or) == pkgCache::Dep::Or) @@ -140,7 +140,7 @@ static void WriteScenarioLimitedDependency(FILE* output, bool orGroup = false; for (pkgCache::DepIterator Dep = Ver.DependsList(); Dep.end() == false; ++Dep) { - if (Dep.IsMultiArchImplicit() == true) + if (Dep.IsImplicit() == true) continue; if (orGroup == false) { @@ -156,7 +156,7 @@ static void WriteScenarioLimitedDependency(FILE* output, orGroup = false; continue; } - dependencies[Dep->Type].append(Dep.TargetPkg().Name()); + dependencies[Dep->Type].append(Dep.TargetPkg().FullName((Dep->CompareOp & pkgCache::Dep::ArchSpecific) != pkgCache::Dep::ArchSpecific)); if (Dep->Version != 0) dependencies[Dep->Type].append(" (").append(pkgCache::CompTypeDeb(Dep->CompareOp)).append(" ").append(Dep.TargetVer()).append(")"); if ((Dep->CompareOp & pkgCache::Dep::Or) == pkgCache::Dep::Or) diff --git a/apt-pkg/pkgcache.cc b/apt-pkg/pkgcache.cc index 5034ee38a..e8c95738e 100644 --- a/apt-pkg/pkgcache.cc +++ b/apt-pkg/pkgcache.cc @@ -727,21 +727,7 @@ bool pkgCache::DepIterator::IsIgnorable(PkgIterator const &PT) const // ignore group-conflict on a M-A:same package - but not our implicit dependencies // so that we can have M-A:same packages conflicting with their own real name if ((PV->MultiArch & pkgCache::Version::Same) == pkgCache::Version::Same) - { - // Replaces: ${self}:other ( << ${binary:Version}) - if (S2->Type == pkgCache::Dep::Replaces) - { - if (S2->CompareOp == pkgCache::Dep::Less && strcmp(PV.VerStr(), TargetVer()) == 0) - return false; - } - // Breaks: ${self}:other (!= ${binary:Version}) - else if (S2->Type == pkgCache::Dep::DpkgBreaks) - { - if (S2->CompareOp == pkgCache::Dep::NotEquals && strcmp(PV.VerStr(), TargetVer()) == 0) - return false; - } - return true; - } + return IsMultiArchImplicit() == false; return false; } @@ -756,27 +742,12 @@ bool pkgCache::DepIterator::IsIgnorable(PrvIterator const &Prv) const if (Prv.OwnerPkg()->Group == Pkg->Group) return true; // Implicit group-conflicts should not be applied on providers of other groups - if (Pkg->Group == TargetPkg()->Group && Prv.OwnerPkg()->Group != Pkg->Group) + if (IsMultiArchImplicit() && Prv.OwnerPkg()->Group != Pkg->Group) return true; return false; } /*}}}*/ -// DepIterator::IsMultiArchImplicit - added by the cache generation /*{{{*/ -// --------------------------------------------------------------------- -/* MultiArch can be translated to SingleArch for an resolver and we did so, - by adding dependencies to help the resolver understand the problem, but - sometimes it is needed to identify these to ignore them… */ -bool pkgCache::DepIterator::IsMultiArchImplicit() const -{ - if (ParentPkg()->Arch != TargetPkg()->Arch && - (S2->Type == pkgCache::Dep::Replaces || - S2->Type == pkgCache::Dep::DpkgBreaks || - S2->Type == pkgCache::Dep::Conflicts)) - return true; - return false; -} - /*}}}*/ // DepIterator::IsSatisfied - check if a version satisfied the dependency /*{{{*/ bool pkgCache::DepIterator::IsSatisfied(VerIterator const &Ver) const { @@ -787,6 +758,20 @@ bool pkgCache::DepIterator::IsSatisfied(PrvIterator const &Prv) const return Owner->VS->CheckDep(Prv.ProvideVersion(),S2->CompareOp,TargetVer()); } /*}}}*/ +// DepIterator::IsImplicit - added by the cache generation /*{{{*/ +bool pkgCache::DepIterator::IsImplicit() const +{ + if (IsMultiArchImplicit() == true) + return true; + if (IsNegative() || S2->Type == pkgCache::Dep::Replaces) + { + if ((S2->CompareOp & pkgCache::Dep::ArchSpecific) != pkgCache::Dep::ArchSpecific && + strcmp(ParentPkg().Arch(), TargetPkg().Arch()) != 0) + return true; + } + return false; +} + /*}}}*/ // ostream operator to handle string representation of a dependecy /*{{{*/ // --------------------------------------------------------------------- /* */ @@ -1067,19 +1052,5 @@ pkgCache::DescIterator pkgCache::VerIterator::TranslatedDescription() const } /*}}}*/ -// PrvIterator::IsMultiArchImplicit - added by the cache generation /*{{{*/ -// --------------------------------------------------------------------- -/* MultiArch can be translated to SingleArch for an resolver and we did so, - by adding provides to help the resolver understand the problem, but - sometimes it is needed to identify these to ignore them… */ -bool pkgCache::PrvIterator::IsMultiArchImplicit() const -{ - pkgCache::PkgIterator const Owner = OwnerPkg(); - pkgCache::PkgIterator const Parent = ParentPkg(); - if (strcmp(Owner.Arch(), Parent.Arch()) != 0 || Owner.Group()->Name == Parent.Group()->Name) - return true; - return false; -} - /*}}}*/ pkgCache::~pkgCache() {} diff --git a/apt-pkg/pkgcache.h b/apt-pkg/pkgcache.h index fba692982..8a726085e 100644 --- a/apt-pkg/pkgcache.h +++ b/apt-pkg/pkgcache.h @@ -149,8 +149,12 @@ class pkgCache /*{{{*/ The lower 4 bits are used to indicate what operator is being specified and the upper 4 bits are flags. OR indicates that the next package is or'd with the current package. */ - enum DepCompareOp {Or=0x10,NoOp=0,LessEq=0x1,GreaterEq=0x2,Less=0x3, - Greater=0x4,Equals=0x5,NotEquals=0x6}; + enum DepCompareOp {NoOp=0,LessEq=0x1,GreaterEq=0x2,Less=0x3, + Greater=0x4,Equals=0x5,NotEquals=0x6, + Or=0x10, /*!< or'ed with the next dependency */ + MultiArchImplicit=0x20, /*!< generated internally, not spelled out in the index */ + ArchSpecific=0x40 /*!< was decorated with an explicit architecture in index */ + }; }; struct State @@ -178,6 +182,10 @@ class pkgCache /*{{{*/ NotAutomatic=(1<<0), /*!< archive has a default pin of 1 */ ButAutomaticUpgrades=(1<<1), /*!< (together with the previous) archive has a default pin of 100 */ }; + enum ProvidesFlags { + MultiArchImplicit=pkgCache::Dep::MultiArchImplicit, /*!< generated internally, not spelled out in the index */ + ArchSpecific=pkgCache::Dep::ArchSpecific /*!< was decorated with an explicit architecture in index */ + }; }; protected: @@ -725,9 +733,9 @@ struct pkgCache::Provides /** \brief version in the provides line (if any) This version allows dependencies to depend on specific versions of a - Provides, as well as allowing Provides to override existing packages. - This is experimental. Note that Debian doesn't allow versioned provides */ + Provides, as well as allowing Provides to override existing packages. */ map_stringitem_t ProvideVersion; + map_flags_t Flags; /** \brief next provides (based of package) */ map_pointer_t NextProvides; // Provides /** \brief next provides (based of version) */ diff --git a/apt-pkg/pkgcachegen.cc b/apt-pkg/pkgcachegen.cc index a82483d15..9acb2563a 100644 --- a/apt-pkg/pkgcachegen.cc +++ b/apt-pkg/pkgcachegen.cc @@ -709,16 +709,16 @@ bool pkgCacheGenerator::AddImplicitDepends(pkgCache::GrpIterator &G, { // Replaces: ${self}:other ( << ${binary:Version}) NewDepends(D, V, VerStrIdx, - pkgCache::Dep::Less, pkgCache::Dep::Replaces, + pkgCache::Dep::Less | pkgCache::Dep::MultiArchImplicit, pkgCache::Dep::Replaces, OldDepLast); // Breaks: ${self}:other (!= ${binary:Version}) NewDepends(D, V, VerStrIdx, - pkgCache::Dep::NotEquals, pkgCache::Dep::DpkgBreaks, + pkgCache::Dep::NotEquals | pkgCache::Dep::MultiArchImplicit, pkgCache::Dep::DpkgBreaks, OldDepLast); } else { // Conflicts: ${self}:other NewDepends(D, V, 0, - pkgCache::Dep::NoOp, pkgCache::Dep::Conflicts, + pkgCache::Dep::NoOp | pkgCache::Dep::MultiArchImplicit, pkgCache::Dep::Conflicts, OldDepLast); } } @@ -737,16 +737,16 @@ bool pkgCacheGenerator::AddImplicitDepends(pkgCache::VerIterator &V, map_stringitem_t const VerStrIdx = V->VerStr; // Replaces: ${self}:other ( << ${binary:Version}) NewDepends(D, V, VerStrIdx, - pkgCache::Dep::Less, pkgCache::Dep::Replaces, + pkgCache::Dep::Less | pkgCache::Dep::MultiArchImplicit, pkgCache::Dep::Replaces, OldDepLast); // Breaks: ${self}:other (!= ${binary:Version}) NewDepends(D, V, VerStrIdx, - pkgCache::Dep::NotEquals, pkgCache::Dep::DpkgBreaks, + pkgCache::Dep::NotEquals | pkgCache::Dep::MultiArchImplicit, pkgCache::Dep::DpkgBreaks, OldDepLast); } else { // Conflicts: ${self}:other NewDepends(D, V, 0, - pkgCache::Dep::NoOp, pkgCache::Dep::Conflicts, + pkgCache::Dep::NoOp | pkgCache::Dep::MultiArchImplicit, pkgCache::Dep::Conflicts, OldDepLast); } return true; @@ -913,8 +913,8 @@ map_pointer_t pkgCacheGenerator::NewDescription(pkgCache::DescIterator &Desc, bool pkgCacheGenerator::NewDepends(pkgCache::PkgIterator &Pkg, pkgCache::VerIterator &Ver, string const &Version, - unsigned int const &Op, - unsigned int const &Type, + uint8_t const Op, + uint8_t const Type, map_stringitem_t* &OldDepLast) { map_stringitem_t index = 0; @@ -940,8 +940,8 @@ bool pkgCacheGenerator::NewDepends(pkgCache::PkgIterator &Pkg, bool pkgCacheGenerator::NewDepends(pkgCache::PkgIterator &Pkg, pkgCache::VerIterator &Ver, map_pointer_t const Version, - unsigned int const &Op, - unsigned int const &Type, + uint8_t const Op, + uint8_t const Type, map_pointer_t* &OldDepLast) { void const * const oldMap = Map.Data(); @@ -1040,8 +1040,8 @@ bool pkgCacheGenerator::ListParser::NewDepends(pkgCache::VerIterator &Ver, const string &PackageName, const string &Arch, const string &Version, - unsigned int Op, - unsigned int Type) + uint8_t const Op, + uint8_t const Type) { pkgCache::GrpIterator Grp; Dynamic DynGrp(Grp); @@ -1073,12 +1073,11 @@ bool pkgCacheGenerator::ListParser::NewDepends(pkgCache::VerIterator &Ver, } /*}}}*/ // ListParser::NewProvides - Create a Provides element /*{{{*/ -// --------------------------------------------------------------------- -/* */ bool pkgCacheGenerator::ListParser::NewProvides(pkgCache::VerIterator &Ver, - const string &PkgName, + const string &PkgName, const string &PkgArch, - const string &Version) + const string &Version, + uint8_t const Flags) { pkgCache &Cache = Owner->Cache; @@ -1097,6 +1096,7 @@ bool pkgCacheGenerator::ListParser::NewProvides(pkgCache::VerIterator &Ver, pkgCache::PrvIterator Prv(Cache,Cache.ProvideP + Provides,Cache.PkgP); Dynamic DynPrv(Prv); Prv->Version = Ver.Index(); + Prv->Flags = Flags; Prv->NextPkgProv = Ver->ProvidesList; Ver->ProvidesList = Prv.Index(); if (Version.empty() == false) { diff --git a/apt-pkg/pkgcachegen.h b/apt-pkg/pkgcachegen.h index c56b5abae..c5527ff30 100644 --- a/apt-pkg/pkgcachegen.h +++ b/apt-pkg/pkgcachegen.h @@ -82,11 +82,11 @@ class APT_HIDDEN pkgCacheGenerator /*{{{*/ bool NewFileVer(pkgCache::VerIterator &Ver,ListParser &List); bool NewFileDesc(pkgCache::DescIterator &Desc,ListParser &List); bool NewDepends(pkgCache::PkgIterator &Pkg, pkgCache::VerIterator &Ver, - std::string const &Version, unsigned int const &Op, - unsigned int const &Type, map_pointer_t* &OldDepLast); + std::string const &Version, uint8_t const Op, + uint8_t const Type, map_pointer_t* &OldDepLast); bool NewDepends(pkgCache::PkgIterator &Pkg, pkgCache::VerIterator &Ver, - map_pointer_t const Version, unsigned int const &Op, - unsigned int const &Type, map_pointer_t* &OldDepLast); + map_pointer_t const Version, uint8_t const Op, + uint8_t const Type, map_pointer_t* &OldDepLast); map_pointer_t NewVersion(pkgCache::VerIterator &Ver,const std::string &VerStr,map_pointer_t const Next) APT_DEPRECATED { return NewVersion(Ver, VerStr, 0, 0, Next); } map_pointer_t NewVersion(pkgCache::VerIterator &Ver,const std::string &VerStr, @@ -162,10 +162,11 @@ class APT_HIDDEN pkgCacheGenerator::ListParser inline map_stringitem_t WriteString(const std::string &S) {return Owner->WriteStringInMap(S);}; inline map_stringitem_t WriteString(const char *S,unsigned int Size) {return Owner->WriteStringInMap(S,Size);}; bool NewDepends(pkgCache::VerIterator &Ver,const std::string &Package, const std::string &Arch, - const std::string &Version,unsigned int Op, - unsigned int Type); + const std::string &Version,uint8_t const Op, + uint8_t const Type); bool NewProvides(pkgCache::VerIterator &Ver,const std::string &PkgName, - const std::string &PkgArch, const std::string &Version); + const std::string &PkgArch, const std::string &Version, + uint8_t const Flags); public: diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index 71dceb559..cfdc13259 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -51,6 +51,7 @@ static bool addArgumentsAPTCache(std::vector &Args, char cons addArg(0, "conflicts", "APT::Cache::ShowConflicts", 0); addArg(0, "enhances", "APT::Cache::ShowEnhances", 0); addArg(0, "recurse", "APT::Cache::RecurseDepends", 0); + addArg(0, "implicit", "APT::Cache::ShowImplicit", 0); } else if (CmdMatches("search")) { diff --git a/cmdline/apt-cache.cc b/cmdline/apt-cache.cc index 2fc721f69..1eb891e8e 100644 --- a/cmdline/apt-cache.cc +++ b/cmdline/apt-cache.cc @@ -689,6 +689,7 @@ static bool ShowDepends(CommandLine &CmdL, bool const RevDepends) bool const ShowBreaks = _config->FindB("APT::Cache::ShowBreaks", Important == false); bool const ShowEnhances = _config->FindB("APT::Cache::ShowEnhances", Important == false); bool const ShowOnlyFirstOr = _config->FindB("APT::Cache::ShowOnlyFirstOr", false); + bool const ShowImplicit = _config->FindB("APT::Cache::ShowImplicit", false); while (verset.empty() != true) { @@ -709,12 +710,16 @@ static bool ShowDepends(CommandLine &CmdL, bool const RevDepends) case pkgCache::Dep::Depends: if (!ShowDepends) continue; break; case pkgCache::Dep::Recommends: if (!ShowRecommends) continue; break; case pkgCache::Dep::Suggests: if (!ShowSuggests) continue; break; - case pkgCache::Dep::Replaces: if (!ShowReplaces) continue; break; case pkgCache::Dep::Conflicts: if (!ShowConflicts) continue; break; + case pkgCache::Dep::Replaces: if (!ShowReplaces) continue; break; + case pkgCache::Dep::Conflicts: if (!ShowConflicts) continue; break; case pkgCache::Dep::DpkgBreaks: if (!ShowBreaks) continue; break; case pkgCache::Dep::Enhances: if (!ShowEnhances) continue; break; } + if (ShowImplicit == false && D.IsImplicit()) + continue; pkgCache::PkgIterator Trg = RevDepends ? D.ParentPkg() : D.TargetPkg(); + bool const showNoArch = RevDepends || (D->CompareOp & pkgCache::Dep::ArchSpecific) != pkgCache::Dep::ArchSpecific; if((Installed && Trg->CurrentVer != 0) || !Installed) { @@ -728,9 +733,9 @@ static bool ShowDepends(CommandLine &CmdL, bool const RevDepends) if (ShowDepType == true) cout << D.DepType() << ": "; if (Trg->VersionList == 0) - cout << "<" << Trg.FullName(true) << ">"; + cout << "<" << Trg.FullName(showNoArch) << ">"; else - cout << Trg.FullName(true); + cout << Trg.FullName(showNoArch); if (ShowVersion == true && D->Version != 0) cout << " (" << pkgCache::CompTypeDeb(D->CompareOp) << ' ' << D.TargetVer() << ')'; cout << std::endl; diff --git a/doc/apt-cache.8.xml b/doc/apt-cache.8.xml index a9f6c8da2..e20f66770 100644 --- a/doc/apt-cache.8.xml +++ b/doc/apt-cache.8.xml @@ -281,12 +281,23 @@ Reverse Provides: - Per default the depends and - rdepends print all dependencies. This can be tweaked with + Per default the depends and + rdepends print all dependencies. This can be tweaked with these flags which will omit the specified dependency type. Configuration Item: APT::Cache::ShowDependencyType e.g. APT::Cache::ShowRecommends. + + + Per default depends and rdepends + print only dependencies explicitly expressed in the metadata. With this flag + it will also show dependencies implicitely added based on the encountered data. + A Conflicts: foo e.g. expresses implicitely that this package + also conflicts with the package foo from any other architecture. + Configuration Item: APT::Cache::ShowImplicit. + + + Print full package records when searching. Configuration Item: APT::Cache::ShowFull. diff --git a/test/integration/test-apt-cache b/test/integration/test-apt-cache index a22b08c20..a8ddfd889 100755 --- a/test/integration/test-apt-cache +++ b/test/integration/test-apt-cache @@ -16,6 +16,9 @@ Recommends: cool (>= 2) | cooler (<< 5)' "$DESCR" insertpackage 'unstable' 'bar' 'all' '1' 'Depends: bar Breaks: foo (<< 1) Replaces: foo (<< 1)' "$DESCR" +insertpackage 'unstable' 'specific' 'all' '1' 'Depends: bar:i386, specific:amd64 +Breaks: foo:amd64 (<< 1) +Replaces: foo:i386 (<< 1)' "$DESCR" setupaptarchive @@ -44,6 +47,7 @@ testsuccess test -s dump.output testsuccessequal 'dpkg bar +specific fancy foo' aptcache pkgnames testsuccessequal 'bar' aptcache pkgnames bar @@ -57,29 +61,60 @@ testsuccessequal " foo | 1 | file:$(readlink -f .)/aptarchive uns testsuccessequal 'foo Depends: bar Conflicts: - Conflicts: |Recommends: Recommends: ' aptcache depends foo testsuccessequal 'foo Depends: bar Conflicts: Conflicts: + |Recommends: + Recommends: ' aptcache depends foo --implicit +testsuccessequal 'foo + Depends: bar + Conflicts: Recommends: ' aptcache depends foo -o APT::Cache::ShowOnlyFirstOr=1 testsuccessequal 'foo Depends: bar Conflicts: Conflicts: + Recommends: ' aptcache depends foo -o APT::Cache::ShowOnlyFirstOr=1 --implicit +testsuccessequal 'foo + Depends: bar + Conflicts: |Recommends: (>= 2) Recommends: (<< 5)' aptcache depends foo -o APT::Cache::ShowVersion=1 testsuccessequal 'foo Depends: bar Conflicts: - Conflicts: ' aptcache depends foo --no-recommends + Conflicts: + |Recommends: (>= 2) + Recommends: (<< 5)' aptcache depends foo -o APT::Cache::ShowVersion=1 --implicit testsuccessequal 'foo - Depends: bar' aptcache depends foo --important + Depends: bar + Conflicts: ' aptcache depends foo --no-recommends +testsuccessequal 'foo + Depends: bar + Conflicts: + Conflicts: ' aptcache depends foo --no-recommends --implicit +testsuccessequal 'foo + Depends: bar' aptcache depends foo --important --implicit testsuccessequal 'foo + Conflicts: ' aptcache depends foo --important --no-depends --conflicts +testsuccessequal 'foo + Conflicts: + Conflicts: ' aptcache depends foo --important --no-depends --conflicts --implicit +testsuccessequal 'foo + Depends: bar Conflicts: - Conflicts: ' aptcache depends foo --important --no-depends --conflicts + |Recommends: + Recommends: +bar + Depends: bar + Breaks: foo + Replaces: foo + + +' aptcache depends foo --recurse testsuccessequal 'foo Depends: bar Conflicts: @@ -96,29 +131,58 @@ bar -' aptcache depends foo --recurse +' aptcache depends foo --recurse --implicit +testsuccessequal 'foo + Depends: bar +bar + Depends: bar + Replaces: foo' aptcache depends foo --recurse --important --replaces testsuccessequal 'foo Depends: bar bar Depends: bar Replaces: foo Replaces: -' aptcache depends foo --recurse --important --replaces +' aptcache depends foo --recurse --important --replaces --implicit +testsuccessequal 'bar + Depends: bar + Breaks: foo + Replaces: foo' aptcache depends bar +testsuccessequal 'bar + Depends: bar + Breaks: foo + Breaks: + Replaces: foo + Replaces: ' aptcache depends bar --implicit +testsuccessequal 'specific + Depends: + Depends: specific:amd64 + Breaks: foo:amd64 + Replaces: ' aptcache depends specific +testsuccessequal 'specific + Depends: + Depends: specific:amd64 + Breaks: foo:amd64 + Replaces: ' aptcache depends specific --implicit ## rdpends testsuccessequal 'foo Reverse Depends: bar + specific bar' aptcache rdepends foo testsuccessequal 'foo Reverse Depends: Breaks: bar + Breaks: specific Replaces: bar' aptcache rdepends foo -o APT::Cache::ShowDependencyType=1 testsuccessequal 'foo Reverse Depends: Breaks: bar (<< 1) + Breaks: specific (<< 1) Replaces: bar (<< 1)' aptcache rdepends foo -o APT::Cache::ShowDependencyType=1 -o APT::Cache::ShowVersion=1 testsuccessequal 'foo Reverse Depends: - Breaks: bar (<< 1)' aptcache rdepends foo -o APT::Cache::ShowDependencyType=1 -o APT::Cache::ShowVersion=1 --important --breaks + Breaks: bar (<< 1) + Breaks: specific (<< 1)' aptcache rdepends foo -o APT::Cache::ShowDependencyType=1 -o APT::Cache::ShowVersion=1 --important --breaks diff --git a/test/integration/test-ordering-ignore-not-matching-breaks b/test/integration/test-ordering-ignore-not-matching-breaks deleted file mode 100755 index 7c1365bdd..000000000 --- a/test/integration/test-ordering-ignore-not-matching-breaks +++ /dev/null @@ -1,56 +0,0 @@ -#!/bin/sh -set -e - -TESTDIR=$(readlink -f $(dirname $0)) -. $TESTDIR/framework -setupenvironment -configarchitecture 'amd64' 'i386' - -insertpackage 'unstable-mp' 'crda' 'i386,amd64' '1.1.1-1ubuntu4mp' 'Provides: wireless-crda -Multi-Arch: foreign' -insertpackage 'unstable-m' 'crda' 'i386,amd64' '1.1.1-1ubuntu4m' 'Multi-Arch: foreign' -insertpackage 'unstable-p' 'crda' 'i386,amd64' '1.1.1-1ubuntu4p' 'Provides: wireless-crda' -insertpackage 'unstable' 'wireless-crda' 'i386,amd64' '1.16' - - -insertinstalledpackage 'wireless-crda' 'amd64' '1.14' - -setupaptarchive - -testsuccessequal 'Reading package lists... -Building dependency tree... -The following NEW packages will be installed: - crda -0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. -Inst crda (1.1.1-1ubuntu4m unstable-m [amd64]) -Conf crda (1.1.1-1ubuntu4m unstable-m [amd64])' aptget install crda -s -t unstable-m - -testsuccessequal 'Reading package lists... -Building dependency tree... -The following NEW packages will be installed: - crda -0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. -Inst crda (1.1.1-1ubuntu4p unstable-p [amd64]) -Conf crda (1.1.1-1ubuntu4p unstable-p [amd64])' aptget install crda -s -t unstable-p - -testsuccessequal 'Reading package lists... -Building dependency tree... -The following NEW packages will be installed: - crda -0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. -Inst crda (1.1.1-1ubuntu4mp unstable-mp [amd64]) -Conf crda (1.1.1-1ubuntu4mp unstable-mp [amd64])' aptget install crda -s -t unstable-mp - -rm rootdir/var/lib/dpkg/status -insertinstalledpackage 'crda' 'amd64' '1.1.1-1ubuntu4mp' 'Provides: wireless-crda -Conflicts: wireless-crda (<< 1.15) -Replaces: wireless-crda ( << 1.15) -Multi-arch: foreign' - -testsuccessequal 'Reading package lists... -Building dependency tree... -The following NEW packages will be installed: - wireless-crda -0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. -Inst wireless-crda (1.16 unstable [amd64]) -Conf wireless-crda (1.16 unstable [amd64])' aptget install wireless-crda -s -t unstable -- cgit v1.2.3-70-g09d2 From 26677b9c25b318c291d778eec758a06471fcba26 Mon Sep 17 00:00:00 2001 From: Guillem Jover Date: Tue, 11 Aug 2015 17:41:27 +0200 Subject: po-fixups: fill Project-Id-Version and Encoding correctly Closes: 612996 --- doc/po/es.po | 2 +- doc/po/fr.po | 2 +- doc/po/it.po | 2 +- doc/po/ja.po | 2 +- doc/po/pl.po | 2 +- doc/po/pt.po | 2 +- doc/po/pt_BR.po | 2 +- po/ar.po | 2 +- po/cs.po | 2 +- po/cy.po | 2 +- po/dz.po | 2 +- po/el.po | 2 +- po/eu.po | 2 +- po/fr.po | 2 +- po/gl.po | 2 +- po/hu.po | 2 +- po/it.po | 2 +- po/km.po | 2 +- po/ko.po | 2 +- po/ku.po | 2 +- po/lt.po | 2 +- po/mr.po | 2 +- po/ne.po | 2 +- po/nn.po | 2 +- po/pt.po | 2 +- po/pt_BR.po | 2 +- po/ro.po | 2 +- po/sk.po | 2 +- po/sv.po | 2 +- po/th.po | 2 +- po/tl.po | 2 +- po/tr.po | 2 +- po/uk.po | 2 +- po/zh_TW.po | 2 +- 34 files changed, 34 insertions(+), 34 deletions(-) (limited to 'doc') diff --git a/doc/po/es.po b/doc/po/es.po index 590ae4aeb..31b22ed00 100644 --- a/doc/po/es.po +++ b/doc/po/es.po @@ -36,7 +36,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt 0.9.7.1\n" +"Project-Id-Version: apt-doc 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2014-11-06 09:45+0100\n" "PO-Revision-Date: 2014-07-04 01:31+0200\n" diff --git a/doc/po/fr.po b/doc/po/fr.po index 85869a2b7..0c9e3a346 100644 --- a/doc/po/fr.po +++ b/doc/po/fr.po @@ -9,7 +9,7 @@ # Jean-Pierre Giraud , 2014. msgid "" msgstr "" -"Project-Id-Version: \n" +"Project-Id-Version: apt-doc 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2014-11-15 17:26+0100\n" diff --git a/doc/po/it.po b/doc/po/it.po index 28e97f14b..fabe6c544 100644 --- a/doc/po/it.po +++ b/doc/po/it.po @@ -8,7 +8,7 @@ # Beatrice Torracca , 2012, 2014, 2015. msgid "" msgstr "" -"Project-Id-Version: \n" +"Project-Id-Version: apt-doc 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-10 09:48+0100\n" "PO-Revision-Date: 2015-01-27 14:11+0200\n" diff --git a/doc/po/ja.po b/doc/po/ja.po index 3e0712125..f504436c0 100644 --- a/doc/po/ja.po +++ b/doc/po/ja.po @@ -6,7 +6,7 @@ # KURASAWA Nozomu, 2003-2006, 2009-2012. msgid "" msgstr "" -"Project-Id-Version: apt 1.0.6\n" +"Project-Id-Version: apt-doc 1.0.6\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2014-11-06 09:45+0100\n" "PO-Revision-Date: 2014-07-10 19:52+0900\n" diff --git a/doc/po/pl.po b/doc/po/pl.po index 481d85af6..2b9114c6e 100644 --- a/doc/po/pl.po +++ b/doc/po/pl.po @@ -9,7 +9,7 @@ # Robert Luberda 2000-2004, 2010, 2012. msgid "" msgstr "" -"Project-Id-Version: apt 0.9.7.3\n" +"Project-Id-Version: apt-doc 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2014-11-06 09:45+0100\n" "PO-Revision-Date: 2014-07-04 02:13+0200\n" diff --git a/doc/po/pt.po b/doc/po/pt.po index 3aefcc2a9..82d470426 100644 --- a/doc/po/pt.po +++ b/doc/po/pt.po @@ -5,7 +5,7 @@ # Américo Monteiro , 2009 - 2014. msgid "" msgstr "" -"Project-Id-Version: apt 1.0.7\n" +"Project-Id-Version: apt-doc 1.0.7\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2014-11-06 09:45+0100\n" "PO-Revision-Date: 2014-08-29 00:34+0100\n" diff --git a/doc/po/pt_BR.po b/doc/po/pt_BR.po index 88b479e75..44e2f17f8 100644 --- a/doc/po/pt_BR.po +++ b/doc/po/pt_BR.po @@ -7,7 +7,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt-doc 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2014-11-06 09:45+0100\n" "PO-Revision-Date: 2004-09-20 17:02+0000\n" diff --git a/po/ar.po b/po/ar.po index 8c1622d91..9bf40386f 100644 --- a/po/ar.po +++ b/po/ar.po @@ -4,7 +4,7 @@ # Ossama M. Khayat , 2005, 2006. msgid "" msgstr "" -"Project-Id-Version: apt_po\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2006-10-20 21:28+0300\n" diff --git a/po/cs.po b/po/cs.po index 4de2b7006..2c732fc29 100644 --- a/po/cs.po +++ b/po/cs.po @@ -5,7 +5,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2014-10-05 06:09+0200\n" diff --git a/po/cy.po b/po/cy.po index db9ec0bbc..7d2d6366e 100644 --- a/po/cy.po +++ b/po/cy.po @@ -4,7 +4,7 @@ # msgid "" msgstr "" -"Project-Id-Version: APT\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2005-06-06 13:46+0100\n" diff --git a/po/dz.po b/po/dz.po index e27ea6a90..12b6499f1 100644 --- a/po/dz.po +++ b/po/dz.po @@ -4,7 +4,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt_po.pot\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2006-09-19 09:49+0530\n" diff --git a/po/el.po b/po/el.po index b2370c6ad..bc3a784b9 100644 --- a/po/el.po +++ b/po/el.po @@ -14,7 +14,7 @@ # Θανάσης Νάτσης , 2012. msgid "" msgstr "" -"Project-Id-Version: apt_po_el\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2008-08-26 18:25+0300\n" diff --git a/po/eu.po b/po/eu.po index 528c6cc51..9e1647035 100644 --- a/po/eu.po +++ b/po/eu.po @@ -5,7 +5,7 @@ # Piarres Beobide , 2005, 2006, 2007, 2008, 2009. msgid "" msgstr "" -"Project-Id-Version: apt_po_eu\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2009-05-17 00:41+0200\n" diff --git a/po/fr.po b/po/fr.po index 0a437907f..0e3ae69b1 100644 --- a/po/fr.po +++ b/po/fr.po @@ -7,7 +7,7 @@ # Julien Patriarca , 2013. msgid "" msgstr "" -"Project-Id-Version: fr\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2013-12-15 16:45+0100\n" diff --git a/po/gl.po b/po/gl.po index 9ec10122b..9ad26c4e7 100644 --- a/po/gl.po +++ b/po/gl.po @@ -8,7 +8,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt_po_gl\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2011-05-12 15:28+0100\n" diff --git a/po/hu.po b/po/hu.po index fcf53f727..9475a9f42 100644 --- a/po/hu.po +++ b/po/hu.po @@ -5,7 +5,7 @@ # Gabor Kelemen , 2004, 2005, 2011, 2012. msgid "" msgstr "" -"Project-Id-Version: apt trunk\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2012-06-25 17:09+0200\n" diff --git a/po/it.po b/po/it.po index 6d9793c55..ca7a11027 100644 --- a/po/it.po +++ b/po/it.po @@ -6,7 +6,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-04-11 08:08+0200\n" "PO-Revision-Date: 2015-04-07 16:51+0100\n" diff --git a/po/km.po b/po/km.po index 8ae605510..263c30dea 100644 --- a/po/km.po +++ b/po/km.po @@ -8,7 +8,7 @@ # Khoem Sokhem , 2006. msgid "" msgstr "" -"Project-Id-Version: apt_po_km\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2006-10-10 09:48+0700\n" diff --git a/po/ko.po b/po/ko.po index e5b08fdec..969736cca 100644 --- a/po/ko.po +++ b/po/ko.po @@ -3,7 +3,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2010-08-30 02:31+0900\n" diff --git a/po/ku.po b/po/ku.po index 3b2d6ed12..f66836634 100644 --- a/po/ku.po +++ b/po/ku.po @@ -6,7 +6,7 @@ # Erdal Ronahi , 2008. msgid "" msgstr "" -"Project-Id-Version: apt-ku\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2008-05-08 12:48+0200\n" diff --git a/po/lt.po b/po/lt.po index 6d0365183..67ff3c8fb 100644 --- a/po/lt.po +++ b/po/lt.po @@ -6,7 +6,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2008-08-02 01:47-0400\n" diff --git a/po/mr.po b/po/mr.po index 31bdb63c9..69c133592 100644 --- a/po/mr.po +++ b/po/mr.po @@ -4,7 +4,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2008-11-20 23:27+0530\n" diff --git a/po/ne.po b/po/ne.po index b5fb2c6fc..83ffc1d37 100644 --- a/po/ne.po +++ b/po/ne.po @@ -4,7 +4,7 @@ # Shiva Pokharel , 2006. msgid "" msgstr "" -"Project-Id-Version: apt_po\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2006-06-12 14:35+0545\n" diff --git a/po/nn.po b/po/nn.po index 5c8c39967..1f079e98a 100644 --- a/po/nn.po +++ b/po/nn.po @@ -7,7 +7,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt_nn\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2005-02-14 23:30+0100\n" diff --git a/po/pt.po b/po/pt.po index 0e106a7d1..28b8be563 100644 --- a/po/pt.po +++ b/po/pt.po @@ -5,7 +5,7 @@ # Miguel Figueiredo , 2005-2012. msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2012-06-29 15:45+0100\n" diff --git a/po/pt_BR.po b/po/pt_BR.po index c50792b79..ad8c325c4 100644 --- a/po/pt_BR.po +++ b/po/pt_BR.po @@ -5,7 +5,7 @@ # Felipe Augusto van de Wiel (faw) , 2006-2008. msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2008-11-17 02:33-0200\n" diff --git a/po/ro.po b/po/ro.po index 7b94bd3d8..68628517b 100644 --- a/po/ro.po +++ b/po/ro.po @@ -5,7 +5,7 @@ # Eddy Petrișor , 2008. msgid "" msgstr "" -"Project-Id-Version: ro\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2008-11-15 02:21+0200\n" diff --git a/po/sk.po b/po/sk.po index 0aeb167ea..776b139c1 100644 --- a/po/sk.po +++ b/po/sk.po @@ -8,7 +8,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2012-06-28 20:49+0100\n" diff --git a/po/sv.po b/po/sv.po index bdf00ff65..7e7801f84 100644 --- a/po/sv.po +++ b/po/sv.po @@ -7,7 +7,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2010-08-24 21:18+0100\n" diff --git a/po/th.po b/po/th.po index c016205c5..731b1de46 100644 --- a/po/th.po +++ b/po/th.po @@ -6,7 +6,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2014-12-12 13:00+0700\n" diff --git a/po/tl.po b/po/tl.po index be3371704..d6e1059e8 100644 --- a/po/tl.po +++ b/po/tl.po @@ -8,7 +8,7 @@ # msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2007-03-29 21:36+0800\n" diff --git a/po/tr.po b/po/tr.po index 8b8691a55..5a5c4fcd5 100644 --- a/po/tr.po +++ b/po/tr.po @@ -6,7 +6,7 @@ # Rosetta Contributors, 2009. msgid "" msgstr "" -"Project-Id-Version: apt\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-04-13 07:23+0200\n" "PO-Revision-Date: 2015-06-21 16:54+0200\n" diff --git a/po/uk.po b/po/uk.po index 514403641..1494413e9 100644 --- a/po/uk.po +++ b/po/uk.po @@ -10,7 +10,7 @@ # binary = двійковий msgid "" msgstr "" -"Project-Id-Version: apt-all\n" +"Project-Id-Version: apt 1.0.5\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2012-09-25 20:19+0300\n" diff --git a/po/zh_TW.po b/po/zh_TW.po index f37e5e0ed..6b78c8340 100644 --- a/po/zh_TW.po +++ b/po/zh_TW.po @@ -6,7 +6,7 @@ # $Id: zh_TW.po,v 1.11 2004/04/30 04:50:38 mdz Exp $ msgid "" msgstr "" -"Project-Id-Version: 0.5.4\n" +"Project-Id-Version: apt 0.5.4\n" "Report-Msgid-Bugs-To: APT Development Team \n" "POT-Creation-Date: 2015-03-09 02:17+0100\n" "PO-Revision-Date: 2009-01-28 10:41+0800\n" -- cgit v1.2.3-70-g09d2 From 35bf76cf3035761c8721e51250bbeb5f4facf8ee Mon Sep 17 00:00:00 2001 From: David Kalnischkies Date: Tue, 11 Aug 2015 19:24:24 +0200 Subject: remove Dir:: scope limit of RootDir in the documentation RootDir doesn't only effect Dir-scope but all FindDir directories, so document it accordingly. Closes: 659387 --- doc/apt.conf.5.xml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/apt.conf.5.xml b/doc/apt.conf.5.xml index 103d0622c..6b5ebf2d7 100644 --- a/doc/apt.conf.5.xml +++ b/doc/apt.conf.5.xml @@ -690,7 +690,7 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; The configuration item RootDir has a special - meaning. If set, all paths in Dir:: will be + meaning. If set, all paths will be relative to RootDir, even paths that are specified absolutely. So, for instance, if RootDir is set to @@ -699,6 +699,7 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; /var/lib/dpkg/status, then the status file will be looked up in /tmp/staging/var/lib/dpkg/status. + If you want to prefix only relative paths, set Dir instead. -- cgit v1.2.3-70-g09d2 From 29efb9dda712554a2f05eb0475e3cd0a6b8a90fb Mon Sep 17 00:00:00 2001 From: Jonathan Nieder Date: Tue, 11 Aug 2015 22:07:18 +0200 Subject: document VERSION 2 (and 3) pre-install-pkgs hook interface [Commiter: Patch adapted to apply to current version of the manpage and added/moved a few words about Version 3 to make it fit better] Closes: 627188 --- doc/apt.conf.5.xml | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/apt.conf.5.xml b/doc/apt.conf.5.xml index 6b5ebf2d7..d5e185757 100644 --- a/doc/apt.conf.5.xml +++ b/doc/apt.conf.5.xml @@ -775,10 +775,34 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";}; install to the commands, one per line on the requested file descriptor, defaulting to standard input. - Version 2 of this protocol dumps more information, including the - protocol version, the APT configuration space and the packages, files - and versions being changed. Version 3 adds the architecture and MultiArch - flag to each version being dumped. + Version 2 of this protocol sends more information through the requested + file descriptor: a line with the text VERSION 2, + the APT configuration space, and a list of package actions with filename + and version information. + + Each configuration directive line has the form + key=value. Special characters (equal signs, newlines, + nonprintable characters, quotation marks, and percent signs in + key and newlines, nonprintable characters, and percent + signs in value) are %-encoded. Lists are represented + by multiple key::=value lines with the same key. The + configuration section ends with a blank line. + + Package action lines consist of five fields in Version 2: old version, direction + of version change (< for upgrades, > for downgrades, = for no + change), new version, action. The version fields are "-" for no version + at all (for example when installing a package for the first time; no + version is treated as earlier than any real version, so that is an + upgrade, indicated as - < 1.23.4). The action field + is "**CONFIGURE**" if the package is being configured, "**REMOVE**" if it + is being removed, or the filename of a .deb file if it is being + unpacked. + + In Version 3 after each version field follows the architecture + of this version, which is "-" if there is no version, and a field showing + the MultiArch type "same", foreign", "allowed" or "none". Note that "none" + is an incorrect typename which is just kept to remain compatible, it + should be read as "no" and users are encouraged to support both. The version of the protocol to be used for the command cmd can be chosen by setting -- cgit v1.2.3-70-g09d2 From a4256c6bb57dbb59767824133a9a42eceeadc522 Mon Sep 17 00:00:00 2001 From: Guillem Jover Date: Tue, 14 Oct 2014 13:52:33 +0200 Subject: Do not set unhonored DPKG_NO_TSTP variable for dpkg Support for that variable was removed in dpkg in 1.15.6, in commit 6f037003e8b96878b485efb7cbd1f846e3bf4e97. Closes: #765366 --- apt-pkg/deb/dpkgpm.cc | 3 --- doc/dpkg-tech.dbk | 10 +--------- 2 files changed, 1 insertion(+), 12 deletions(-) (limited to 'doc') diff --git a/apt-pkg/deb/dpkgpm.cc b/apt-pkg/deb/dpkgpm.cc index c578cc338..644e4d8e4 100644 --- a/apt-pkg/deb/dpkgpm.cc +++ b/apt-pkg/deb/dpkgpm.cc @@ -1545,9 +1545,6 @@ bool pkgDPkgPM::Go(APT::Progress::PackageManager *progress) _exit(100); } - /* No Job Control Stop Env is a magic dpkg var that prevents it - from using sigstop */ - putenv((char *)"DPKG_NO_TSTP=yes"); execvp(Args[0], (char**) &Args[0]); cerr << "Could not exec dpkg!" << endl; _exit(100); diff --git a/doc/dpkg-tech.dbk b/doc/dpkg-tech.dbk index 2584cf640..f95716cf4 100644 --- a/doc/dpkg-tech.dbk +++ b/doc/dpkg-tech.dbk @@ -404,15 +404,7 @@ As yet unwritten. You can refer to the other manuals for now. See -DPKG_NO_TSTP - if set to a non-null value, this variable causes dpkg to run a -child shell process instead of sending itself a SIGTSTP, when the user selects -to background the dpkg process when it asks about conffiles. - - - - -SHELL - used to determine which shell to run in the case when DPKG_NO_TSTP -is set. +SHELL - used to determine which shell to run. -- cgit v1.2.3-70-g09d2 From 14c50b58e938cf78ce2d32d4ec39979f7575c543 Mon Sep 17 00:00:00 2001 From: Johannes Schauer Date: Sun, 26 Apr 2015 09:58:05 +0200 Subject: use a=experimental instead n=experimental in pin documentation Closes: 783343 --- doc/apt_preferences.5.xml | 2 +- doc/po/apt-doc.pot | 2 +- doc/po/de.po | 4 ++-- doc/po/es.po | 4 ++-- doc/po/fr.po | 4 ++-- doc/po/it.po | 4 ++-- doc/po/ja.po | 4 ++-- doc/po/pl.po | 4 ++-- doc/po/pt.po | 4 ++-- doc/po/pt_BR.po | 2 +- 10 files changed, 17 insertions(+), 17 deletions(-) (limited to 'doc') diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml index 16e6a7aa0..5ea59bf9c 100644 --- a/doc/apt_preferences.5.xml +++ b/doc/apt_preferences.5.xml @@ -267,7 +267,7 @@ expression surrounded by slashes). Package: gnome* /kde/ -Pin: release n=experimental +Pin: release a=experimental Pin-Priority: 500 diff --git a/doc/po/apt-doc.pot b/doc/po/apt-doc.pot index 98c2b66b5..fc3b5a6c6 100644 --- a/doc/po/apt-doc.pot +++ b/doc/po/apt-doc.pot @@ -4241,7 +4241,7 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" diff --git a/doc/po/de.po b/doc/po/de.po index 4eb458d17..12d6a4c4d 100644 --- a/doc/po/de.po +++ b/doc/po/de.po @@ -6071,11 +6071,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/es.po b/doc/po/es.po index 31b22ed00..9ace0bd4f 100644 --- a/doc/po/es.po +++ b/doc/po/es.po @@ -6126,11 +6126,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/fr.po b/doc/po/fr.po index 0c9e3a346..8d7c29f37 100644 --- a/doc/po/fr.po +++ b/doc/po/fr.po @@ -6045,11 +6045,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/it.po b/doc/po/it.po index fabe6c544..21d5611ea 100644 --- a/doc/po/it.po +++ b/doc/po/it.po @@ -6072,11 +6072,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/ja.po b/doc/po/ja.po index f504436c0..5d6c183cb 100644 --- a/doc/po/ja.po +++ b/doc/po/ja.po @@ -5812,11 +5812,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/pl.po b/doc/po/pl.po index 2b9114c6e..3d89db060 100644 --- a/doc/po/pl.po +++ b/doc/po/pl.po @@ -5509,11 +5509,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/pt.po b/doc/po/pt.po index 82d470426..dc91fddca 100644 --- a/doc/po/pt.po +++ b/doc/po/pt.po @@ -6007,11 +6007,11 @@ msgstr "" #, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" #. type: Content of: diff --git a/doc/po/pt_BR.po b/doc/po/pt_BR.po index 44e2f17f8..b6bd0fe09 100644 --- a/doc/po/pt_BR.po +++ b/doc/po/pt_BR.po @@ -4393,7 +4393,7 @@ msgstr "" #, fuzzy, no-wrap msgid "" "Package: gnome* /kde/\n" -"Pin: release n=experimental\n" +"Pin: release a=experimental\n" "Pin-Priority: 500\n" msgstr "" "\n" -- cgit v1.2.3-70-g09d2 From 809aa216c630f1cc61b0c3b9d992d4a3be14be3c Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Wed, 12 Aug 2015 20:44:40 +0200 Subject: policy: Be more strict about parsing pin files, and document prio 0 Treat invalid pin priorities and overflows as an error. Closes: #429912 --- apt-pkg/policy.cc | 13 ++++++++++--- apt-pkg/tagfile.cc | 9 ++++++++- doc/apt_preferences.5.xml | 4 ++++ 3 files changed, 22 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/apt-pkg/policy.cc b/apt-pkg/policy.cc index bf6ec0ff7..76c36b71b 100644 --- a/apt-pkg/policy.cc +++ b/apt-pkg/policy.cc @@ -478,11 +478,18 @@ bool ReadPinFile(pkgPolicy &Plcy,string File) } for (; Word != End && isspace(*Word) != 0; Word++); - short int priority = Tags.FindI("Pin-Priority", 0); + int priority = Tags.FindI("Pin-Priority", 0); + if (priority < std::numeric_limits::min() || + priority > std::numeric_limits::max() || + _error->PendingError()) { + return _error->Error(_("%s: Value %s is outside the range of valid pin priorities (%d to %d)"), + File.c_str(), Tags.FindS("Pin-Priority").c_str(), + std::numeric_limits::min(), + std::numeric_limits::max()); + } if (priority == 0) { - _error->Warning(_("No priority (or zero) specified for pin")); - continue; + return _error->Error(_("No priority (or zero) specified for pin")); } istringstream s(Name); diff --git a/apt-pkg/tagfile.cc b/apt-pkg/tagfile.cc index 253b1b7a3..8acecd735 100644 --- a/apt-pkg/tagfile.cc +++ b/apt-pkg/tagfile.cc @@ -533,9 +533,16 @@ signed int pkgTagSection::FindI(const char *Tag,signed long Default) const return Default; strncpy(S,Start,Stop-Start); S[Stop - Start] = 0; - + + errno = 0; char *End; signed long Result = strtol(S,&End,10); + if (errno == ERANGE) + _error->Errno("strtol", _("Cannot convert %s to integer"), S); + if (Result < std::numeric_limits::min() || Result > std::numeric_limits::max()) { + errno = ERANGE; + _error->Errno("", _("Cannot convert %s to integer"), S); + } if (S == End) return Default; return Result; diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml index 16e6a7aa0..5703203b0 100644 --- a/doc/apt_preferences.5.xml +++ b/doc/apt_preferences.5.xml @@ -339,6 +339,10 @@ only if there is no installed version of the package P < 0 prevents the version from being installed + +P = 0 +has undefined behaviour, do not use it. + -- cgit v1.2.3-70-g09d2 From 85b9b32933eaeda6119d7a0224b5070ab8230c6b Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Wed, 12 Aug 2015 20:46:00 +0200 Subject: apt_preferences(5): Re-document how priorities are calculated The old text did not match either the old or the new implementation, so let's rewrite it to explain the new implementation. Closes: #554773 --- doc/apt_preferences.5.xml | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml index 5703203b0..81e0bd220 100644 --- a/doc/apt_preferences.5.xml +++ b/doc/apt_preferences.5.xml @@ -346,11 +346,14 @@ only if there is no installed version of the package -If any specific-form records match an available package version then the -first such record determines the priority of the package version. -Failing that, -if any general-form records match an available package version then the -first such record determines the priority of the package version. + +The first specific-form record matching an available package version determines +the priority of the package version. +Failing that, the priority of the package is defined as the maximum of all +priorities defined by generic-form records matching the version. +Records defined using patterns in the Pin field other than "*" are treated like +specific-form records. + For example, suppose the APT preferences file contains the three records presented earlier: -- cgit v1.2.3-70-g09d2 From df7c9fd2224c1de4a155796e292fb919f559a674 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Thu, 13 Aug 2015 21:14:00 +0200 Subject: Accept --upgradeable as synonym for --upgradable It's a tiny diff, so why not? But no need to document it. Closes: #787846 --- apt-private/private-cmndline.cc | 1 + doc/apt.8.xml | 3 ++- doc/po/apt-doc.pot | 3 ++- doc/po/de.po | 4 ++-- doc/po/es.po | 2 +- doc/po/fr.po | 2 +- doc/po/it.po | 4 ++-- doc/po/ja.po | 4 ++-- doc/po/pl.po | 2 +- doc/po/pt.po | 2 +- doc/po/pt_BR.po | 2 +- 11 files changed, 16 insertions(+), 13 deletions(-) (limited to 'doc') diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index cfdc13259..fa8416824 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -242,6 +242,7 @@ static bool addArgumentsAPT(std::vector &Args, char const * c if (CmdMatches("list")) { addArg(0,"installed","APT::Cmd::Installed",0); + addArg(0,"upgradeable","APT::Cmd::Upgradable",0); addArg(0,"upgradable","APT::Cmd::Upgradable",0); addArg(0,"manual-installed","APT::Cmd::Manual-Installed",0); addArg('v', "verbose", "APT::Cmd::List-Include-Summary", 0); diff --git a/doc/apt.8.xml b/doc/apt.8.xml index 29bf96751..e00b6417a 100644 --- a/doc/apt.8.xml +++ b/doc/apt.8.xml @@ -44,7 +44,8 @@ display a list of packages. It supports shell pattern for matching package names and the following options: , - , + , + , are supported. diff --git a/doc/po/apt-doc.pot b/doc/po/apt-doc.pot index 98c2b66b5..509b5da34 100644 --- a/doc/po/apt-doc.pot +++ b/doc/po/apt-doc.pot @@ -531,7 +531,8 @@ msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " ", , " -" are supported." +", are " +"supported." msgstr "" #. type: Content of: diff --git a/doc/po/de.po b/doc/po/de.po index 4eb458d17..bc8d8c899 100644 --- a/doc/po/de.po +++ b/doc/po/de.po @@ -662,13 +662,13 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" "list wird benutzt, um eine Paketliste anzuzeigen. Es " "unterstützt Shell-Muster zur Beschränkung auf passende Paketnamen. Die " "folgenden Optionen werden unterstützt: , " -", ." +", , ." #. type: Content of: #: apt.8.xml:54 diff --git a/doc/po/es.po b/doc/po/es.po index 590ae4aeb..5cba392d4 100644 --- a/doc/po/es.po +++ b/doc/po/es.po @@ -742,7 +742,7 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" diff --git a/doc/po/fr.po b/doc/po/fr.po index 85869a2b7..66f5e39c3 100644 --- a/doc/po/fr.po +++ b/doc/po/fr.po @@ -658,7 +658,7 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" "La commande list est utilisée pour afficher une liste de " diff --git a/doc/po/it.po b/doc/po/it.po index 28e97f14b..21fbebd93 100644 --- a/doc/po/it.po +++ b/doc/po/it.po @@ -710,13 +710,13 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" "list viene usato per visualizzare un elenco di pacchetti. " "Permette l'uso dei modelli di shell per la corrispondenza con nomi di " "pacchetto e sono gestite le seguenti opzioni: , " -", ." +", , ." #. type: Content of: #: apt.8.xml:54 diff --git a/doc/po/ja.po b/doc/po/ja.po index 3e0712125..ba72f0d02 100644 --- a/doc/po/ja.po +++ b/doc/po/ja.po @@ -704,12 +704,12 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" "パッケージ一覧を表示するには list を使います。パッケージ名" "のマッチングにシェルパターン、そしてオプション 、 を" +"option>、 , 、 を" "サポートしています。" #. type: Content of: diff --git a/doc/po/pl.po b/doc/po/pl.po index 481d85af6..8081be57b 100644 --- a/doc/po/pl.po +++ b/doc/po/pl.po @@ -700,7 +700,7 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" diff --git a/doc/po/pt.po b/doc/po/pt.po index 3aefcc2a9..702cae479 100644 --- a/doc/po/pt.po +++ b/doc/po/pt.po @@ -707,7 +707,7 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" "list é usado para mostrar uma lista de pacotes. Suporta " diff --git a/doc/po/pt_BR.po b/doc/po/pt_BR.po index 88b479e75..5f03f7dac 100644 --- a/doc/po/pt_BR.po +++ b/doc/po/pt_BR.po @@ -523,7 +523,7 @@ msgstr "" msgid "" "list is used to display a list of packages. It supports " "shell pattern for matching package names and the following options: " -", , , , , are supported." msgstr "" -- cgit v1.2.3-70-g09d2 From b381a482eab0fc7b65b63cf0512ef1f97d775e34 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 11:49:45 +0200 Subject: Replace --force-yes by various options starting with --allow This enables more fine grained control over such exceptions. --- apt-private/private-cmndline.cc | 3 + apt-private/private-download.cc | 6 +- apt-private/private-install.cc | 43 ++++++---- doc/apt-get.8.xml | 26 +++++- test/integration/test-allow | 98 ++++++++++++++++++++++ .../integration/test-apt-get-update-unauth-warning | 2 +- test/integration/test-apt-never-markauto-sections | 2 +- test/integration/test-apt-update-nofallback | 2 +- test/integration/test-apt-update-rollback | 4 +- ...bug-712116-dpkg-pre-install-pkgs-hook-multiarch | 2 +- test/integration/test-releasefile-verification | 4 +- 11 files changed, 167 insertions(+), 25 deletions(-) create mode 100755 test/integration/test-allow (limited to 'doc') diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index fa8416824..487349c8c 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -202,6 +202,9 @@ static bool addArgumentsAPTGet(std::vector &Args, char const addArg(0,"ignore-hold","APT::Ignore-Hold",0); addArg(0,"upgrade","APT::Get::upgrade",0); addArg(0,"only-upgrade","APT::Get::Only-Upgrade",0); + addArg(0,"allow-change-held-packages","APT::Get::allow-change-held-packages",CommandLine::Boolean); + addArg(0,"allow-remove-essential","APT::Get::allow-remove-essential",CommandLine::Boolean); + addArg(0,"allow-downgrades","APT::Get::allow-downgrades",CommandLine::Boolean); addArg(0,"force-yes","APT::Get::force-yes",0); addArg(0,"print-uris","APT::Get::Print-URIs",0); addArg(0,"trivial-only","APT::Get::Trivial-Only",0); diff --git a/apt-private/private-download.cc b/apt-private/private-download.cc index 099146187..18a9b1fbc 100644 --- a/apt-private/private-download.cc +++ b/apt-private/private-download.cc @@ -114,10 +114,12 @@ bool AuthPrompt(std::vector const &UntrustedList, bool const Prompt return true; } - else if (_config->FindB("APT::Get::Force-Yes",false) == true) + else if (_config->FindB("APT::Get::Force-Yes",false) == true) { + _error->Warning(_("--force-yes is deprecated, use one of the options starting with --allow instead.")); return true; + } - return _error->Error(_("There are problems and -y was used without --force-yes")); + return _error->Error(_("There were unauthenticated packages and -y was used without --allow-unauthenticated")); } /*}}}*/ bool AcquireRun(pkgAcquire &Fetcher, int const PulseInterval, bool * const Failure, bool * const TransientNetworkFailure)/*{{{*/ diff --git a/apt-private/private-install.cc b/apt-private/private-install.cc index d2b4bed51..96e33f7de 100644 --- a/apt-private/private-install.cc +++ b/apt-private/private-install.cc @@ -58,7 +58,8 @@ bool InstallPackages(CacheFile &Cache,bool ShwKept,bool Ask, bool Safety) } } - bool Fail = false; + bool Hold = false; + bool Downgrade = false; bool Essential = false; // Show all the various warning indicators @@ -66,13 +67,17 @@ bool InstallPackages(CacheFile &Cache,bool ShwKept,bool Ask, bool Safety) ShowNew(c1out,Cache); if (ShwKept == true) ShowKept(c1out,Cache); - Fail |= !ShowHold(c1out,Cache); + Hold = !ShowHold(c1out,Cache); if (_config->FindB("APT::Get::Show-Upgraded",true) == true) ShowUpgraded(c1out,Cache); - Fail |= !ShowDowngraded(c1out,Cache); + Downgrade = !ShowDowngraded(c1out,Cache); + if (_config->FindB("APT::Get::Download-Only",false) == false) Essential = !ShowEssential(c1out,Cache); - Fail |= Essential; + + // All kinds of failures + bool Fail = (Essential || Downgrade || Hold); + Stats(c1out,Cache); // Sanity check @@ -89,7 +94,25 @@ bool InstallPackages(CacheFile &Cache,bool ShwKept,bool Ask, bool Safety) // No remove flag if (Cache->DelCount() != 0 && _config->FindB("APT::Get::Remove",true) == false) return _error->Error(_("Packages need to be removed but remove is disabled.")); - + + // Fail safe check + if (_config->FindI("quiet",0) >= 2 || + _config->FindB("APT::Get::Assume-Yes",false) == true) + { + if (_config->FindB("APT::Get::Force-Yes",false) == true) { + _error->Warning(_("--force-yes is deprecated, use one of the options starting with --allow instead.")); + } + + if (Fail == true && _config->FindB("APT::Get::Force-Yes",false) == false) { + if (Essential == true && _config->FindB("APT::Get::allow-remove-essential", false) == false) + return _error->Error(_("Essential packages were removed and -y was used without --allow-remove-essential.")); + if (Downgrade == true && _config->FindB("APT::Get::allow-downgrades", false) == false) + return _error->Error(_("Packages were downgraded and -y was used without --allow-downgrades.")); + if (Hold == true && _config->FindB("APT::Get::allow-change-held-packages", false) == false) + return _error->Error(_("Held packages were changed and -y was used without --allow-change-held-packages.")); + } + } + // Run the simulator .. if (_config->FindB("APT::Get::Simulate") == true) { @@ -173,15 +196,7 @@ bool InstallPackages(CacheFile &Cache,bool ShwKept,bool Ask, bool Safety) if (CheckFreeSpaceBeforeDownload(_config->FindDir("Dir::Cache::Archives"), (FetchBytes - FetchPBytes)) == false) return false; - // Fail safe check - if (_config->FindI("quiet",0) >= 2 || - _config->FindB("APT::Get::Assume-Yes",false) == true) - { - if (Fail == true && _config->FindB("APT::Get::Force-Yes",false) == false) - return _error->Error(_("There are problems and -y was used without --force-yes")); - } - - if (Essential == true && Safety == true) + if (Essential == true && Safety == true && _config->FindB("APT::Get::allow-remove-essential", false) == false) { if (_config->FindB("APT::Get::Trivial-Only",false) == true) return _error->Error(_("Trivial Only specified but this is not a trivial operation.")); diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml index 81a9036c4..76a53aec2 100644 --- a/doc/apt-get.8.xml +++ b/doc/apt-get.8.xml @@ -429,12 +429,36 @@ Configuration Item: APT::Get::Only-Upgrade. + + This is a dangerous option that will cause apt to continue + without prompting if it is doing downgrades. It + should not be used except in very special situations. Using + it can potentially destroy your system! + Configuration Item: APT::Get::allow-downgrades. Introduced in APT 1.1. + + + + Force yes; this is a dangerous option that will cause apt to continue + without prompting if it is removing essentials. It + should not be used except in very special situations. Using + it can potentially destroy your system! + Configuration Item: APT::Get::allow-remove-essential. Introduced in APT 1.1. + + + + Force yes; this is a dangerous option that will cause apt to continue + without prompting if it is changing held packages. It + should not be used except in very special situations. Using + it can potentially destroy your system! + Configuration Item: APT::Get::allow-change-held-packages. Introduced in APT 1.1. + + Force yes; this is a dangerous option that will cause apt to continue without prompting if it is doing something potentially harmful. It should not be used except in very special situations. Using force-yes can potentially destroy your system! - Configuration Item: APT::Get::force-yes. + Configuration Item: APT::Get::force-yes. This is deprecated and replaced by , , in 1.1. diff --git a/test/integration/test-allow b/test/integration/test-allow new file mode 100755 index 000000000..3d773ee95 --- /dev/null +++ b/test/integration/test-allow @@ -0,0 +1,98 @@ +#!/bin/sh +# +# Test for --allow-remove-essential and friends replacing --force-yes +# +set -e + +TESTDIR=$(readlink -f $(dirname $0)) +. $TESTDIR/framework +setupenvironment +configarchitecture 'amd64' + +insertpackage 'unstable' 'downgrade' 'all' '1' +insertinstalledpackage 'downgrade' 'all' '2' + +insertpackage 'unstable' 'hold' 'all' '2' +insertinstalledpackage 'hold' 'all' '1' + +insertinstalledpackage 'essential' 'all' '1' 'Essential: yes' + +setupaptarchive + +testsuccess aptmark hold hold + +# Test --allow-remove--essential + +testfailureequal 'Reading package lists... +Building dependency tree... +The following packages will be REMOVED: + essential +WARNING: The following essential packages will be removed. +This should NOT be done unless you know exactly what you are doing! + essential +0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded. +E: Essential packages were removed and -y was used without --allow-remove-essential.' aptget remove essential -y -s + +testsuccessequal 'Reading package lists... +Building dependency tree... +The following packages will be REMOVED: + essential +WARNING: The following essential packages will be removed. +This should NOT be done unless you know exactly what you are doing! + essential +0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded. +Remv essential [1]' aptget remove essential -y --allow-remove-essential -s + +# Test --allow-change-held-packages (should not influence dist-upgrade, but an install) + +testsuccessequal 'Reading package lists... +Building dependency tree... +Calculating upgrade... +The following packages have been kept back: + hold +0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.' aptget dist-upgrade --allow-change-held-packages -s + +testfailureequal 'Reading package lists... +Building dependency tree... +The following held packages will be changed: + hold +The following packages will be upgraded: + hold +1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. +E: Held packages were changed and -y was used without --allow-change-held-packages.' aptget install hold -y -s + +testfailureequal 'Reading package lists... +Building dependency tree... +The following held packages will be changed: + hold +The following packages will be upgraded: + hold +1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. +E: Held packages were changed and -y was used without --allow-change-held-packages.' aptget install hold -y -s + +testsuccessequal 'Reading package lists... +Building dependency tree... +The following held packages will be changed: + hold +The following packages will be upgraded: + hold +1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. +Inst hold [1] (2 unstable [all]) +Conf hold (2 unstable [all])' aptget install hold -y -s --allow-change-held-packages + +# Test --allow-downgrades + +testfailureequal 'Reading package lists... +Building dependency tree... +The following packages will be DOWNGRADED: + downgrade +0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 1 not upgraded. +E: Packages were downgraded and -y was used without --allow-downgrades.' aptget install downgrade=1 -y -s + +testsuccessequal 'Reading package lists... +Building dependency tree... +The following packages will be DOWNGRADED: + downgrade +0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 1 not upgraded. +Inst downgrade [2] (1 unstable [all]) +Conf downgrade (1 unstable [all])' aptget install downgrade=1 --allow-downgrades -y -s diff --git a/test/integration/test-apt-get-update-unauth-warning b/test/integration/test-apt-get-update-unauth-warning index bc8e6d3ea..4c45f8f26 100755 --- a/test/integration/test-apt-get-update-unauth-warning +++ b/test/integration/test-apt-get-update-unauth-warning @@ -81,4 +81,4 @@ W: The repository 'file:$APTARCHIVE unstable Release' does not have a Release fi # ensure we can not install the package testfailureequal "WARNING: The following packages cannot be authenticated! foo -E: There are problems and -y was used without --force-yes" aptget install -qq -y foo +E: There were unauthenticated packages and -y was used without --allow-unauthenticated" aptget install -qq -y foo diff --git a/test/integration/test-apt-never-markauto-sections b/test/integration/test-apt-never-markauto-sections index a469b4c15..9f490a1bd 100755 --- a/test/integration/test-apt-never-markauto-sections +++ b/test/integration/test-apt-never-markauto-sections @@ -65,7 +65,7 @@ testmarkedauto # test that installed/upgraded auto-pkgs are not set to manual -testsuccess aptget install browser=41 -y --force-yes +testsuccess aptget install browser=41 -y --allow-downgrades testmarkedmanual 'browser' 'dpkg' 'foreignpkg:i386' 'nosection' testmarkedauto diff --git a/test/integration/test-apt-update-nofallback b/test/integration/test-apt-update-nofallback index 2f4ddc016..6e9db2cae 100755 --- a/test/integration/test-apt-update-nofallback +++ b/test/integration/test-apt-update-nofallback @@ -101,7 +101,7 @@ test_from_inrelease_to_unsigned_with_override() # but that the individual packages are still considered untrusted testfailureequal "WARNING: The following packages cannot be authenticated! evil -E: There are problems and -y was used without --force-yes" aptget install -qq -y evil +E: There were unauthenticated packages and -y was used without --allow-unauthenticated" aptget install -qq -y evil } test_cve_2012_0214() diff --git a/test/integration/test-apt-update-rollback b/test/integration/test-apt-update-rollback index 646484e7b..503b81985 100755 --- a/test/integration/test-apt-update-rollback +++ b/test/integration/test-apt-update-rollback @@ -120,7 +120,7 @@ test_unauthenticated_to_invalid_inrelease() { listcurrentlistsdirectory > lists.before testfailureequal "WARNING: The following packages cannot be authenticated! old -E: There are problems and -y was used without --force-yes" aptget install -qq -y old +E: There were unauthenticated packages and -y was used without --allow-unauthenticated" aptget install -qq -y old # go to authenticated but not correct add_new_package '+1hour' @@ -133,7 +133,7 @@ E: Some index files failed to download. They have been ignored, or old ones used testfailure ls rootdir/var/lib/apt/lists/*_InRelease testfailureequal "WARNING: The following packages cannot be authenticated! old -E: There are problems and -y was used without --force-yes" aptget install -qq -y old +E: There were unauthenticated packages and -y was used without --allow-unauthenticated" aptget install -qq -y old } test_inrelease_to_unauth_inrelease() { diff --git a/test/integration/test-bug-712116-dpkg-pre-install-pkgs-hook-multiarch b/test/integration/test-bug-712116-dpkg-pre-install-pkgs-hook-multiarch index 62355a6b5..93a33d30f 100755 --- a/test/integration/test-bug-712116-dpkg-pre-install-pkgs-hook-multiarch +++ b/test/integration/test-bug-712116-dpkg-pre-install-pkgs-hook-multiarch @@ -48,7 +48,7 @@ DPkg::Tools::options::\"./${hook}-v${1}.sh\"::Version \"$1\";" > rootdir/etc/apt observehook() { rm -f ${hook}-v2.list ${hook}-v3.list msgtest 'Observe hooks while' "$*" - testsuccess --nomsg aptget "$@" -y --force-yes + testsuccess --nomsg aptget "$@" -y --allow-downgrades } testrun() { diff --git a/test/integration/test-releasefile-verification b/test/integration/test-releasefile-verification index 06701c623..c4d1455eb 100755 --- a/test/integration/test-releasefile-verification +++ b/test/integration/test-releasefile-verification @@ -69,7 +69,7 @@ The following NEW packages will be installed: After this operation, 5370 kB of additional disk space will be used. WARNING: The following packages cannot be authenticated! apt -E: There are problems and -y was used without --force-yes' aptget install apt -dy +E: There were unauthenticated packages and -y was used without --allow-unauthenticated' aptget install apt -dy } failaptnew() { @@ -83,7 +83,7 @@ The following NEW packages will be installed: After this operation, 5808 kB of additional disk space will be used. WARNING: The following packages cannot be authenticated! apt -E: There are problems and -y was used without --force-yes' aptget install apt -dy +E: There were unauthenticated packages and -y was used without --allow-unauthenticated' aptget install apt -dy } # fake our downloadable file -- cgit v1.2.3-70-g09d2 From 2e38192cec76b3b98fe8aeb5a9a1e0fa3573d3a4 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 14:42:08 +0200 Subject: Mention that source order only matter per version Closes: #617445 --- doc/sources.list.5.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml index 3bc8a94ac..e27eddb0e 100644 --- a/doc/sources.list.5.xml +++ b/doc/sources.list.5.xml @@ -36,7 +36,7 @@ designed to support any number of active sources and a variety of source media. The files list one source per line (one line style) or contain multiline stanzas defining one or more sources per stanza (deb822 style), with the - most preferred source listed first. The information available from the + most preferred source listed first (in case a single version is available from more than one source). The information available from the configured sources is acquired by apt-get update (or by an equivalent command from another APT front-end). -- cgit v1.2.3-70-g09d2 From 5fba9462b635c95d2cb329a388d469bd50889974 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 16:57:02 +0200 Subject: apt-cache(8): Mention that --names-only search provides Closes: #618017 --- doc/apt-cache.8.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/apt-cache.8.xml b/doc/apt-cache.8.xml index e20f66770..ac3aaa37a 100644 --- a/doc/apt-cache.8.xml +++ b/doc/apt-cache.8.xml @@ -172,7 +172,7 @@ Reverse Provides: If is given then output identical to show is produced for each matched package, and if is given then the long description - is not searched, only the package name is. + is not searched, only the package name and provided packages are. Separate arguments can be used to specify multiple search patterns that are and'ed together. @@ -319,7 +319,7 @@ Reverse Provides: - Only search on the package names, not the long descriptions. + Only search on the package and provided package names, not the long descriptions. Configuration Item: APT::Cache::NamesOnly. -- cgit v1.2.3-70-g09d2 From 1bdfd2c9de1126522fee08007908a05af32ee8b1 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 17:22:30 +0200 Subject: apt_preferences(5): Correct default pin assignment documentation This was broken, as higher pins were also assigned to versions that are installed. Closes: #623706 --- doc/apt_preferences.5.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml index 28b795d43..cad57f6b8 100644 --- a/doc/apt_preferences.5.xml +++ b/doc/apt_preferences.5.xml @@ -115,12 +115,12 @@ from archives which in their Release files are marked as "N priority 500 -to the versions that are not installed and do not belong to the target release. +to the versions that do not belong to the target release. priority 990 -to the versions that are not installed and belong to the target release. +to the versions that belong to the target release. -- cgit v1.2.3-70-g09d2 From c38847ebbbab115b01dd1421dee55d5a50d404f1 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 17:26:48 +0200 Subject: apt_preferences(5): Mention overlapping of pin matches --- doc/apt_preferences.5.xml | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc') diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml index cad57f6b8..be6f12f6a 100644 --- a/doc/apt_preferences.5.xml +++ b/doc/apt_preferences.5.xml @@ -123,6 +123,9 @@ from archives which in their Release files are marked as "N to the versions that belong to the target release. + +The highest of those priorities whose description matches the version is assigned to the +version. If the target release has not been specified then APT simply assigns -- cgit v1.2.3-70-g09d2 From f66738d7fb8978eaa30a179ae4f3bcc4ca7aa58f Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 18:27:24 +0200 Subject: Make auto-remove and auto-clean aliases for the versions without - Some people type them instead of autoremove and autoclean, so make them happy. Closes: #274159 Makes-Happy: Ansgar --- apt-private/private-cmndline.cc | 4 ++-- apt-private/private-install.cc | 3 ++- cmdline/apt-get.cc | 2 ++ doc/apt-get.8.xml | 4 ++-- 4 files changed, 8 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc index 487349c8c..3a1564b23 100644 --- a/apt-private/private-cmndline.cc +++ b/apt-private/private-cmndline.cc @@ -169,14 +169,14 @@ static bool addArgumentsAPTGet(std::vector &Args, char const addArg(0,"format","APT::Get::IndexTargets::Format", CommandLine::HasArg); addArg(0,"release-info","APT::Get::IndexTargets::ReleaseInfo", 0); } - else if (CmdMatches("clean", "autoclean", "check", "download", "changelog") || + else if (CmdMatches("clean", "autoclean", "auto-clean", "check", "download", "changelog") || CmdMatches("markauto", "unmarkauto")) // deprecated commands ; else if (CmdMatches("moo")) addArg(0, "color", "APT::Moo::Color", 0); if (CmdMatches("install", "remove", "purge", "upgrade", "dist-upgrade", - "dselect-upgrade", "autoremove", "clean", "autoclean", "check", + "dselect-upgrade", "autoremove", "auto-remove", "clean", "autoclean", "auto-clean", "check", "build-dep", "full-upgrade", "source")) { addArg('s', "simulate", "APT::Get::Simulate", 0); diff --git a/apt-private/private-install.cc b/apt-private/private-install.cc index 96e33f7de..e61c4ca51 100644 --- a/apt-private/private-install.cc +++ b/apt-private/private-install.cc @@ -519,7 +519,8 @@ bool DoCacheManipulationFromCommandLine(CommandLine &CmdL, CacheFile &Cache, _config->Set("APT::Get::Purge", true); fallback = MOD_REMOVE; } - else if (strcasecmp(CmdL.FileList[0], "autoremove") == 0) + else if (strcasecmp(CmdL.FileList[0], "autoremove") == 0 || + strcasecmp(CmdL.FileList[0], "auto-remove") == 0) { _config->Set("APT::Get::AutomaticRemove", "true"); fallback = MOD_REMOVE; diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc index 61ed41164..80e344740 100644 --- a/cmdline/apt-get.cc +++ b/cmdline/apt-get.cc @@ -1643,6 +1643,7 @@ int main(int argc,const char *argv[]) /*{{{*/ {"remove",&DoInstall}, {"purge",&DoInstall}, {"autoremove",&DoInstall}, + {"auto-remove",&DoInstall}, {"markauto",&DoMarkAuto}, {"unmarkauto",&DoMarkAuto}, {"dist-upgrade",&DoDistUpgrade}, @@ -1650,6 +1651,7 @@ int main(int argc,const char *argv[]) /*{{{*/ {"build-dep",&DoBuildDep}, {"clean",&DoClean}, {"autoclean",&DoAutoClean}, + {"auto-clean",&DoAutoClean}, {"check",&DoCheck}, {"source",&DoSource}, {"download",&DoDownload}, diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml index 76a53aec2..785b4e9a8 100644 --- a/doc/apt-get.8.xml +++ b/doc/apt-get.8.xml @@ -214,7 +214,7 @@ &cachedir;/archives/partial/. - + (and the alias since 1.1) Like clean, autoclean clears out the local repository of retrieved package files. The difference is that it only removes package files that can no longer be downloaded, and are largely @@ -224,7 +224,7 @@ erased if it is set to off. - + (and the alias since 1.1) autoremove is used to remove packages that were automatically installed to satisfy dependencies for other packages and are now no longer needed. -- cgit v1.2.3-70-g09d2 From 825220b5aef6db9dfb8a34c41a3ac7c3ce477be2 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 20:35:18 +0200 Subject: apt: Add autoremove and auto-remove commands --- cmdline/apt.cc | 3 +++ doc/apt.8.xml | 5 +++++ 2 files changed, 8 insertions(+) (limited to 'doc') diff --git a/cmdline/apt.cc b/cmdline/apt.cc index 2f7eddb61..92db34cfa 100644 --- a/cmdline/apt.cc +++ b/cmdline/apt.cc @@ -55,6 +55,7 @@ static bool ShowHelp(CommandLine &) "\n" " install - install packages\n" " remove - remove packages\n" + " autoremove - Remove automatically all unused packages\n" "\n" " upgrade - upgrade the system by installing/upgrading packages\n" " full-upgrade - upgrade the system by removing/installing/upgrading packages\n" @@ -76,6 +77,8 @@ int main(int argc, const char *argv[]) /*{{{*/ // package stuff {"install",&DoInstall}, {"remove", &DoInstall}, + {"autoremove", &DoInstall}, + {"auto-remove", &DoInstall}, {"purge", &DoInstall}, // system wide stuff diff --git a/doc/apt.8.xml b/doc/apt.8.xml index e00b6417a..18b97f547 100644 --- a/doc/apt.8.xml +++ b/doc/apt.8.xml @@ -86,6 +86,11 @@ installed instead of removed. + (and the alias since 1.1) + autoremove is used to remove packages that were automatically + installed to satisfy dependencies for other packages and are now no longer needed. + + edit-sources lets you edit your sources.list file and provides basic sanity checks. -- cgit v1.2.3-70-g09d2 From dd5978bff4d0bea85b3adbf52a194afd25a655b3 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 22:43:00 +0200 Subject: apt-cache(8): Drop the #versions >= #package names comparison Closes: #691281 --- doc/apt-cache.8.xml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/apt-cache.8.xml b/doc/apt-cache.8.xml index ac3aaa37a..a8f1b4586 100644 --- a/doc/apt-cache.8.xml +++ b/doc/apt-cache.8.xml @@ -122,8 +122,7 @@ Reverse Provides: Total distinct versions is the number of package versions - found in the cache; this value is therefore at least equal to the - number of total package names. If more than one distribution is being accessed + found in the cache. If more than one distribution is being accessed (for instance, "stable" and "unstable"), this value can be considerably larger than the number of total package names. -- cgit v1.2.3-70-g09d2 From 7f97aa748270231d63b6419a271ba74029eebcd3 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Fri, 14 Aug 2015 23:57:13 +0200 Subject: doc/files.dbk: Improve documentation for {src,}pkgcache.bin This should be enough for most persons. Closes: #465551 --- doc/files.dbk | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/files.dbk b/doc/files.dbk index f513e0008..714255a53 100644 --- a/doc/files.dbk +++ b/doc/files.dbk @@ -198,11 +198,11 @@ installation
Binary Package Cache (srcpkgcache.bin and pkgcache.bin) Please see cache.sgml for a complete description of what this file -is. The cache file is updated whenever the contents of the lists -directory changes. If the cache is erased, corrupted or of a non-matching +is. The cache file is updated whenever the Packages or Release files of the lists +directory or the dpkg status file changes. If the cache is erased, corrupted or of a non-matching version it will be automatically rebuilt by all of the tools that need it. srcpkgcache.bin contains a cache of all of the -package files in the source list. This allows regeneration of the cache +package, release files in the source list. In comparison to pkgcache.bin, it does not include the /var/lib/dpkg/status file. This allows regeneration of the cache when the status files change to use a prebuilt version for greater speed.
-- cgit v1.2.3-70-g09d2 From 35ea8b154b105727cf0a0f0238d51b82a0d7c045 Mon Sep 17 00:00:00 2001 From: Julian Andres Klode Date: Sat, 15 Aug 2015 15:33:40 +0200 Subject: Document the general effect of the comma operator Closes: #574939 --- doc/apt_preferences.5.xml | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'doc') diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml index be6f12f6a..79132e007 100644 --- a/doc/apt_preferences.5.xml +++ b/doc/apt_preferences.5.xml @@ -254,6 +254,11 @@ Pin-Priority: 500 + +The effect of the comma operator is similar to an "and" in logic: All +conditions must be satisfied for the pin to match. There is one exception: +For any type of condition (such as two "a" conditions), only the last such +condition is checked. -- cgit v1.2.3-70-g09d2