summaryrefslogtreecommitdiff
path: root/apt-pkg/edsp.cc
Commit message (Collapse)AuthorAgeFilesLines
* Introduce JSONL performance counter loggingJulian Andres Klode2026-02-101-0/+2
| | | | | | | | Introduce a scoped object that starts measuring performance counters and then dumps them into a JSONL file for later analysis. Add performance contexts for APT::Solver and pkgDepCache::Init() as starting points.
* solver3: Extract DependencySolverJulian Andres Klode2026-01-051-1/+1
| | | | Extract almost all dependency logic into a subclass
* solver3: Refactor to use a namespaceJulian Andres Klode2026-01-051-1/+1
| | | | | | | This removes the need for the forward references, thus fixing part of the libc++ issues pointed out in [merge-511]. [merge-511] https://salsa.debian.org/apt-team/apt/-/merge_requests/511/diffs
* edsp: Do not skip lines (fixes Architecture reading)Julian Andres Klode2025-08-261-31/+31
| | | | | | | | | | | | | | | We skipped the Architecture: line because when the outer loop got to it, the inner loop started and read the next line after it. Get rid of the nested loop instead, and first run a loop to find the request line, and then a second loop to parse the request flags. An alternative would be to use a combined loop that supports non-compliant protocol implementations, but that's harder to read and we don't expect there to be any especially on the request side.
* edsp: Clear architecture cacheJulian Andres Klode2025-08-261-0/+4
| | | | We need to actually use the architecture values we set
* edsp: Dump minimized Dir::Log::Solver logsJulian Andres Klode2025-03-201-1/+1
|
* edsp: Add functionality to only write relevant packagesJulian Andres Klode2025-03-201-0/+30
| | | | | | | | | | | | | Add a variant of WriteLimitedScenario that does not take a pkgset, but creates one on the fly based on the transitive closure of installed packages, to be installed packages, and their dependencies and provides. This includes more packages than strictly necessary, i.e. we do not really need to include foo just because a package we want to install Conflicts with it, but oh well, this is easy to read. Trying to extend the filtering is hard as the dependencies being written are also filtered...
* edsp: Set Forbid-New-Install/Forbid-Remove: no if other is setJulian Andres Klode2025-03-101-0/+4
| | | | | | | | The upgrade code may also set Upgrade: yes, and this sets both to yes, so `apt upgrade` dumps behave like `apt-get upgrade` ones, which of course is not intentional. moo
* edsp: Write 'Size' to the EDSP filesJulian Andres Klode2025-03-031-0/+2
| | | | | | This allows solvers to optimize by download size; and gives us a better means of interpreting "downloadable" debs (they must have a size > 0 after all).
* edsp: Use buffered writes for EDSP dumpsJulian Andres Klode2025-02-171-1/+1
| | | | This makes the dump almost instant rather than take forever.
* solver3: Pass EDSP flags directly rather than via configJulian Andres Klode2025-02-141-2/+2
| | | | | | | This was a rather silly way to communicate state, and it was in the wrong place. Notably also, multiple calls to the solver had the options sticky, that is, if you run upgrade and then it calls ResolveByKeep(), for example.
* solver3: Defer 3.0 'deep' autoremoval to 3.1, fix autoremoveJulian Andres Klode2025-02-141-1/+1
| | | | | | Restore the depcache's MarkRequired logic for 3.0 solver; and change the MarkInstall() call to pass a more correct value for FromUser, to not override an existing automatic status.
* Add pkgCache::{Priority,DepType}_NoL10n to avoid duplicationDavid Kalnischkies2025-01-051-17/+6
| | | | | | We don't have many places, but lets reduce the amount of duplicating these short strings, so that we may find all the places we have to change if that ever happens.
* apt-pkg/edsp.cc: APT::StringView -> std::string_viewнаб2024-11-121-9/+8
|
* edsp: solver3: Show some progressJulian Andres Klode2024-05-241-7/+16
| | | | | Ignore the Solving dependencies... line in the testing framework for compatibility with the existing test cases.
* Initial implementation of the 3.0 solverJulian Andres Klode2024-05-141-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a simple backtracking brute-force solver with heurisitcs, this initial version has the following known gaps: - Errors are not kept from branches, the error reporting after backtracking isn't particularly useful. - We cannot show automatically removed packages - We cannot replace packages with others - We do not have conflict-driven clause learning yet Untested: - Multi-arch This solver is fundamentally different in key aspects: - It solves smaller dependency groups before larger ones, leading us to avoid installing A in A|B if B is installed more often and more consistently. - It only keeps the automatic packages reachable via the strongest path. Currently it only implements autoremoval, but not display of autoremoval as we simply enqueue all automatically installed packages at the end when not doing automatic removal. This will need some translation where we Solve() first, and then Solve() again with the automatically installed packages added such that we can mark them as Garbage for display purposes. - It does not remove manually installed packages. Hook the solver in via the EDSP framework, this allows us to achieve easy initial integration without lots of issues. A lot of this work was planned and executed in my free time and then some leaked into work time I suppose. Implementation notes: - Restore the full backlog of items The annoying thing is that we record only when an item was enqueued and not the level at which it was installed, so when going back a decision level we might have to reinstall packages that were queued at an earlier decision level because they were only installed at a later decision level. - When picking one version, reject the others - Propagate conflicts up to reverse dependencies This will recursively mark every reverse dependency that can no longer be satisfied as MUSTNOT. Also make sure to recursively call Reject(Ver) from Reject(Pkg) to make sure we trigger the Rejections there. This means we now end up having Recursion in the algorithm. An alternative approach would be to push *reject* items to the heap and then do them, but this is not entirely straight forward and it may simply not be necessary. - Sort upgrades before other optional installs containing subsets If I want to upgrade a package A, I schedule A3|A2|A1; if another thing depends specifically on A1; we'd not be installed. Hence we need to sort upgrades first. This only is needed for optional packages; manual packages will figure this out naturally. - Rescoring is lazily implemented. Instead of calling make_heap() after rescoring items, we just mark the items as dirty and reinsert them. We also only rescore from the main solve loop, Reject() marks the heap as needing a rescore due to a Conflict (as some versions will no longer be installable), and RescoreWorkIfNeeded() then will do the rescoring. - Recursive unit propagation: Install() and Reject() recursively call each other to promote decisions across single-version dependencies (or across not-anymore satisfiable reverse-depends). - Make Reason constructors explicit, this enhances readability This makes calls like the one in here be Reject(object, Reason(otherObject)) Ensuring that it's clear that the 2nd argument is a reason at the caller side. - Split Decision into Decision and Hint vs. first draft When branching/deciding, we do not want to override SHOULD and MAY. We do not actually use them yet, and we do actually clean them when backtracking, but let's at least keep the data structure correct. Convert the enum to a 16-bit integer so we can still fit in the same space as before.
* Change the default log level from debug to noticeJulian Andres Klode2024-04-191-2/+2
| | | | | | | | | | We never used the debug level before, so we can do that. This allows us to have the new audit level. We did call DumpErrors() with DEBUG in two debug code paths, so don't touch those. debug
* Modernize standard library includesJulian Andres Klode2024-02-201-4/+4
| | | | | | This was automated with sed and git-clang-format, and then I had to fix up the top of policy.cc by hand as git-clang-format accidentally indented it by two spaces.
* edsp: Add support for phased updates (test in LP#1990586)Julian Andres Klode2022-09-281-0/+7
| | | | | | | | Add Machine-ID to the first stanza, and copy Phased-Update-Percentage to package stanzas. This will be tested at a later state by the EDSP test case for bug 1990586.
* Avoid .c_str() on strings feed into pkgTagSection::FindSDavid Kalnischkies2022-04-011-1/+1
| | | | | | FindS has a APT::StringView based API nowadays, so we can avoid these explicit calls also allowing us to avoid the std::string in input or output entirely or at least move it a few branches down.
* Streamline access to barbarian architecture functionalityDavid Kalnischkies2021-09-041-1/+6
| | | | | | | | | | | | | | | | | | | | | | APT is not the place this information should be stored at, but it is a good place to experiment and see what will be (not) needed in the future for a proper implementation higher up the stack. This is why "BarbarianArchitectures" is chosen instead of a more neutral and/or sensible "VeryForeign" and isn't readily exported in the API to other clients for this PoC as a to be drawn up standard will likely require potentially incompatible changes. Having a then outdated and slightly different implementation block a "good" name would be bad. The functionality itself mostly exists (ignoring bugs) since the introduction of MultiArch as we always had the risk of encountering packages of architectures not known to dpkg (forced onto the system, potentially before MultiArch) we had to deal with somehow and other edge cases. All this commit really does is allowing what could previously only be achieved with editing sources.list and some conf options via a single config option: -o APT::BarbarianArchitectures=foo,bar
* Refactor architecture handling in EDSP/EIPP codeDavid Kalnischkies2021-09-041-20/+18
| | | | | | This just moves code around without actually changing anything. Gbp-Dch: Ignore
* Deduplicate EDSP Provides line of M-A:foreign packagesDavid Kalnischkies2020-06-141-9/+17
| | | | | | | | | | | | M-A:foreign causes Provides to apply to all architectures and as we wanted to avoid resolver changes for M-A those are done by explicitly creating these provides instead of forcing the resolvers to learn about this. The EDSP is a different beast though & we don't need this trick here especially as it leads to needless (but harmless) duplication. No sort+unique is done to avoid changing order (not that it should matter, but just to be sure), but the sets should be small enough to not make a huge difference either way.
* Tell EDSP solvers about all installed pkgs ignoring archDavid Kalnischkies2020-06-141-1/+1
| | | | | | | | We usually tell EDSP solvers only about architectures we are configured to treat as native/foreign, but the system could have packages from other architectures installed (even if very unlikely) which could influence the solution (e.g. requiring a removal) so we make sure to tell them.
* Ensure EDSP doesn't use a dangling architecture stringDavid Kalnischkies2020-05-251-4/+2
| | | | | | | | | | | ../apt-pkg/edsp.cc:861:23: error: object backing the pointer will be destroyed at the end of the full-expression [-Wdangling-gsl] const char *arch = _config->Find("APT::Architecture").c_str(); Compilers are probably optimizing it the way the patch does by hand now. Small string optimisation helps likely as well. Othwise that should have failed left and right as EDSP is used by experimental and such builders to talk to aspcud. Reported-By: clang
* Rename _count() macro to APT_ARRAY_SIZE()Julian Andres Klode2020-01-071-2/+2
|
* edsp: Remove deprecated functionsJulian Andres Klode2019-02-261-328/+2
|
* Reformat and sort all includes with clang-formatJulian Andres Klode2017-07-121-9/+9
| | | | | | | | | | | | | This makes it easier to see which headers includes what. The changes were done by running git grep -l '#\s*include' \ | grep -E '.(cc|h)$' \ | xargs sed -i -E 's/(^\s*)#(\s*)include/\1#\2 include/' To modify all include lines by adding a space, and then running ./git-clang-format.sh.
* Drop cacheiterators.h includeJulian Andres Klode2017-07-121-1/+0
| | | | | Including cacheiterators.h before pkgcache.h fails because pkgcache.h depends on cacheiterators.h.
* avoid explicit types for pkg counts by autoDavid Kalnischkies2017-06-261-16/+16
| | | | | | | | | Changes nothing on the program front and as the datatypes are sufficently comparable fixes no bug either, but problems later on if we ever change the types of those and prevent us using types which are too large for the values we want to store waste (a tiny bit of) resources. Gbp-Dch: Ignore
* ensure generation of valid EDSP error stanzasDavid Kalnischkies2016-12-311-2/+6
| | | | | | | | | The crude way of preparing a message to be a multiline value failed at generation valid deb822 in case the error message ended with a new line like the resolving errors from apt do. apt itself can parse these, but other tools like grep-dctrl choke on it, so be nice and print valid. Reported-By: Johannes 'josch' Schauer on IRC
* edsp: try 2 to read responses even if writing failedDavid Kalnischkies2016-09-071-15/+27
| | | | | | | | | | | | | | | | Commit b60c8a89c281f2bb945d426d2215cbf8f5760738 improved the situation, but due to inconsistency mostly for planners, not for solvers. As the idea of hiding errors if we show another error is a bit scary (as the extern error might be a followup of our intern error, rather than the reason for our intern error as it is at the moment) we don't discard the errors, but if we got an extern error we show them directly removing them from the error list at the end of the run – that list will contain the extern error which hopefully gives us the best of both worlds. The problem itself is the same as before: The externals exiting before apt is done talking to them. Reported-By: Johannes 'josch' Schauer on IRC
* edsp: try to read responses even if writing failedDavid Kalnischkies2016-07-291-15/+20
| | | | | | | If a solver/planner exits before apt is done writing we will generate write errors. Solvers like 'dump' can be pretty quick in failing but produce a valid EDSP error report apt should read, parse and display instead of just discarding even through we had write errors.
* (error) va_list 'args' was opened but not closed by va_end()David Kalnischkies2016-07-271-1/+1
| | | | | Reported-By: cppcheck Gbp-Dch: Ignore
* eipp: avoid producing file warnings in simulationDavid Kalnischkies2016-07-271-37/+33
| | | | | | | | | | | | | | | Simulations are frequently run by unprivileged users which naturally don't have the permissions to write to the default location for the eipp file. Either way is bad as running in simulation mode doesn't mean we don't want to run the logging (as EIPP runs the same regardless of simulation or 'real' run), but showing the warnings is relatively pointless in the default setup, so, in case we would produce errors and perform a simulation we will discard the warnings and carry on. Running apt with an external planner wouldn't have generated these messages btw. Closes: 832614
* EIPP/EDSP log can't be written is a warning, not an errorDavid Kalnischkies2016-07-051-4/+28
| | | | | If other logs can't be written this is a warning to, so for consistency sake translate the errors to warnings.
* report write errors in EDSP/EIPP properly back to callerDavid Kalnischkies2016-07-051-6/+3
| | | | | Unlikely to happen in practice and I wonder more how I could miss these in earlier reviews, but okay, lets fix it for consistency now.
* use +0000 instead of UTC by default as timezone in outputDavid Kalnischkies2016-07-021-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | All apt versions support numeric as well as 3-character timezones just fine and its actually hard to write code which doesn't "accidently" accepts it. So why change? Documenting the Date/Valid-Until fields in the Release file is easy to do in terms of referencing the datetime format used e.g. in the Debian changelogs (policy §4.4). This format specifies only the numeric timezones through, not the nowadays obsolete 3-character ones, so in the interest of least surprise we should use the same format even through it carries a small risk of regression in other clients (which encounter repositories created with apt-ftparchive). In case it is really regressing in practice, the hidden option -o APT::FTPArchive::Release::NumericTimezone=0 can be used to go back to good old UTC as timezone. The EDSP and EIPP protocols use this 'new' format, the text interface used to communicate with the acquire methods does not for compatibility reasons even if none of our methods would be effected and I doubt any other would (in these instances the timezone is 'GMT' as that is what HTTP/1.1 requires). Note that this is only true for apt talking to methods, (libapt-based) methods talking to apt will respond with the 'new' format. It is therefore strongly adviced to support both also in method input.
* eipp: let apt make a plan, not make stuff planeDavid Kalnischkies2016-06-291-14/+14
| | | | | | | | | | | | | | Julian noticed on IRC that I fall victim to a lovely false friend by calling referring to a 'planer' all the time even through these are machines to e.g. remove splinters from woodwork ("make stuff plane"). The term I meant is written in german in this way (= with a single n) but in english there are two, aka: 'planner'. As that is unreleased code switching all instances without any transitional provisions. Also the reason why its skipped in changelog. Thanks: Julian Andres Klode Gbp-Dch: Ignore
* eipp: enable xz-compressed scenario loggingDavid Kalnischkies2016-06-271-1/+15
| | | | | | | | | | | | In 385d9f2f23057bc5808b5e013e77ba16d1c94da4 I implemented the storage of scenario files based on enabling this by default for EIPP, but I implemented it first optionally for EDSP to have it independent. The reasons mentioned in the earlier commit (debugging and bugreports) obviously apply here, especially as EIPP solutions aren't user approved, nearly impossible to verify before starting the execution and at the time of error the scenario has changed already, so that reproducing the issue becomes hard(er).
* eipp: rename stanza 'Install' to 'Unpack'David Kalnischkies2016-06-271-3/+3
| | | | | | Freeing 'Install' for future use as an interface for "dpkg --install", which is currently not used by any existent planer, so the implementation of it itself will be delayed until then.
* eipp: add Allow-Temporary-Remove-of-EssentialsDavid Kalnischkies2016-06-271-0/+4
| | | | | | | | A rather special need option, but the internal planer supports this and we have a testcase for it & sometimes it is hit (as a bug through). The option itself mostly serves as a reminder for implementors that they should be careful with removes and especially temporary removes if they perform any.
* eipp: implement Immediate-Configuration flagDavid Kalnischkies2016-06-271-1/+11
| | | | | | | | | APT has 3 modes: no immediate configuration, all packages are configured immediately and its default mode of configuring essentials and pseudo-essentials immediately only. While this seems like a job of different planers at first, it might be handy to have it as an option, too, in case a planer (like apts internal one) supports different modes where the introduction of individual planers would be counter intuitive.
* eipp: properly handle arch-specific providesDavid Kalnischkies2016-06-271-0/+29
| | | | | The generation of the EIPP request was a bit to strict not generation what would actually be needed to be part of the scenario.
* eipp: make no difference between remove & purgeDavid Kalnischkies2016-06-271-5/+11
| | | | | | | | For the order there is no inherent difference between delete or purge, so we don't tell the planer about this and instead decide in apt if a package should be purged or not which also allows us to not tell the planer about rc-only purges as we can trivially do this on our on as there is no need to plan such purges.
* eipp: provide the internal planer as an external oneDavid Kalnischkies2016-06-271-14/+140
| | | | | | Testing the current implementation can benefit from being able to be feed an EIPP request and produce a fully compliant response. It is also a great test for EIPP in general.
* eipp: output at most two versions per packageDavid Kalnischkies2016-06-271-20/+53
| | | | | | | | | | | | | We can trim generation time and size of the EIPP scenario considerable if we we avoid telling the planers about "uninteresting" packages. This is one of the simpler but already very effective reductions: Do not tell planers about versions which are neither installed nor are to be installed as they have no effect on the plan we don't need to tell the planer about them. EDSP solvers need to know about all versions for better choice and error messages, but planers really don't. Git-Dch: Ignore
* eipp: implement version 0.1 of the protocolDavid Kalnischkies2016-06-271-0/+180
| | | | | | | | | | | | | | | | | | | | | | | The very first step in introducing the "external installation planer protocol" (short: EIPP) as part of my GSoC2016 project. The description reads: APT-based tools like apt-get, aptitude, synaptic, … work with the user to figure out how their system should look like after they are done installing/removing packages and their dependencies. The actual installation/removal of packages is done by dpkg with the constrain that dependencies must be fulfilled at any point in time (e.g. to run maintainer scripts). Historically APT has a super micro-management approach to this task which hasn't aged that well over the years mostly ignoring changes in dpkg and growing into an unmaintainable mess hardly anyone can debug and everyone fears to touch – especially as more and more requirements are tacked onto it like handling cycles and triggers, dealing with "important" packages first, package sources on removable media, touch minimal groups to be able to interrupt the process if needed (e.g. unattended-upgrades) which not only sky-rocket complexity but also can be mutually exclusive as you e.g. can't have minimal groups and minimal trigger executions at the same time.
* don't leak EDSP solver output fdDavid Kalnischkies2016-06-101-1/+1
|
* edsp: drop privileges before executing solversDavid Kalnischkies2016-06-081-1/+6
| | | | | | | | | | | | | | | Most (if not all) solvers should be able to run perfectly fine without root privileges as they get the entire state they are supposed to work on via stdin and do not perform any action directly, but just pass suggestions on via stdout. The new default is to run them all as _apt hence, but each solver can configure another user if it chooses/must. The security benefits are minimal at best, but it helps preventing silly mistakes (see 35f3ed061f10a25a3fb28bc988fddbb976344c4d) and that is always good. Note that our 'apt' and 'dump' solver already dropped privileges if they had them.
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-07-18 22:10:40 +0000