aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/remote-store.cc
AgeCommit message (Collapse)Author
2023-04-15Make `KeyedBuildResult`, `BuildResult` like before, and fix bug another wayJohn Ericson
In https://github.com/NixOS/nix/pull/6311#discussion_r834863823, I realized since derivation goals' wanted outputs can "grow" due to overlapping dependencies (See `DerivationGoal::addWantedOutputs`, called by `Worker::makeDerivationGoalCommon`), the previous bug fix had an unfortunate side effect of causing more pointless rebuilds. In paticular, we have this situation: 1. Goal made from `DerivedPath::Built { foo, {a} }`. 2. Goal gives on on substituting, starts building. 3. Goal made from `DerivedPath::Built { foo, {b} }`, in fact is just modified original goal. 4. Though the goal had gotten as far as building, so all outputs were going to be produced, `addWantedOutputs` no longer knows that and so the goal is flagged to be restarted. This might sound far-fetched with input-addressed drvs, where we usually basically have all our goals "planned out" before we start doing anything, but with CA derivation goals and especially RFC 92, where *drv resolution* means goals are created after some building is completed, it is more likely to happen. So the first thing to do was restore the clearing of `wantedOutputs` we used to do, and then filter the outputs in `buildPathsWithResults` to only get the ones we care about. But fix also has its own side effect in that the `DerivedPath` in the `BuildResult` in `DerivationGoal` cannot be trusted; it is merely the *first* `DerivedPath` for which this goal was originally created. To remedy this, I made `BuildResult` be like it was before, and instead made `KeyedBuildResult` be a subclass wit the path. Only `buildPathsWithResults` returns `KeyedBuildResult`s, everything else just becomes like it was before, where the "key" is unambiguous from context. I think separating the "primary key" field(s) from the other fields is good practical in general anyways. (I would like to do the same thing for `ValidPathInfo`.) Among other things, it allows constructions like `std::map<Key, ThingWithKey>` where doesn't contain duplicate keys and just precludes the possibility of those duplicate keys being out of sync. We might leverage the above someday to overload `buildPathsWithResults` to take a *set* of return a *map* per the above. ----- Unfortunately, we need to avoid C++20 strictness on designated initializers. (BTW https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html this offers some new syntax for this use-case. Hopefully this will be adopted and we can eventually use it.) No having that yet, maybe it would be better to not make `KeyedBuildResult` a subclass to just avoid this. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-04-14Legacy vs non-legacy `to_string`/`parse` for `DerivedPath`John Ericson
As requested by @roberth, it is good to call out the specific instances we care about, which is `!` for the RPC protocols, and `^` for humans. This doesn't take advantage of parametricity as much, but since the human and computer interfaces are good to decouple anyways (we don't care if they drift further apart over time in the slightest) some separation and slight duplication is fine. Also, unit test both round trips.
2023-04-07Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-04-06Add `Store::isTrustedClient()`matthewcroughan
This function returns true or false depending on whether the Nix client is trusted or not. Mostly relevant when speaking to a remote store with a daemon. We include this information in `nix ping store` and `nix doctor` Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2023-04-01Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2023-04-01Merge commit 'aa99005004bccc9be506a2a2f162f78bad4bcb41' into ca-drv-exoticJohn Ericson
2023-03-30Use "raw pattern" for content address typesJohn Ericson
We weren't because this ancient PR predated it! This is actually a new version of the pattern which addresses some issues identified in #7479.
2023-03-20Move enabled experimental feature to libutil structJohn Ericson
This is needed in subsequent commits to allow the settings and CLI args infrastructure itself to read this setting.
2023-02-28Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2023-02-01Don't send plugin-files to the daemon.Shea Levy
This is radically unsafe and the daemon has already loaded its plugins anyway. Fixes cachix/devenv#276
2023-01-14Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2023-01-14Make `ValidPathInfo` have plain `StorePathSet` references like beforeJohn Ericson
This change can wait for another PR.
2023-01-14Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-01-11Remove default constructor from `OutputsSpec`John Ericson
This forces us to be explicit. It also requires to rework how `from_json` works. A `JSON_IMPL` is added to assist with this.
2023-01-11Split `OutputsSpec` and `ExtendedOutputsSpec`, use the former moreJohn Ericson
`DerivedPath::Built` and `DerivationGoal` were previously using a regular set with the convention that the empty set means all outputs. But it is easy to forget about this rule when processing those sets. Using `OutputSpec` forces us to get it right.
2023-01-06Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2023-01-06Do big rename to clean up codeJohn Ericson
- `PathReferences` -> `References` - `PathReferences<StorePath>` -> `StoreReference` - `references` -> `others` - `hasSelfReference` -> `self` And get rid of silly subclassing
2023-01-06Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2023-01-06Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-01-02Fix why-depends for CA derivations (again)Théophane Hufschmitt
This has the same goal as b13fd4c58e81b2b2b0d72caa5ce80de861622610,but achieves it in a different way in order to not break `nix why-depends --derivation`.
2022-12-07Trivial changes from the lazy-trees branchEelco Dolstra
2022-08-22Merge remote-tracking branch 'origin/master' into parallel-nix-copyEelco Dolstra
2022-08-17Remove warnLargeDump()Eelco Dolstra
This message was unhelpful (#1184) and probably misleading since memory is O(1) in most cases now.
2022-06-08Restore the "low-latency" ssh copyingThéophane Hufschmitt
2022-05-30RemoteStore::queryRealisationUncached(): Fix potential noexcept violationEelco Dolstra
2022-05-04Get rid of most `.at` calls (#6393)Alain Zscheile
Use one of `get` or `getOr` instead which will either return a null-pointer (with a nicer error message) or a default value when the key is missing.
2022-03-10Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2022-03-10Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2022-03-09BuildResult: Use DerivedPathEelco Dolstra
2022-03-08Add Store::buildPathsWithResults()Eelco Dolstra
This function is like buildPaths(), except that it returns a vector of BuildResults containing the exact statuses and output paths of each derivation / substitution. This is convenient for functions like Installable::build(), because they then don't need to do another series of calls to get the outputs of CA derivations. It's also a precondition to impure derivations, where we *can't* query the output of those derivations since they're not stored in the Nix database. Note that PathSubstitutionGoal can now also return a BuildStatus.
2022-03-03Factor out a `GcStore` interfaceJohn Ericson
Starts progress on #5729. The idea is that we should not have these default methods throwing "unimplemented". This is a small step in that direction. I kept `addTempRoot` because it is a no-op, rather than failure. Also, as a practical matter, it is called all over the place, while doing other tasks, so the downcasting would be annoying. Maybe in the future I could move the "real" `addTempRoot` to `GcStore`, and the existing usecases use a `tryAddTempRoot` wrapper to downcast or do nothing, but I wasn't sure whether that was a good idea so with a bias to less churn I didn't do it yet.
2022-03-01Move `BuildResult` defintion to its own headerJohn Ericson
Just like we did for `ValidPathInfo` in d92d4f85a5c8a2a2385c084500a8b6bd54b54e6c.
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-02-16addToStoreFromDump(): Take std::string_viewEelco Dolstra
2022-01-25nix store ping: Report Nix daemon versionEelco Dolstra
Fixes #5952.
2022-01-18Add command 'nix store copy-log'Eelco Dolstra
Fixes #5222.
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2021-12-22Remove CPU lockingEelco Dolstra
This was already accidentally disabled in ba87b08. It also no longer appears to be beneficial, and in fact slow things down, e.g. when evaluating a NixOS system configuration: elapsed time: median = 3.8170 mean = 3.8202 stddev = 0.0195 min = 3.7894 max = 3.8600 [rejected, p=0.00000, Δ=0.36929±0.02513]
2021-12-07Tweak warningEelco Dolstra
2021-12-06Fallback when the daemon is too old for CA derivationsregnat
Fix #5712
2021-11-16Merge pull request #5472 from NixOS/async-realisation-substitutionEelco Dolstra
async realisation substitution
2021-11-09Merge pull request #5494 from tweag/balsoft/allow-references-in-addPathEelco Dolstra
Allow references in addPath
2021-11-09addToStore, addToStoreFromDump: refactor: pass refs by const referenceAlexander Bantyev
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2021-11-05Make the flake options work when using the daemonregnat
When setting flake-local options (with the `nixConfig` field), forward these options to the daemon in case we’re using one. This is necessary in particular for options like `binary-caches` or `post-build-hook` to make sense. Fix <https://github.com/NixOS/nix/commit/343239fc8a1993f707a990c2cd54a41f1fa3de99#r44356843>
2021-11-03Expose an async interface for `queryRealisation`regnat
Doesn’t change much so far because everything is still using it synchronously, but should allow the binary cache to fetch stuff in parallel
2021-10-23addToStore, addToStoreFromDump: add references argumentAlexander Bantyev
Allow to pass a set of references to be added as info to the added paths.
2021-10-13Remove syncWithGC()Eelco Dolstra
2021-10-01Merge branch 'path-info' into ca-drv-exoticJohn Ericson
2021-10-01Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2021-09-30Merge branch 'path-info' into ca-drv-exoticJohn Ericson