| Age | Commit message (Collapse) | Author |
|---|
|
Build ca derivations remotely
|
|
Mostly removing useless comments and adding spaces before `&`
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
|
|
That way we can copy the realisations too (in addition to the store
paths themselves)
|
|
- Pass it the name of the outputs rather than their output paths (as
these don't exist for ca derivations)
- Get the built output paths from the remote builder
- Register the new received realisations
|
|
Once a build is done, get back to the original derivation, and register
all the newly built outputs for this derivation.
This allows Nix to work properly with derivations that don't have all
their build inputs available − thus allowing garbage collection and
(once it's implemented) binary substitution
|
|
|
|
Thanks @regnat and @edolstra for catching this and comming up with the
solution.
They way I had generalized those is wrong, because local settings for
non-local stores is confusing default. And due to the nature of C++
inheritance, fixing the defaults is more annoying than it should be.
Additionally, I thought we might just drop the check in the substitution
logic since `Store::addToStore` is now streaming, but @regnat rightfully
pointed out that as it downloads dependencies first, that would still be
too late, and also waste effort on possibly unneeded/unwanted
dependencies.
The simple and correct thing to do is just make a store method for the
boolean logic, keeping all the setting and key stuff the way it was
before. That new method is both used by `LocalStore::addToStore` and the
substitution goal check. Perhaps we might eventually make it fancier,
e.g. sending the ValidPathInfo to remote stores for them to validate,
but this is good enough for now.
|
|
|
|
|
|
Remote stores still override so the other end schedules.
|
|
|
|
We embrace virtual the rest of the way, and get rid of the
`assert(false)` 0-param constructors.
We also list config base classes first, so the constructor order is
always:
1. all the configs
2. all the stores
Each in the same order
|
|
PRs #4370 and #4348 had a bad interaction in that the second broke the fist
one in a not trivial way.
The issue was that since #4348 the logic for detecting whether a
derivation output is already built requires some logic that was specific
to the `LocalStore`.
It happens though that most of this logic could be upstreamed to any `Store`,
which is what this commit does.
|
|
For each known realisation, store:
- its output
- its output path
This comes with a set of needed changes:
- New `realisations` module declaring the types needed for describing
these mappings
- New `Store::registerDrvOutput` method registering all the needed informations
about a derivation output (also replaces `LocalStore::linkDeriverToPath`)
- new `Store::queryRealisation` method to retrieve the informations for a
derivations
This introcudes some redundancy on the remote-store side between
`wopQueryDerivationOutputMap` and `wopQueryRealisation`.
However we might need to keep both (regardless of backwards compat)
because we sometimes need to get some infos for all the outputs of a
derivation (where `wopQueryDerivationOutputMap` is handy), but all the
stores can't implement it − because listing all the outputs of a
derivation isn't really possible for binary caches where the server
doesn't allow to list a directory.
|
|
Allow substituting paths when building remotely using `ssh-ng://`
|
|
|
|
Until now, it was not possible to substitute missing paths from e.g.
`https://cache.nixos.org` on a remote server when building on it using
the new `ssh-ng` protocol.
This is because every store implementation except legacy `ssh://`
ignores the substitution flag passed to `Store::queryValidPaths` while
the `legacy-ssh-store` substitutes the remote store using
`cmdQueryValidPaths` when the remote store is opened with `nix-store
--serve`.
This patch slightly modifies the daemon protocol to allow passing an
integer value suggesting whether to substitute missing paths during
`wopQueryValidPaths`. To implement this on the daemon-side, the
substitution logic from `nix-store --serve` has been moved into a
protected method named `Store::substitutePaths` which gets currently
called from `LocalStore::queryValidPaths` and `Store::queryValidPaths`
if `maybeSubstitute` is `true`.
Fixes #2770
|
|
This matches the already-existing `local-fs-store.cc`.
|
|
|
|
Fixes https://github.com/NixOS/nixos-org-configurations/issues/123.
|
|
Get rid of Hash::dummy from BinaryCacheStore
|
|
|
|
See comments for security concerns.
Also optimize goal creation by not traversing map twice.
|
|
Also checked that all usages satisfy the requirement and
removed dead code.
|
|
remove-storetype-delegate-regStore
|
|
|
|
So that it can be printed by `nix describe-stores`
|
|
|
|
|
|
When opening a store, only try the stores whose `uriSchemes()` include
the current one
|
|
Using virtual inheritance means that only the default constructors of
the parent classes will be called, which isn't what we want
|
|
Rework the `Store` hierarchy so that there's now one hierarchy for the
store configs and one for the implementations (where each implementation
extends the corresponding config). So a class hierarchy like
```
StoreConfig-------->Store
| |
v v
SubStoreConfig----->SubStore
| |
v v
SubSubStoreConfig-->SubSubStore
```
(with virtual inheritance to prevent DDD).
The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
|
|
Add a new `init()` method to the `Store` class that is supposed to
handle all the effectful initialisation needed to set-up the store.
The constructor should remain side-effect free and just initialize the
c++ data structure.
The goal behind that is that we can create “dummy” instances of each
store to query static properties about it (the parameters it accepts for
example)
|
|
|
|
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
|
|
remove-storetype-delegate-regStore
|
|
|
|
|
|
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
- `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
|
|
`queryDerivationOutputMap` no longer assumes all outputs have a mapping
|
|
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
- `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
|
|
remove-storetype-delegate-regStore
|
|
|
|
Make `system-features` a store setting
|
|
This seems more correct. It also means one can specify the features a
store should support with --store and remote-store=..., which is useful.
I use this to clean up the build remotes test.
|
|
|
|
|
|
drv-outputs-map-allow-missing
|
|
|
|
|
