aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2021-03-17Remove unimplemented hashAlgoOptRobert Hensing
It was in the header but never implemented.
2021-03-15Merge pull request #4587 from ↵Eelco Dolstra
obsidiansystems/derivation-goal-detect-invalid-output Throw error for derivation goal with bogus wanted output
2021-03-15Merge pull request #4580 from obsidiansystems/restore-test-build-remote-ca-fixedEelco Dolstra
Restore now-working build-remote-content-addressed-fixed test
2021-03-15Properly sign the unresolved drvsregnat
Don't let them inherit the signature from the parent one (because it makes no sense to do so), but re-sign them after they have been built
2021-03-15Check the signatures when copying store paths aroundregnat
Broken atm
2021-03-15pathInfoIsTrusted -> pathInfoIsUntrustedregnat
I guess the rationale behind the old name wath that `pathInfoIsTrusted(info)` returns `true` iff we would need to `blindly` trust the path (because it has no valid signature and `requireSigs` is set), but I find it to be a really confusing footgun because it's quite natural to give it the opposite meaning.
2021-03-15Add some logic for signing realisationsregnat
Not exposed anywhere, but built realisations are now signed (and this should be forwarded when copy-ing them around)
2021-03-15Merge pull request #4592 from NixOS/ca/remote-cacheEelco Dolstra
Substitute content-addressed derivations
2021-03-05Avoid some StorePath -> Path -> StorePath roundtripsJohn Ericson
There were done when StorePath was defined in Rust and there were some FFI issues. This is no longer an issue.
2021-03-01Fix bad wanted output error as requestedJohn Ericson
- UsageError -> Error - include drv path too
2021-03-01Allow substituting drv outputs when buildingregnat
2021-03-01SubstitutionGoal -> PathSubstitutionGoalregnat
To prepare for the upcoming DrvOutputSubstitutionGoal
2021-03-01Merge pull request #4574 from grahamc/libstore-ssh-host-keyEelco Dolstra
libstore: support passing a builder's public SSH host key
2021-02-28Throw error for derivation goal with bogus wanted outputJohn Ericson
2021-02-27Fix testing fixed-output derivations in double sandboxesJohn Ericson
What happened was that Nix was trying to unconditionally mount these paths in fixed-output derivations, but since the outer derivation was pure, those paths did not exist. The solution is to only mount those paths when they exist.
2021-02-26Properly propagate libseccomp linker flagsPuck Meerburg
2021-02-26Remove unused `redirectedBadOutputs`John Ericson
2021-02-26Remove temporary `#if 0...#endif` from previous commitJohn Ericson
2021-02-26Split {,local-}derivation-goal.{cc,hh}John Ericson
This separates the scheduling logic (including simple hook pathway) from the local-store needing code. This should be the final split for now. I'm reasonably happy with how it's turning out, even before I'm done moving code into `local-derivation-goal`. Benefits: 1. This will help "witness" that the hook case is indeed a lot simpler, and also compensate for the increased complexity that comes from content-addressed derivation outputs. 2. It also moves us ever so slightly towards a world where we could use off-the-shelf storage or sandboxing, since `local-derivation-goal` would be gutted in those cases, but `derivation-goal` should remain nearly the same. The new `#if 0` in the new files will be deleted in the following commit. I keep it here so if it turns out more stuff can be moved over, it's easy to do so in a way that preserves ordering --- and thus prevents conflicts. N.B. ```sh git diff HEAD^^ --color-moved --find-copies-harder --patience --stat ``` makes nicer output.
2021-02-26Copy {,local-}derivation-goal.{cc,h}John Ericson
Doing this prior to splitting, so we get better diff with default options (e.g. on GitHub).
2021-02-26Merge pull request #4477 from NixOS/ca/build-remoteEelco Dolstra
Build ca derivations remotely
2021-02-26Simplify the case where the drv is a purely input-addressed oneregnat
2021-02-26Update src/libstore/build/derivation-goal.ccEelco Dolstra
2021-02-26Use std::make_uniqueEelco Dolstra
2021-02-26Merge remote-tracking branch 'origin/deduplicate-static-hashes-computation'Eelco Dolstra
2021-02-25Make `nix copy` work without the ca-derivations flagregnat
The experimental feature was by mistake required for `nix copy` to work at oll
2021-02-25Apply @edolstra stylistic suggestionsThéophane Hufschmitt
Mostly removing useless comments and adding spaces before `&` Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-25Ensure that the ca-derivations bit is set when copying realisationsregnat
This should already hold, but better ensure it for future-proof-nees
2021-02-25Don't crash when copying realisations to a non-ca remoteregnat
Rather throw a proper exception, and catch&log it on the client side
2021-02-25Use `RealisedPath`s in `copyPaths`regnat
That way we can copy the realisations too (in addition to the store paths themselves)
2021-02-25distributed builds: load remote builder host key from the machines fileGraham Christensen
This is already used by Hydra, and is very useful when materializing a remote builder list from service discovery. This allows the service discovery tool to only sync one file instead of two.
2021-02-24Initialize plugins after handling initial command line flagsShea Levy
This is technically a breaking change, since attempting to set plugin files after the first non-flag argument will now throw an error. This is acceptable given the relative lack of stability in a plugin interface and the need to tie the knot somewhere once plugins can actually define new subcommands.
2021-02-24Bail if plugin-files is set after plugins have been loaded.Shea Levy
We know the flag will be ignored but the user wants it to take effect.
2021-02-23Merge pull request #4551 from danieldk/system-features-compute-levelEelco Dolstra
Add x86_64 compute levels as system features
2021-02-23Make `DerivationGoal::drv` a full Derivationregnat
This field used to be a `BasicDerivation`, but this `BasicDerivation` was downcasted to a `Derivation` when needed (implicitely or not), so we might as well make it a full `Derivation` and upcast it when needed. This also allows getting rid of a weird duplication in the way we compute the static output hashes for the derivation. We had to do it differently and in a different place depending on whether the derivation was a full derivation or just a basic drv, but we can now do it unconditionally on the full derivation. Fix #4559
2021-02-23Properly bypass the registering step when all outputs are presentregnat
There was already some logic for that, but it didn't handle the case of content-addressed outputs, so extend it a bit for that
2021-02-23Also send ca outputs to the build hookregnat
Otherwise they don't get registered, triggering an assertion failure at some point later
2021-02-23RemoteStore: Send back the new realisationsregnat
To allow it to build ca derivations remotely
2021-02-23LegacySSHStore: Send back the new realisationsregnat
To allow it to build ca derivations remotely
2021-02-23LocalStore: Send back the new realisationsregnat
To allow it to build ca derivations remotely
2021-02-23Make the build-hook work with ca derivationsregnat
- Pass it the name of the outputs rather than their output paths (as these don't exist for ca derivations) - Get the built output paths from the remote builder - Register the new received realisations
2021-02-22Warn about instability of plugin APIShea Levy
2021-02-22Add x86_64 compute levels as additional system typesDaniël de Kok
When performing distributed builds of machine learning packages, it would be nice if builders without the required SIMD instructions can be excluded as build nodes. Since x86_64 has accumulated a large number of different instruction set extensions, listing all possible extensions would be unwieldy. AMD, Intel, Red Hat, and SUSE have recently defined four different microarchitecture levels that are now part of the x86-64 psABI supplement and will be used in glibc 2.33: https://gitlab.com/x86-psABIs/x86-64-ABI https://lwn.net/Articles/844831/ This change uses libcpuid to detect CPU features and then uses them to add the supported x86_64 levels to the additional system types. For example on a Ryzen 3700X: $ ~/aps/bin/nix -vv --version | grep "Additional system" Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
2021-02-19Remove the drv resolution caching mechanismregnat
It isn't needed anymore now that don't need to eagerly resolve everything like we used to do. So we can safely get rid of it
2021-02-19Store the output hashes in the initialOutputs of the drv goalregnat
That way we 1. Don't have to recompute them several times 2. Can compute them in a place where we know the type of the parent derivation, meaning that we don't need the casting dance we had before
2021-02-19Don't expose the "bang" drvoutput syntaxThéophane Hufschmitt
It's not fixed nor useful atm, so better keep it hidden Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-19Syntactic fixesThéophane Hufschmitt
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-19Register the realisations for unresolved drvsregnat
Once a build is done, get back to the original derivation, and register all the newly built outputs for this derivation. This allows Nix to work properly with derivations that don't have all their build inputs available − thus allowing garbage collection and (once it's implemented) binary substitution
2021-02-09Revert "narinfo: Change NAR URLs to be addressed on the NAR hash instead of ↵Graham Christensen
the compressed hash"
2021-02-09Merge pull request #4464 from tweag/nar-narhash-addressedEelco Dolstra
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash