aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/build
AgeCommit message (Collapse)Author
2021-06-24Eventually delete the CA paths lock filesregnat
Mark the lockfiles as having to eventually be deleted so that they don’t stay laying around in the store at the end of the build Fix #4936
2021-06-24Make CA derivations compatible with recursive Nixregnat
Add an access-control list to the realisations in recursive-nix (similar to the already existing one for store paths), so that we can build content-addressed derivations in the restricted store. Fix #4353
2021-06-23Display the diverging paths in case of a realisation mismatchregnat
2021-06-23Gracefully ignore a substituter if it holds an incompatible realisationregnat
2021-06-23Merge pull request #4838 from NixOS/ca/recursively-substitute-realisationsEelco Dolstra
Recursively substitute the realisations
2021-06-23Merge pull request #4836 from NixOS/ca/track-drvoutput-dependencies-2-le-retourEelco Dolstra
Track the dependencies of CA realisations
2021-06-21Remove a possible existing store path when building CA derivationsregnat
In case a previous interrupted build left a garbage path laying around, remove it before trying to move the path to its final location. Fix #4858
2021-06-18UDSRemoteStore: Support the 'root' store parameterEelco Dolstra
Useful when we're using a daemon with a chroot store, e.g. $ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot Then the client can now connect with $ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
2021-05-26Recursively substitute the realisationsregnat
Make sure that whenever we substitute a realisation, we also substitute its entire closure
2021-05-26Aso track the output path of the realisation dependenciesregnat
2021-05-26Properly track the drvoutput references when buildingregnat
2021-05-25Make the Nar hash non moduloregnat
It makes much more sense to have the Nar hash be a plain straight hash rather than a hash modulo
2021-05-12Relock wait for build slot goalsMatthew Bauer
When we don’t have enough free job slots to run a goal, we put it in the waitForBuildSlot list & unlock its output locks. This will continue from where we left off (tryLocalBuild). However, we need the locks to get reacquired when/if the goal ever restarts. So, we need to send it back through tryToBuild to get reqacquire those locks. I think this bug was introduced in https://github.com/NixOS/nix/pull/4570. It leads to some builds starting without proper locks.
2021-05-05Properly normalize the content-addressed pathsregnat
Make sure that their timestamp are always normalized. Otherwise, strange − and non-deterministic − things might happen, like https://github.com/NixOS/nixpkgs/issues/121813 Fix #4775
2021-05-02Mark `__impureHostDeps` paths as optionalEmily
Starting in macOS 11, the on-disk dylib bundles are no longer available, but nixpkgs needs to be able to keep compatibility with older versions that require `/usr/lib/libSystem.B.dylib` in `__impureHostDeps`. Allow it to keep backwards compatibility with these versions by marking these dependencies as optional. Fixes #4658.
2021-04-22Always register the realisations of input-addressed drvsregnat
Fix #4725
2021-04-19Include sys/wait.h everywhere WIFEXITED etc is usedAlyssa Ross
This is required on NetBSD, and I think FreeBSD too.
2021-04-07PathSubstitutionGoal: Clean up pipeEelco Dolstra
If there were many top-level goals (which are not destroyed until the very end), commands like $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' \ /run/current-system --no-check-sigs --substitute-on-destination could fail with "Too many open files". So now we do some explicit cleanup from amDone(). It would be cleaner to separate goals from their temporary internal state, but that would be a bigger refactor.
2021-04-05Rename BuildableJohn Ericson
2021-04-05"newtype" BuildableReqJohn Ericson
This makes for better types errors and allows us to give it methods.
2021-04-05Use `BuildableReq` for `buildPaths` and `ensurePath`John Ericson
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {} }` could mean "build `brvPath`" or "substitute `drvPath`" depending on context. It also brings the internals closer in line to the new CLI, by generalizing the `Buildable` type is used there and makes that distinction already. In doing so, relegate `StorePathWithOutputs` to being a type just for backwards compatibility (CLI and RPC).
2021-03-15Merge pull request #4587 from ↵Eelco Dolstra
obsidiansystems/derivation-goal-detect-invalid-output Throw error for derivation goal with bogus wanted output
2021-03-15Merge pull request #4580 from obsidiansystems/restore-test-build-remote-ca-fixedEelco Dolstra
Restore now-working build-remote-content-addressed-fixed test
2021-03-15Properly sign the unresolved drvsregnat
Don't let them inherit the signature from the parent one (because it makes no sense to do so), but re-sign them after they have been built
2021-03-15pathInfoIsTrusted -> pathInfoIsUntrustedregnat
I guess the rationale behind the old name wath that `pathInfoIsTrusted(info)` returns `true` iff we would need to `blindly` trust the path (because it has no valid signature and `requireSigs` is set), but I find it to be a really confusing footgun because it's quite natural to give it the opposite meaning.
2021-03-15Add some logic for signing realisationsregnat
Not exposed anywhere, but built realisations are now signed (and this should be forwarded when copy-ing them around)
2021-03-01Fix bad wanted output error as requestedJohn Ericson
- UsageError -> Error - include drv path too
2021-03-01Allow substituting drv outputs when buildingregnat
2021-03-01SubstitutionGoal -> PathSubstitutionGoalregnat
To prepare for the upcoming DrvOutputSubstitutionGoal
2021-02-28Throw error for derivation goal with bogus wanted outputJohn Ericson
2021-02-27Fix testing fixed-output derivations in double sandboxesJohn Ericson
What happened was that Nix was trying to unconditionally mount these paths in fixed-output derivations, but since the outer derivation was pure, those paths did not exist. The solution is to only mount those paths when they exist.
2021-02-26Remove unused `redirectedBadOutputs`John Ericson
2021-02-26Remove temporary `#if 0...#endif` from previous commitJohn Ericson
2021-02-26Split {,local-}derivation-goal.{cc,hh}John Ericson
This separates the scheduling logic (including simple hook pathway) from the local-store needing code. This should be the final split for now. I'm reasonably happy with how it's turning out, even before I'm done moving code into `local-derivation-goal`. Benefits: 1. This will help "witness" that the hook case is indeed a lot simpler, and also compensate for the increased complexity that comes from content-addressed derivation outputs. 2. It also moves us ever so slightly towards a world where we could use off-the-shelf storage or sandboxing, since `local-derivation-goal` would be gutted in those cases, but `derivation-goal` should remain nearly the same. The new `#if 0` in the new files will be deleted in the following commit. I keep it here so if it turns out more stuff can be moved over, it's easy to do so in a way that preserves ordering --- and thus prevents conflicts. N.B. ```sh git diff HEAD^^ --color-moved --find-copies-harder --patience --stat ``` makes nicer output.
2021-02-26Copy {,local-}derivation-goal.{cc,h}John Ericson
Doing this prior to splitting, so we get better diff with default options (e.g. on GitHub).
2021-02-26Merge pull request #4477 from NixOS/ca/build-remoteEelco Dolstra
Build ca derivations remotely
2021-02-26Simplify the case where the drv is a purely input-addressed oneregnat
2021-02-26Update src/libstore/build/derivation-goal.ccEelco Dolstra
2021-02-26Use std::make_uniqueEelco Dolstra
2021-02-23Make `DerivationGoal::drv` a full Derivationregnat
This field used to be a `BasicDerivation`, but this `BasicDerivation` was downcasted to a `Derivation` when needed (implicitely or not), so we might as well make it a full `Derivation` and upcast it when needed. This also allows getting rid of a weird duplication in the way we compute the static output hashes for the derivation. We had to do it differently and in a different place depending on whether the derivation was a full derivation or just a basic drv, but we can now do it unconditionally on the full derivation. Fix #4559
2021-02-23Properly bypass the registering step when all outputs are presentregnat
There was already some logic for that, but it didn't handle the case of content-addressed outputs, so extend it a bit for that
2021-02-23Also send ca outputs to the build hookregnat
Otherwise they don't get registered, triggering an assertion failure at some point later
2021-02-23LocalStore: Send back the new realisationsregnat
To allow it to build ca derivations remotely
2021-02-23Make the build-hook work with ca derivationsregnat
- Pass it the name of the outputs rather than their output paths (as these don't exist for ca derivations) - Get the built output paths from the remote builder - Register the new received realisations
2021-02-19Store the output hashes in the initialOutputs of the drv goalregnat
That way we 1. Don't have to recompute them several times 2. Can compute them in a place where we know the type of the parent derivation, meaning that we don't need the casting dance we had before
2021-02-19Register the realisations for unresolved drvsregnat
Once a build is done, get back to the original derivation, and register all the newly built outputs for this derivation. This allows Nix to work properly with derivations that don't have all their build inputs available − thus allowing garbage collection and (once it's implemented) binary substitution
2021-02-05Add more instrumentation for #4270Eelco Dolstra
2021-02-05Add a trace to readLine() failuresEelco Dolstra
Hopefully this helps to diagnose 'error: unexpected EOF reading a line' on macOS.
2021-01-29Merge pull request #4461 from NixOS/ca/error-logging-fixesEelco Dolstra
Fix some logging with ca derivations
2021-01-26Fix the error message when a dep is missingregnat
Fix a mismatch in the errors thrown when a needed output was missing from an input derivation that was leading to a wrong and quite misleading error message