aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/build
AgeCommit message (Collapse)Author
2022-06-23Merge pull request #6673 from asymmetric/warnThéophane Hufschmitt
libstore: improve warning message on missing sig
2022-06-23Embed the sandbox shell into the statically linked 'nix' binaryEelco Dolstra
With this, Nix will write a copy of the sandbox shell to /bin/sh in the sandbox rather than bind-mounting it from the host filesystem. This makes /bin/sh work out of the box with nix-static, i.e. you no longer get /nix/store/qa36xhc5gpf42l3z1a8m1lysi40l9p7s-bootstrap-stage4-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory
2022-06-23Fix build-remote in nix-staticEelco Dolstra
'build-remote' is now executed via /proc/self/exe so it always works.
2022-06-15libstore: improve warning message on missing sigLorenzo Manacorda
Clarifies that the substitute will be ignored/skipped.
2022-05-04Fix compiler warningEelco Dolstra
2022-05-04Move json stuff out of util.ccEelco Dolstra
2022-05-04Get rid of most `.at` calls (#6393)Alain Zscheile
Use one of `get` or `getOr` instead which will either return a null-pointer (with a nicer error message) or a default value when the key is missing.
2022-04-28Fix passing $OUT_PATHS to the post-build hookEelco Dolstra
Fixes #6446.
2022-04-27local-derivation-goal.cc: seccomp filters for MIPS secondary arch/abiAdam Joseph
A mips64el Linux MIPS kernel can execute userspace code using any of three ABIs: mips64el-linux-*abin64 mips64el-linux-*abin32 mipsel-linux-* The first of these is the native 64-bit ABI, and the only ABI with 64-bit pointers; this is sometimes called "n64". The last of these is the old legacy 32-bit ABI, whose binaries can execute natively on 32-bit MIPS hardware; this is sometimes called "o32". The second ABI, "n32" is essentially the 64-bit ABI with 32-bit pointers and address space. Hardware 64-bit integer/floating arithmetic is still allowed, as well as the much larger mips64 register set and more-efficient calling convention. Let's enable seccomp filters for all of these. Likewise for big endian (mips64-linux-*).
2022-04-10Log builder args and environment variablesSebastian Blunt
Previously it only logged the builder's path, this changes it to log the arguments at the same log level, and the environment variables at the vomit level. This helped me debug https://github.com/svanderburg/node2nix/issues/75
2022-04-08Remove duplicate "error:"Eelco Dolstra
2022-04-06Fix empty 'nix copy' error messageEelco Dolstra
This was caused by SubstitutionGoal not setting the errorMsg field in its BuildResult. We now get a more descriptive message than in 2.7.0, e.g. error: path '/nix/store/13mh...' is required, but there is no substituter that can build it instead of the misleading (since there was no build) error: build of '/nix/store/13mh...' failed Fixes #6295.
2022-03-31needsNetworkAccess() -> isSandboxed()Eelco Dolstra
2022-03-31Fix macOS buildEelco Dolstra
2022-03-31Support fixed-output derivations depending on impure derivationsEelco Dolstra
2022-03-31Add support for impure derivationsEelco Dolstra
Impure derivations are derivations that can produce a different result every time they're built. Example: stdenv.mkDerivation { name = "impure"; __impure = true; # marks this derivation as impure outputHashAlgo = "sha256"; outputHashMode = "recursive"; buildCommand = "date > $out"; }; Some important characteristics: * This requires the 'impure-derivations' experimental feature. * Impure derivations are not "cached". Thus, running "nix-build" on the example above multiple times will cause a rebuild every time. * They are implemented similar to CA derivations, i.e. the output is moved to a content-addressed path in the store. The difference is that we don't register a realisation in the Nix database. * Pure derivations are not allowed to depend on impure derivations. In the future fixed-output derivations will be allowed to depend on impure derivations, thus forming an "impurity barrier" in the dependency graph. * When sandboxing is enabled, impure derivations can access the network in the same way as fixed-output derivations. In relaxed sandboxing mode, they can access the local filesystem.
2022-03-25Use wantOutputEelco Dolstra
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-24Retry substitution after an incomplete closure only onceEelco Dolstra
This avoids an infinite loop in the final test in tests/binary-cache.sh. I think this was only not triggered previously by accident (because we were clearing wantedOutputs in between).
2022-03-24Only return wanted outputsEelco Dolstra
2022-03-24Random cleanupEelco Dolstra
2022-03-24Random cleanupEelco Dolstra
2022-03-18Generalize `DerivationType` in preparation for impure derivationsJohn Ericson
2022-03-17Clean up `DerivationOutput`, and headersJohn Ericson
1. `DerivationOutput` now as the `std::variant` as a base class. And the variants are given hierarchical names under `DerivationOutput`. In 8e0d0689be797f9e42f9b43b06f50c1af7f20b4a @matthewbauer and I didn't know a better idiom, and so we made it a field. But this sort of "newtype" is anoying for literals downstream. Since then we leaned the base class, inherit the constructors trick, e.g. used in `DerivedPath`. Switching to use that makes this more ergonomic, and consistent. 2. `store-api.hh` and `derivations.hh` are now independent. In bcde5456cc3295061a0726881c3e441444dd6680 I swapped the dependency, but I now know it is better to just keep on using incomplete types as much as possible for faster compilation and good separation of concerns.
2022-03-11Factor out a `LogStore` interfaceJohn Ericson
Continue progress on #5729. Just as I hoped, this uncovered an issue: the daemon protocol is missing a way to query build logs. This doesn't effect `unix://`, but does effect `ssh://`. A FIXME is left for this, so we come back to it later.
2022-03-09BuildResult: Remove unused drvPath fieldEelco Dolstra
2022-03-09BuildResult: Use DerivedPathEelco Dolstra
2022-03-08Add Store::buildPathsWithResults()Eelco Dolstra
This function is like buildPaths(), except that it returns a vector of BuildResults containing the exact statuses and output paths of each derivation / substitution. This is convenient for functions like Installable::build(), because they then don't need to do another series of calls to get the outputs of CA derivations. It's also a precondition to impure derivations, where we *can't* query the output of those derivations since they're not stored in the Nix database. Note that PathSubstitutionGoal can now also return a BuildStatus.
2022-03-07Merge pull request #6029 from Ma27/nix-log-ssh-ngThéophane Hufschmitt
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-03Factor out a `GcStore` interfaceJohn Ericson
Starts progress on #5729. The idea is that we should not have these default methods throwing "unimplemented". This is a small step in that direction. I kept `addTempRoot` because it is a no-op, rather than failure. Also, as a practical matter, it is called all over the place, while doing other tasks, so the downcasting would be annoying. Maybe in the future I could move the "real" `addTempRoot` to `GcStore`, and the existing usecases use a `tryAddTempRoot` wrapper to downcast or do nothing, but I wasn't sure whether that was a good idea so with a bias to less churn I didn't do it yet.
2022-03-01Move `BuildResult` defintion to its own headerJohn Ericson
Just like we did for `ValidPathInfo` in d92d4f85a5c8a2a2385c084500a8b6bd54b54e6c.
2022-02-28Merge remote-tracking branch 'upstream/master' into auto-uid-allocationJohn Ericson
2022-02-28libstore/build: add a few explanatory comments; simplifyMaximilian Bosch
2022-02-28libstore/derivation-goal: avoid double-parsing of JSON messagesMaximilian Bosch
To avoid that JSON messages are parsed twice in case of remote builds with `ssh-ng://`, I split up the original `handleJSONLogMessage` into three parts: * `parseJSONMessage(const std::string&)` checks if it's a message in the form of `@nix {...}` and tries to parse it (and prints an error if the parsing fails). * `handleJSONLogMessage(nlohmann::json&, ...)` reads the fields from the message and passes them to the logger. * `handleJSONLogMessage(const std::string&, ...)` behaves as before, but uses the two functions mentioned above as implementation. In case of `ssh-ng://`-logs the first two methods are invoked manually.
2022-02-28ssh-ng: also store build logs to make them accessible by `nix log`Maximilian Bosch
Right now when building a derivation remotely via $ nix build -j0 -f . hello -L --builders 'ssh://builder' it's possible later to read through the entire build-log by running `nix log -f . hello`. This isn't possible however when using `ssh-ng` rather than `ssh`. The reason for that is that there are two different ways to transfer logs in Nix through e.g. an SSH tunnel (that are used by `ssh`/`ssh-ng` respectively): * `ssh://` receives its logs from the fd pointing to `builderOut`. This is directly passed to the "log-sink" (and to the logger on each `\n`), hence `nix log` works here. * `ssh-ng://` however expects JSON-like messages (i.e. `@nix {log data in here}`) and passes it directly to the logger without doing anything with the `logSink`. However it's certainly possible to extract log-lines from this format as these have their own message-type in the JSON payload (i.e. `resBuildLogLine`). This is basically what I changed in this patch: if the code-path for `builderOut` is not reached and a `logSink` is initialized, the message was successfully processed by the JSON logger (i.e. it's in the expected format) and the line is of the expected type (i.e. `resBuildLogLine`), the line will be written to the log-sink as well. Closes #5079
2022-02-28Fix clang warningEelco Dolstra
2022-02-28Fix Darwin buildEelco Dolstra
Fixes #6169
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-02-21Remove std::set aliasEelco Dolstra
2022-02-16addToStoreFromDump(): Take std::string_viewEelco Dolstra
2022-02-12Fix using sandbox without user namespacesSebastian Ullrich
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2022-01-17serialise.hh: Use std::string_viewEelco Dolstra
2022-01-11Merge pull request #5898 from layus/repair-path-linksEelco Dolstra
Make --repair-path also repair corrupt optimised links
2022-01-11Prefer RepairFlag over bool when applicableGuillaume Maudoux
2022-01-11Make --repair-path also repair corrupt optimised linksGuillaume Maudoux
There already existed a smoke test for the link content length, but it appears that there exists some corruptions pernicious enough to replace the file content with zeros, and keeping the same length. --repair-path now goes as far as checking the content of the link, making it true to its name and actually repairing the path for such coruption cases.
2022-01-10'target' points to 'source'Eelco Dolstra
2022-01-10Merge branch 'recursive-nix-fix' of https://github.com/L-as/nixEelco Dolstra
2022-01-07DerivationGoal::loadDerivation(): Don't use derivationFromPath()Eelco Dolstra
This causes a recursive call to ensurePath(), which is not a good idea.
2022-01-06Merge pull request #5749 from GavinRay97/patch-1Domen Kožar
Better diagnostics if no valid signature found
2021-12-14Merge pull request #5769 from NixOS/ca/get-build-statsEelco Dolstra
Add a crude tracing mechansim for the build results