aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/local-store.cc
AgeCommit message (Collapse)Author
2021-11-08Merge branch 'fix-writable-shell' of https://github.com/yorickvP/nixEelco Dolstra
2021-11-05addPath: allow paths with referencesAlexander Bantyev
Since 4806f2f6b0fd2cae401b89fe19d8c528ffd88b5f, we can't have paths with references passed to builtins.{path,filterSource}. This prevents many cases of those functions called on IFD outputs from working. Resolve this by passing the references found in the original path to the added path.
2021-11-03Expose an async interface for `queryRealisation`regnat
Doesn’t change much so far because everything is still using it synchronously, but should allow the binary cache to fetch stuff in parallel
2021-10-28Merge remote-tracking branch 'origin/master' into non-blocking-gcEelco Dolstra
2021-10-26Make experimental-features a proper typeregnat
Rather than having them plain strings scattered through the whole codebase, create an enum containing all the known experimental features. This means that - Nix can now `warn` when an unkwown experimental feature is passed (making it much nicer to spot typos and spot deprecated features) - It’s now easy to remove a feature altogether (once the feature isn’t experimental anymore or is dropped) by just removing the field for the enum and letting the compiler point us to all the now invalid usages of it.
2021-10-23addToStore, addToStoreFromDump: add references argumentAlexander Bantyev
Allow to pass a set of references to be added as info to the added paths.
2021-10-15Restore parent mount namespace in restoreProcessContextYorick van Pelt
This ensures any started processes can't write to /nix/store (except during builds). This partially reverts 01d07b1e, which happened because of #2646. The problem was only happening after nix downloads anything, causing me to suspect the download thread. The problem turns out to be: "A process can't join a new mount namespace if it is sharing filesystem-related attributes with another process", in this case this process is the curl thread. Ideally, we might kill it before spawning the shell process, but it's inside a static variable in the getFileTransfer() function. So instead, stop it from sharing FS state using unshare(). A strategy such as the one from #5057 (single-threaded chroot helper binary) is also very much on the table. Fixes #4337.
2021-10-14pathInfoCache: Use the entire base name as the cache keyEelco Dolstra
This fixes a bug in the garbage collector where if a path /nix/store/abcd-foo is valid, but we do a isValidPath("/nix/store/abcd-foo.lock") first, then a negative entry for /nix/store/abcd is added to pathInfoCache, so /nix/store/abcd-foo is subsequently considered invalid and deleted.
2021-10-13Fix auto-gcEelco Dolstra
2021-10-13Non-blocking garbage collectorEelco Dolstra
The garbage collector no longer blocks other processes from adding/building store paths or adding GC roots. To prevent the collector from deleting store paths just added by another process, processes need to connect to the garbage collector via a Unix domain socket to register new temporary roots.
2021-10-13Remove trash directoryEelco Dolstra
2021-10-01Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2021-09-30Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2021-09-30`std::visit` by referenceJohn Ericson
I had started the trend of doing `std::visit` by value (because a type error once mislead me into thinking that was the only form that existed). While the optomizer in principle should be able to deal with extra coppying or extra indirection once the lambdas inlined, sticking with by reference is the conventional default. I hope this might even improve performance.
2021-09-15Merge remote-tracking branch 'upstream/master' into auto-uid-allocationMatthew Kenigsberg
2021-09-14Remove the 'ca-references' feature checkEelco Dolstra
Fixes #3422. Fixes #4425.
2021-09-01StyleEelco Dolstra
2021-08-14Add ignored_acls settingMagic_RB
Signed-off-by: Magic_RB <magic_rb@redalder.org>
2021-08-08Short-circuit querying substituters on successSebastian Ullrich
2021-07-16Be more lenient when realisations have a conflicting dependency setregnat
- This can legitimately happen (for example because of a non-determinism causing a build-time dependency to be kept or not as a runtime reference) - Because of older Nix versions, it can happen that we encounter a realisation with an (erroneously) empty set of dependencies, in which case we don’t want to fail, but just warn the user and try to fix it.
2021-06-23Display the diverging paths in case of a realisation mismatchregnat
2021-06-23Fix indentationregnat
2021-06-23Properly fail when trying to register an incoherent realisationregnat
2021-06-23Merge pull request #4836 from NixOS/ca/track-drvoutput-dependencies-2-le-retourEelco Dolstra
Track the dependencies of CA realisations
2021-06-23Fix indentationregnat
2021-06-23Reformat the sql statementsregnat
2021-06-21Shorten a stupidly long sql query nameregnat
2021-06-18UDSRemoteStore: Support the 'root' store parameterEelco Dolstra
Useful when we're using a daemon with a chroot store, e.g. $ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot Then the client can now connect with $ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
2021-06-01Check the CA hash when importing stuff in the local storeregnat
When adding a path to the local store (via `LocalStore::addToStore`), ensure that the `ca` field of the provided `ValidPathInfo` does indeed correspond to the content of the path. Otherwise any untrusted user (or any binary cache) can add arbitrary content-addressed paths to the store (as content-addressed paths don’t need a signature).
2021-05-26Aso track the output path of the realisation dependenciesregnat
2021-05-26Add a db migration scriptregnat
2021-05-26Store the realisation deps on the local storeregnat
2021-05-26Remove the remaining occurenceses of a NarHash moduloregnat
2021-05-25Make the Nar hash non moduloregnat
It makes much more sense to have the Nar hash be a plain straight hash rather than a hash modulo
2021-04-27Merge remote-tracking branch 'upstream/master' into auto-uid-allocationJohn Ericson
2021-04-05Merge branch 'master' into path-infoJohn Ericson
2021-03-15Check the signatures when copying store paths aroundregnat
Broken atm
2021-03-15pathInfoIsTrusted -> pathInfoIsUntrustedregnat
I guess the rationale behind the old name wath that `pathInfoIsTrusted(info)` returns `true` iff we would need to `blindly` trust the path (because it has no valid signature and `requireSigs` is set), but I find it to be a really confusing footgun because it's quite natural to give it the opposite meaning.
2021-03-15Add some logic for signing realisationsregnat
Not exposed anywhere, but built realisations are now signed (and this should be forwarded when copy-ing them around)
2021-02-27Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2021-02-25Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2021-02-25Don't crash when copying realisations to a non-ca remoteregnat
Rather throw a proper exception, and catch&log it on the client side
2021-02-19Syntactic fixesThéophane Hufschmitt
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-19Register the realisations for unresolved drvsregnat
Once a build is done, get back to the original derivation, and register all the newly built outputs for this derivation. This allows Nix to work properly with derivations that don't have all their build inputs available − thus allowing garbage collection and (once it's implemented) binary substitution
2021-01-25Merge pull request #4467 from edolstra/error-formattingEelco Dolstra
Improve error formatting
2021-01-25Merge pull request #4387 from obsidiansystems/non-local-store-buildEelco Dolstra
Make `nix-build --store whatever` work
2021-01-21Improve error formattingEelco Dolstra
Changes: * The divider lines are gone. These were in practice a bit confusing, in particular with --show-trace or --keep-going, since then there were multiple lines, suggesting a start/end which wasn't the case. * Instead, multi-line error messages are now indented to align with the prefix (e.g. "error: "). * The 'description' field is gone since we weren't really using it. * 'hint' is renamed to 'msg' since it really wasn't a hint. * The error is now printed *before* the location info. * The 'name' field is no longer printed since most of the time it wasn't very useful since it was just the name of the exception (like EvalError). Ideally in the future this would be a unique, easily googleable error ID (like rustc). * "trace:" is now just "…". This assumes error contexts start with something like "while doing X". Example before: error: --- AssertionError ---------------------------------------------------------------------------------------- nix at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix 6| 7| x = assert false; 1; | ^ 8| assertion 'false' failed ----------------------------------------------------- show-trace ----------------------------------------------------- trace: while evaluating the attribute 'x' of the derivation 'hello-2.10' at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix 191| // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) { 192| name = "${attrs.pname}-${attrs.version}"; | ^ 193| } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) { Example after: error: assertion 'false' failed at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix 6| 7| x = assert false; 1; | ^ 8| … while evaluating the attribute 'x' of the derivation 'hello-2.10' at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix 191| // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) { 192| name = "${attrs.pname}-${attrs.version}"; | ^ 193| } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
2021-01-18Fix the drv output map for non ca derivationsregnat
With the `ca-derivation` experimental features, non-ca derivations used to have their output paths returned as unknown as long as they weren't built (because of a mistake in the code that systematically erased the previous value)
2021-01-15Make public keys and `requireSigs` local-store specific againJohn Ericson
Thanks @regnat and @edolstra for catching this and comming up with the solution. They way I had generalized those is wrong, because local settings for non-local stores is confusing default. And due to the nature of C++ inheritance, fixing the defaults is more annoying than it should be. Additionally, I thought we might just drop the check in the substitution logic since `Store::addToStore` is now streaming, but @regnat rightfully pointed out that as it downloads dependencies first, that would still be too late, and also waste effort on possibly unneeded/unwanted dependencies. The simple and correct thing to do is just make a store method for the boolean logic, keeping all the setting and key stuff the way it was before. That new method is both used by `LocalStore::addToStore` and the substitution goal check. Perhaps we might eventually make it fancier, e.g. sending the ValidPathInfo to remote stores for them to validate, but this is good enough for now.
2021-01-15Merge remote-tracking branch 'upstream/master' into non-local-store-buildJohn Ericson