aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/local-store.hh
AgeCommit message (Collapse)Author
2020-09-04Fix some of the issues raised by @edolstraJohn Ericson
- More and better comments - The easier renames
2020-08-28Merge remote-tracking branch 'upstream/master' into single-ca-drv-buildJohn Ericson
2020-08-27Merge pull request #3434 from Ericson2314/derivation-header-include-orderEelco Dolstra
Revise division of labor in deserialization of derivations
2020-08-20Rename drv output querying functions, like masterJohn Ericson
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap` - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
2020-08-20Rename drv output querying functionsJohn Ericson
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap` - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
2020-08-18Merge remote-tracking branch 'upstream/master' into ↵John Ericson
derivation-header-include-order
2020-08-07Squashed get CA derivations buildingJohn Ericson
2020-08-04Merge remote-tracking branch 'upstream/master' into ↵John Ericson
drv-outputs-map-allow-missing
2020-08-03`addToStore` in terms of `addToStoreFromDump` is not local-store-specificJohn Ericson
2020-08-01Merge remote-tracking branch 'upstream/master' into ↵John Ericson
derivation-header-include-order
2020-07-31Merge branch 'master' of github.com:NixOS/nix into drv-outputs-map-allow-missingCarlo Nucera
2020-07-30Merge remote-tracking branch 'origin/master' into substitute-other-storedirMatthew Bauer
2020-07-30unsigned long long -> uint64_tEelco Dolstra
2020-07-24`queryDerivationOutputMap` no longer assumes all outputs have a mappingJohn Ericson
This assumption is broken by CA derivations. Making a PR now to do the breaking daemon change as soon as possible (if it is already too late, we can bump protocol intead).
2020-07-15Get rid of `LocalStore::addToStoreCommon`John Ericson
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking a store and then doing nothing too fancy with it. `LocalStore::addToStore(...Path...)` is now just a simple wrapper with a bare-bones sinkToSource of the right dump command.
2020-07-15Merge branch 'master' of github.com:NixOS/nix into ↵Carlo Nucera
derivation-header-include-order
2020-07-14LocalStore::addToStoreFromDump copy in chunksJohn Ericson
Rather than copying byte-by-byte, we let the coroutine know how much data we would like it to send back to us.
2020-07-14Dedup `LocalStore::addToStore*`John Ericson
The downsides is that the coroutine has byte-by-byte loop transfer. Will fix that next.
2020-07-14Crudely make `addToStoreFromDump` take `Source` not stringJohn Ericson
I just as little beyond the type as possible, so the implementation changes this enables can be reviewed separately.
2020-07-13Remove 'accessor' from addToStore()Eelco Dolstra
This is only used by hydra-queue-runner and it's better to implement it there.
2020-07-05Merge remote-tracking branch 'upstream/master' into ↵John Ericson
derivation-header-include-order
2020-07-02Merge remote-tracking branch 'origin/master' into substitute-other-storedirMatthew Bauer
2020-06-24Add a way to get all the outputs of a derivation with their labelregnat
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by adding a `queryDerivationOutputMap` that returns the map `outputName=>outputPath` (not that this is not equivalent to merging the results of `queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't preserve the order, so we would end up with an incorrect mapping). squash! Add a way to get all the outputs of a derivation with their label Rename StorePathMap to OutputPathMap
2020-06-17Add StorePathCAMap for querySubstitutablePathInfosMatthew Bauer
I’m not 100% sure this is wanted since it kind of makes everything have to know about ca even if they don’t really want to. But it also make things easier in dealing with looking up ca.
2020-06-17Merge remote-tracking branch 'origin/master' into substitute-other-storedirMatthew Bauer
2020-06-17Don't need abstract `struct Derivation` in local-storeJohn Ericson
2020-06-12Allow substituting from different storeDirMatthew Bauer
Substituters can substitute from one store dir to another with a little bit of help. The store api just needs to have a CA so it can recompute the store path based on the new store dir. We can only do this for fixed output derivations with no references, though.
2020-06-12Remove Store::queryDerivationOutputNames()Eelco Dolstra
This function was used in only one place, where it could easily be replaced by readDerivation() since it's not performance-critical. (This function appears to have been modelled after queryDerivationOutputs(), which exists only to make the garbage collector faster.)
2020-03-29Replace some `bool recursive` with a new `FileIngestionMethod` enumJohn Ericson
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-11-22getEnv(): Return std::optionalEelco Dolstra
This allows distinguishing between an empty value and no value.
2019-10-29queryPathInfoUncached(): Return const ValidPathInfoEelco Dolstra
2019-10-29Move addToStoreFromDump to StoreEelco Dolstra
2019-10-09Remove world-writability from per-user directoriesEelco Dolstra
'nix-daemon' now creates subdirectories for users when they first connect. Fixes #509 (CVE-2019-17365). Should also fix #3127.
2019-09-03Add some noexceptsEelco Dolstra
This is to assert that callback functions should never throw (since the context in which they're called may not be able to handle the exception).
2019-08-02Use BSD instead of POSIX file locksEelco Dolstra
POSIX file locks are essentially incompatible with multithreading. BSD locks have much saner semantics. We need this now that there can be multiple concurrent LocalStore::buildPaths() invocations.
2019-03-14findRoots(): Add 'censor' parameterEelco Dolstra
This is less brittle than filtering paths after the fact in nix-daemon.
2019-03-10Unify internal findRootsXxx() apiGuillaume Maudoux
2019-03-10detail memory rootsGuillaume Maudoux
2018-09-02Store: expose the protocol version used by a storeDaiderd Jordan
2018-05-30Simplify the callback mechanismEelco Dolstra
2018-04-13Fix #1921Eelco Dolstra
2018-03-16Reduce substitution memory consumptionEelco Dolstra
copyStorePath() now pipes the output of srcStore->narFromPath() directly into dstStore->addToStore(). The sink used by the former is converted into a source usable by the latter using boost::coroutine2. This is based on [1]. This reduces the maximum resident size of $ nix build --store ~/my-nix/ /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1 --substituters file:///tmp/binary-cache-xz/ --no-require-sigs from 418592 KiB to 53416 KiB. (The previous commit also reduced the runtime from ~4.2s to ~3.4s, not sure why.) A further improvement will be to download files into a Sink. [1] https://github.com/NixOS/nix/compare/master...Mathnerd314:dump-fix-coroutine#diff-dcbcac55a634031f9cc73707da6e4b18 Issue #1969.
2018-02-12Fix hang in build-remoteEelco Dolstra
2018-01-31Rename 1.12 -> 2.0Eelco Dolstra
Following discussion with Shea and Graham. It's a big enough change from the last release. Also, from a semver perspective, 2.0 makes more sense because we did remove some interfaces (like nix-pull/nix-push).
2017-11-20signed-binary-caches -> require-sigsEelco Dolstra
Unlike signed-binary-caches (which could only be '*' or ''), require-sigs is a proper Boolean option. The default is true.
2017-09-14Show the pid of temporary rootsEelco Dolstra
2017-09-14nix-store -q --roots / --gc --print-roots: Print temporary / in-memory rootsEelco Dolstra
For example, $ nix-store -q --roots /nix/store/7phd2sav7068nivgvmj2vpm3v47fd27l-patchelf-0.8pre845_0315148 {temp:1} denotes that the path is only being kept alive by a temporary root (i.e. /nix/var/nix/temproots/). Similarly, $ nix-store --gc --print-roots ... {memory:9} -> /nix/store/094gpjn9f15ip17wzxhma4r51nvsj17p-curl-7.53.1 shows that curl is being used by some process.
2017-09-05Add automatic garbage collectionEelco Dolstra
Nix can now automatically run the garbage collector during builds or while adding paths to the store. The option "min-free = <bytes>" specifies that Nix should run the garbage collector whenever free space in the Nix store drops below <bytes>. It will then delete garbage until "max-free" bytes are available. Garbage collection during builds is asynchronous; running builds are not paused and new builds are not blocked. However, there also is a synchronous GC run prior to the first build/substitution. Currently, no old GC roots are deleted (as in "nix-collect-garbage -d").
2017-09-05GC: Don't delete own temproots fileEelco Dolstra
Since file locks are per-process rather than per-file-descriptor, the garbage collector would always acquire a lock on its own temproots file and conclude that it's stale.