aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/store-api.cc
AgeCommit message (Collapse)Author
2020-01-21Pluggable fetchersEelco Dolstra
Flakes are now fetched using an extensible mechanism. Also lots of other flake cleanups.
2019-12-18Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-12-16nix-store -r: Handle symlinks to store pathsEelco Dolstra
Fixes #3270.
2019-12-11Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-12-04Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-11-26Fix clang warningsEelco Dolstra
2019-11-26Disallow empty store path namesEelco Dolstra
Fixes #3239.
2019-11-04Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-10-29queryPathInfoUncached(): Return const ValidPathInfoEelco Dolstra
2019-10-21Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-10-21Allow content-addressable paths to have referencesEelco Dolstra
This adds a command 'nix make-content-addressable' that rewrites the specified store paths into content-addressable paths. The advantage of such paths is that 1) they can be imported without signatures; 2) they can enable deduplication in cases where derivation changes do not cause output changes (apart from store path hashes). For example, $ nix make-content-addressable -r nixpkgs.cowsay rewrote '/nix/store/g1g31ah55xdia1jdqabv1imf6mcw0nb1-glibc-2.25-49' to '/nix/store/48jfj7bg78a8n4f2nhg269rgw1936vj4-glibc-2.25-49' ... rewrote '/nix/store/qbi6rzpk0bxjw8lw6azn2mc7ynnn455q-cowsay-3.03+dfsg1-16' to '/nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16' We can then copy the resulting closure to another store without signatures: $ nix copy --trusted-public-keys '' ---to ~/my-nix /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16 In order to support self-references in content-addressable paths, these paths are hashed "modulo" self-references, meaning that self-references are zeroed out during hashing. Somewhat annoyingly, this means that the NAR hash stored in the Nix database is no longer necessarily equal to the output of "nix hash-path"; for content-addressable paths, you need to pass the --modulo flag: $ nix path-info --json /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16 | jq -r .[].narHash sha256:0ri611gdilz2c9rsibqhsipbfs9vwcqvs811a52i2bnkhv7w9mgw $ nix hash-path --type sha256 --base32 /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16 1ggznh07khq0hz6id09pqws3a8q9pn03ya3c03nwck1kwq8rclzs $ nix hash-path --type sha256 --base32 /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16 --modulo iq6g2x4q62xp7y7493bibx0qn5w7xz67 0ri611gdilz2c9rsibqhsipbfs9vwcqvs811a52i2bnkhv7w9mgw
2019-10-10Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-10-09OCD performance fix: {find,count}+insert => insertEelco Dolstra
2019-09-04Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-09-03Add some noexceptsEelco Dolstra
This is to assert that callback functions should never throw (since the context in which they're called may not be able to handle the exception).
2019-09-03Ensure that Callback is called only onceEelco Dolstra
Also, make Callback movable but uncopyable.
2019-08-29Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-08-28Set maximum name length in NixMatthew Bauer
Previously we allowed any length of name for Nix derivations. This is bad because different file systems have different max lengths. To make things predictable, I have picked a max. This was done by trying to build this derivation: derivation { name = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; builder = "/no-such-path"; system = "x86_64-linux"; } Take off one a and it will not lead to file name too long. That ends up being 212 a’s. An even smaller max could be picked if we want to support more file systems. Working backwards, this is why: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-${name}.drv.chroot > 255 - 32 - 1 - 4 - 7 = 211
2019-07-11Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-07-10Revert "Fix 'error 9 while decompressing xz file'"Eelco Dolstra
This reverts commit 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91.
2019-07-06Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-07-02checkStoreName: give more precise/verbose error informationGraham Christensen
$ sudo ./inst/bin/nix-instantiate -E '"${./.git}"' error: The path name '.git' is invalid: it is illegal to start the name with a period. Path names are alphanumeric and can include the symbols +-._?= and must not begin with a period. Note: If '.git' is a source file and you cannot rename it on disk, builtins.path { name = ... } can be used to give it an alternative name.
2019-06-24Fix 'error 9 while decompressing xz file'Eelco Dolstra
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952. (cherry picked from commit a67cf5a3585c41dd9f219a2c7aa9cf67fa69520b)
2019-06-24Fix 'error 9 while decompressing xz file'Eelco Dolstra
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952.
2019-06-21Handle store symlinks in flake directoriesEelco Dolstra
E.g. 'nix path-info ./result' inside a flake directory now works again.
2019-05-02Add function for quoting stringsEelco Dolstra
2019-02-25Extract and expose splitUriAndParams functionDzmitry Zaitsau
which splits a URL into localtor and parameter parts
2019-01-18unsupported(): Show the name of the unsupported operationEelco Dolstra
2018-10-29copyStorePath: Fix hash errors when copying from older storeFalco Peijnenburg
This commit partially reverts 48662d151bdf4a38670897beacea9d1bd750376a. When copying from an older store (in my case a store running Nix 1.11.7), nix would throw errors about there being no hash. This is fixed by recalculating the hash.
2018-10-16Fix assertion failure in Store::queryPathInfo()Eelco Dolstra
$ nix-store -qR /nix/store/fnord nix-store: src/libstore/store-api.cc:80: std::__cxx11::string nix::storePathToHash(const Path&): Assertion `base.size() >= storePathHashLen' failed. Aborted
2018-09-26UntabifyEelco Dolstra
2018-08-21Improve 'coroutine has finished' error messageEelco Dolstra
2018-08-03Support escaping in store URIsEelco Dolstra
2018-07-24copyPathsToStore: honour keep-goingLinus Heckemann
2018-05-30Simplify the callback mechanismEelco Dolstra
2018-05-30Modularize config settingsEelco Dolstra
Allow global config settings to be defined in multiple Config classes. For example, this means that libutil can have settings and evaluator settings can be moved out of libstore. The Config classes are registered in a new GlobalConfig class to which config files etc. are applied. Relevant to https://github.com/NixOS/nix/issues/2009 in that it removes the need for ad hoc handling of useCaseHack, which was the underlying cause of that issue.
2018-05-30Fix assertion failure in storePathToHash()Eelco Dolstra
Fixes https://github.com/NixOS/nix/issues/2015.
2018-03-29ValidPathInfo::isContentAddressed(): Ensure there are no referencesEelco Dolstra
2018-03-16Reduce substitution memory consumptionEelco Dolstra
copyStorePath() now pipes the output of srcStore->narFromPath() directly into dstStore->addToStore(). The sink used by the former is converted into a source usable by the latter using boost::coroutine2. This is based on [1]. This reduces the maximum resident size of $ nix build --store ~/my-nix/ /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1 --substituters file:///tmp/binary-cache-xz/ --no-require-sigs from 418592 KiB to 53416 KiB. (The previous commit also reduced the runtime from ~4.2s to ~3.4s, not sure why.) A further improvement will be to download files into a Sink. [1] https://github.com/NixOS/nix/compare/master...Mathnerd314:dump-fix-coroutine#diff-dcbcac55a634031f9cc73707da6e4b18 Issue #1969.
2018-02-13Allow plugins to define new settings.Shea Levy
2018-02-09getDefaultSubstituters(): Skip broken substitutersEelco Dolstra
Fixes #1340.
2018-02-06Add path primop.Shea Levy
builtins.path allows specifying the name of a path (which makes paths with store-illegal names now addable), allows adding paths with flat instead of recursive hashes, allows specifying a filter (so is a generalization of filterSource), and allows specifying an expected hash (enabling safe path adding in pure mode).
2017-11-24nix path-info: Show URL of NARs in binary cachesEelco Dolstra
2017-11-20nix copy: Abbreviate "daemon"Eelco Dolstra
2017-10-25respect SIGINT in nix copy during the paths queries #1629AmineChikhaoui
2017-10-25attempt to fix #1630: make the queries of store paths run in parallel using ↵AmineChikhaoui
a thread pool
2017-10-24Allow shorter syntax for chroot storesEelco Dolstra
You can now say '--store /tmp/nix' instead of '--store local?root=/tmp/nix'.
2017-10-24More progress indicator improvementsEelco Dolstra
Fixes #1599.