aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/export-import.cc
AgeCommit message (Collapse)Author
2020-07-27Merge remote-tracking branch 'upstream/master' into hash-always-has-typeJohn Ericson
2020-07-16Optimize `addToStoreSlow` and remove `TeeParseSink`John Ericson
2020-07-15Merge branch 'master' of github.com:NixOS/nix into hash-always-has-typeCarlo Nucera
2020-07-13Remove 'accessor' from addToStore()Eelco Dolstra
This is only used by hydra-queue-runner and it's better to implement it there.
2020-07-13NarAccessor: Run in constant memoryEelco Dolstra
2020-07-13Make 'nix copy' to file:// binary caches run in constant memoryEelco Dolstra
2020-06-19WIP bug fixingCarlo Nucera
2020-06-18Revert the `enum struct` changeJohn Ericson
Not a regular git revert as there have been many merges and things.
2020-06-18Merge branch 'enum-class' into no-hash-type-unknownJohn Ericson
2020-06-18Merge remote-tracking branch 'upstream/master' into enum-classJohn Ericson
2020-06-16Remove StorePath::clone() and related functionsEelco Dolstra
2020-06-03libutils/hash: remove default encodingzimbatm
This will make it easier to reason about the hash encoding and switch to SRI everywhere where possible.
2020-06-02Merge remote-tracking branch 'upstream/master' into no-hash-type-unknownJohn Ericson
2020-06-02Remove `HashType::Unknown`John Ericson
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may also make `Hash` itself require a known hash type, encoraging people to use `std::optional<Hash>` instead.
2020-05-29Remove addToStore variant as requested by `FIXME`John Ericson
The idea is it's always more flexible to consumer a `Source` than a plain string, and it might even reduce memory consumption. I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but the worked needed for that is far more up for interpretation, so I punted for now.
2020-03-29Use `enum struct` and drop prefixesJohn Ericson
This does a few enums; the rest will be gotten in subsequent commits.
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-07-03Replace a few bool flags with enumsEelco Dolstra
Functions like copyClosure() had 3 bool arguments, which creates a severe risk of mixing up arguments. Also, implement copyClosure() using copyPaths().
2017-05-16Improve progress indicatorEelco Dolstra
2017-03-01TeeSink: Pre-reserve string spaceEelco Dolstra
When receiving a very large file, this can prevent the string from having tobe copied, which temporarily doubles memory consumption.
2017-03-01RemoteStore::addToStore(): Send NAR rather than string containing NAREelco Dolstra
This allows the NAR to be streamed in the future (though we're not doing that yet).
2017-03-01Handle importing NARs containing files greater than 4 GiBEelco Dolstra
Also templatize readInt() to work for various integer types.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-10-21Remove addPathToAccessorEelco Dolstra
2016-10-21BinaryCacheStore: Optionally write a NAR listingEelco Dolstra
The store parameter "write-nar-listing=1" will cause BinaryCacheStore to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’ added to the binary cache. This file contains an XZ-compressed JSON file describing the contents of the NAR, excluding the contents of regular files. E.g. { "version": 1, "root": { "type": "directory", "entries": { "lib": { "type": "directory", "entries": { "Mcrt1.o": { "type": "regular", "size": 1288 }, "Scrt1.o": { "type": "regular", "size": 3920 }, } } } ... } } (The actual file has no indentation.) This is intended to speed up the NixOS channels programs index generator [1], since fetching gazillions of large NARs from cache.nixos.org is currently a bottleneck for updating the regular (non-small) channel. [1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
2016-10-07importPaths(): Fix accessor support for HydraEelco Dolstra
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-05-31nix-copy-closure / build-remote.pl: Disable signature checkingEelco Dolstra
This restores the Nix 1.11 behaviour.
2016-05-04Add a Store::addToStore() variant that accepts a NAREelco Dolstra
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.