aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/local-binary-cache-store.cc
AgeCommit message (Collapse)Author
2020-07-13Make 'nix copy' to s3:// binary caches run in constant memoryEelco Dolstra
2020-07-13Make 'nix copy' to file:// binary caches run in constant memoryEelco Dolstra
2020-07-13Fix 'nix verify --all' on a binary cache and add a testEelco Dolstra
2020-04-21remove 'format' from Error constructor callsBen Burdette
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-09-04BinaryCacheStore: Add index-debug-info optionEelco Dolstra
This integrates the functionality of the index-debuginfo program in nixos-channel-scripts to maintain an index of DWARF debuginfo files in a format usable by dwarffs. Thus the debug info index is updated by Hydra rather than by the channel mirroring script. Example usage: $ nix copy --to 'file:///tmp/binary-cache?index-debug-info=true' /nix/store/vr9mhcch3fljzzkjld3kvkggvpq38cva-nix-2.2.2-debug $ cat /tmp/binary-cache/debuginfo/036b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug {"archive":"../nar/0313h2kdhk4v73xna9ysiksp2v8xrsk5xsw79mmwr3rg7byb4ka8.nar.xz","member":"lib/debug/.build-id/03/6b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug"} Fixes #3083.
2018-05-30Make LocalBinaryCacheStore::narFromPath() run in constant memoryEelco Dolstra
This reduces memory consumption of nix copy --from file://... --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79 from 514 MiB to 18 MiB for an uncompressed binary cache, and from 192 MiB to 53 MiB for a bzipped binary cache. It may also be faster because fetching can happen concurrently with decompression/writing. Continuation of 48662d151bdf4a38670897beacea9d1bd750376a. Issue https://github.com/NixOS/nix/issues/1681.
2018-05-30Simplify the callback mechanismEelco Dolstra
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-03-15S3BinaryCacheStore: Set Content-TypeEelco Dolstra
This is necessary for serving log files to browsers.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-09-16Make computeFSClosure() single-threaded againEelco Dolstra
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-06-01HttpBinaryCacheStore: Fix caching of WantMassQueryEelco Dolstra
Also, test HttpBinaryCacheStore in addition to LocalBinaryCacheStore.
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-05-30Test the NAR info cacheEelco Dolstra
2016-05-30BinaryCacheStore: Remove buildPaths() / ensurePath()Eelco Dolstra
2016-04-29nix verify --all: Support local binary cachesEelco Dolstra
2016-04-29BinaryCacheStore: Make the signing key a parameterEelco Dolstra
2016-04-29Allow parameters in store URIsEelco Dolstra
This is to allow store-specific configuration, e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
2016-04-15BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-03-04Add option binary-cache-secret-key-file for signing binary cachesEelco Dolstra
2016-03-04BinaryCacheStore: Remove publicKeyFile argumentEelco Dolstra
The public key can be derived from the secret key, so there's no need for the user to supply it separately.
2016-03-02Provide function required by HydraEelco Dolstra
2016-03-02Eliminate local-binary-cache-store.hhEelco Dolstra
2016-03-02Make store implementations pluggableEelco Dolstra
This for instance allows hydra-queue-runner to add the S3 backend at runtime.
2016-02-24Move BinaryCacheStore / LocalBinaryCacheStore from HydraEelco Dolstra
So you can now do: $ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...