aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/binary-cache-store.cc
AgeCommit message (Collapse)Author
2020-12-16Fix BinaryCacheStore::registerDrvOutputregnat
Was crashing because coercing a json document into a string is only valid if the json is a string, otherwise we need to call `.dump()`
2020-12-15Fix `addTextToStore` for binary cachesregnat
Because of a too eager refactoring, `addTextToStore` used to throw an error because the input wasn't a valid nar. Partially revert that refactoring to wrap the text into a proper nar (using `dumpString`) to make this method work again
2020-12-11Store the realisations as JSON in the binary cacheregnat
Fix #4332
2020-12-11Store metadata about drv outputs realisationsregnat
For each known realisation, store: - its output - its output path This comes with a set of needed changes: - New `realisations` module declaring the types needed for describing these mappings - New `Store::registerDrvOutput` method registering all the needed informations about a derivation output (also replaces `LocalStore::linkDeriverToPath`) - new `Store::queryRealisation` method to retrieve the informations for a derivations This introcudes some redundancy on the remote-store side between `wopQueryDerivationOutputMap` and `wopQueryRealisation`. However we might need to keep both (regardless of backwards compat) because we sometimes need to get some infos for all the outputs of a derivation (where `wopQueryDerivationOutputMap` is handy), but all the stores can't implement it − because listing all the outputs of a derivation isn't really possible for binary caches where the server doesn't allow to list a directory.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-09-28Update src/libstore/binary-cache-store.ccJohn Ericson
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-09-26Remove redundant nar hash and size settingJohn Ericson
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-09-26Fix up BinaryCacheStore::addToStore taking a pathJohn Ericson
2020-09-26BinaryCacheStore::addTextToStore include CA fieldJohn Ericson
2020-09-23Return more info from `BinaryCacheStore::addToStoreCommon`John Ericson
We don't need it yet, but we could/should in the future, and it's a cost-free change since we already have the reference. I like it. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-09-23Clarify FIXME in `BinaryCacheStore::addToStoreCommon`John Ericson
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-09-23Get rid of Hash::dummy from BinaryCacheStoreJohn Ericson
2020-09-21Move Callback into its own headerEelco Dolstra
This gets rid of the inclusion of <future> in util.hh, cutting compilation time by ~20s (CPU time). Issue #4045.
2020-09-16Separate store configs from the implemsregnat
Rework the `Store` hierarchy so that there's now one hierarchy for the store configs and one for the implementations (where each implementation extends the corresponding config). So a class hierarchy like ``` StoreConfig-------->Store | | v v SubStoreConfig----->SubStore | | v v SubSubStoreConfig-->SubSubStore ``` (with virtual inheritance to prevent DDD). The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
2020-08-14Merge pull request #3899 from obsidiansystems/make-narHash-not-optionalEelco Dolstra
Make narHash in ValidPathInfo not optional
2020-08-13Use `TeeSink` and `TeeSouce` in a few more placesJohn Ericson
2020-08-07Fix .ls file names in binary cachesEelco Dolstra
These are not supposed to include the 'name' part of the store path. This was broken by 759947bf72.
2020-08-06Minimize the usage of `Hash::dummy`John Ericson
2020-08-05Remove Hash::operator bool ()Carlo Nucera
Since the hash is not optional anymore
2020-08-04S3BinaryCacheStore: Fix size determinationEelco Dolstra
2020-08-04BinaryCacheStore: Explicitly flush file sinkEelco Dolstra
The file sink is also flushed in its destructor, but we ignore any exceptions in the destructor. Issue #3886.
2020-08-03Delete compressed NARsEelco Dolstra
Fixes #3891.
2020-07-15Merge branch 'master' of github.com:NixOS/nix into hash-always-has-typeCarlo Nucera
2020-07-13Make 'nix copy' to s3:// binary caches run in constant memoryEelco Dolstra
2020-07-13Remove 'accessor' from addToStore()Eelco Dolstra
This is only used by hydra-queue-runner and it's better to implement it there.
2020-07-13NarAccessor: Run in constant memoryEelco Dolstra
2020-07-13Make 'nix copy' to file:// binary caches run in constant memoryEelco Dolstra
2020-06-19WIP bug fixingCarlo Nucera
2020-06-17Remove unused narInfoFile in binary-cache-storeMatthew Bauer
2020-06-16Remove StorePath::clone() and related functionsEelco Dolstra
2020-06-16StorePath: Rewrite in C++Eelco Dolstra
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB and runtime by 0.8%. This is mostly because we're not parsing the hash part as a hash anymore (just validating that it consists of base-32 characters). Also, replace storePathToHash() by StorePath::hashPart().
2020-06-03Merge remote-tracking branch 'upstream/master' into errors-phase-2Ben Burdette
2020-05-29Remove addToStore variant as requested by `FIXME`John Ericson
The idea is it's always more flexible to consumer a `Source` than a plain string, and it might even reduce memory consumption. I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but the worked needed for that is far more up for interpretation, so I punted for now.
2020-05-29Merge remote-tracking branch 'upstream/master' into errors-phase-2Ben Burdette
2020-05-27Rename some variables named “recursive” to “method”Matthew Bauer
This is much less confusing since recursive is no longer a boolean.
2020-04-29errorinfo constructor testBen Burdette
2020-04-21remove 'format' from Error constructor callsBen Burdette
2020-03-29Replace some `bool recursive` with a new `FileIngestionMethod` enumJohn Ericson
2020-03-12pathInfoCache: Respect disk cache TTLs #3398Robert Hensing
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-10-29queryPathInfoUncached(): Return const ValidPathInfoEelco Dolstra
2019-09-04BinaryCacheStore: Add index-debug-info optionEelco Dolstra
This integrates the functionality of the index-debuginfo program in nixos-channel-scripts to maintain an index of DWARF debuginfo files in a format usable by dwarffs. Thus the debug info index is updated by Hydra rather than by the channel mirroring script. Example usage: $ nix copy --to 'file:///tmp/binary-cache?index-debug-info=true' /nix/store/vr9mhcch3fljzzkjld3kvkggvpq38cva-nix-2.2.2-debug $ cat /tmp/binary-cache/debuginfo/036b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug {"archive":"../nar/0313h2kdhk4v73xna9ysiksp2v8xrsk5xsw79mmwr3rg7byb4ka8.nar.xz","member":"lib/debug/.build-id/03/6b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug"} Fixes #3083.
2019-09-03Add some noexceptsEelco Dolstra
This is to assert that callback functions should never throw (since the context in which they're called may not be able to handle the exception).
2019-09-03Ensure that Callback is called only onceEelco Dolstra
Also, make Callback movable but uncopyable.
2019-07-10Revert "Fix 'error 9 while decompressing xz file'"Eelco Dolstra
This reverts commit 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91.
2019-06-24Fix 'error 9 while decompressing xz file'Eelco Dolstra
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952. (cherry picked from commit a67cf5a3585c41dd9f219a2c7aa9cf67fa69520b)
2018-08-21Fix another 'coroutine has finished' during decompressionEelco Dolstra
https://hydra.nixos.org/build/79867739
2018-08-06Fix Brotli decompression in 'nix log'Eelco Dolstra
This didn't work anymore since decompression was only done in the non-coroutine case. Decompressors are now sinks, just like compressors. Also fixed a bug in bzip2 API handling (we have to handle BZ_RUN_OK rather than BZ_OK), which we didn't notice because there was a missing 'throw': if (ret != BZ_OK) CompressionError("error while compressing bzip2 file");
2018-06-05Don't require --fallback to recover from disappeared binary cache NARsEelco Dolstra
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 12:22:46 +0000