aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/s3-binary-cache-store.cc
AgeCommit message (Collapse)Author
2024-06-01chore: rebrand Nix to Lix when it makes senseRaito Bezarius
Here's my guide so far: $ rg '((?!(recursive).*) Nix (?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))' -g '!doc/' --pcre2 All items from this query have been tackled. For the documentation side: that's for https://git.lix.systems/lix-project/lix/issues/162. Additionally, all remaining references to github.com/NixOS/nix which were not relevant were also replaced. Fixes: https://git.lix.systems/lix-project/lix/issues/148. Fixes: https://git.lix.systems/lix-project/lix/issues/162. Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-17Delete hasPrefix and hasSuffix from the codebaseJade Lovelace
These now have equivalents in the standard lib in C++20. This change was performed with a custom clang-tidy check which I will submit later. Executed like so: ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
2024-03-04Merge pull request #6258 from obsidiansystems/gcc-bug-ergonomicseldritch horrors
Remove bug-avoiding `StoreConfig *` casts for settings (cherry picked from commit e3febfcd532adb23ca05ac465a2b907d6f1a3529) Change-Id: Ifeae276582fdbc781a38581df9de3da67a7e7bf9
2023-04-06Add `Store::isTrustedClient()`matthewcroughan
This function returns true or false depending on whether the Nix client is trusted or not. Mostly relevant when speaking to a remote store with a daemon. We include this information in `nix ping store` and `nix doctor` Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2023-03-22Improve store setting descriptions / Markdown formattingEelco Dolstra
2023-03-21Support per-store Markdown documentationEelco Dolstra
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2023-02-07NarInfoDiskCache: Rename cacheExists -> upToDateCacheExistsRobert Hensing
This is slightly more accurate considering that an outdated record may exist in the persistent cache. Possibly-outdated records are quite relevant as they may be foreign keys to more recent information that we want to keep, but we will not return them here.
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2021-09-22Make setDefault() typedEelco Dolstra
2021-09-17Fix scheme argument to s3 cacheGregor Kleen
2020-12-20Overhaul store subclassingJohn Ericson
We embrace virtual the rest of the way, and get rid of the `assert(false)` 0-param constructors. We also list config base classes first, so the constructor order is always: 1. all the configs 2. all the stores Each in the same order
2020-12-11treat s3 permission errors as file-not-foundMichael Bishop
Signed-off-by: Jonathan Ringer <jonringer117@gmail.com>
2020-12-04Fix compatibility with newer AWS SDKsStéphan Kochen
Tested against AWS SDK 1.8.99. Fixes #3201.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-10-06Remove static variable name clashesEelco Dolstra
This was useful for an experiment with building Nix as a single compilation unit. It's not very useful otherwise but also doesn't hurt...
2020-09-16Fix the s3 storeregnat
Add some necessary casts in the initialisation of the store's config
2020-09-16Fix build issues with gccregnat
2020-09-16Add a name to the storesregnat
So that it can be printed by `nix describe-stores`
2020-09-16Document the new store hierarchyregnat
2020-09-16Properly filter the stores according to their declared uriSchemesregnat
When opening a store, only try the stores whose `uriSchemes()` include the current one
2020-09-16Correctly call all the parent contructors of the storesregnat
Using virtual inheritance means that only the default constructors of the parent classes will be called, which isn't what we want
2020-09-16Separate store configs from the implemsregnat
Rework the `Store` hierarchy so that there's now one hierarchy for the store configs and one for the implementations (where each implementation extends the corresponding config). So a class hierarchy like ``` StoreConfig-------->Store | | v v SubStoreConfig----->SubStore | | v v SubSubStoreConfig-->SubSubStore ``` (with virtual inheritance to prevent DDD). The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
2020-09-16Separate the instantiation and initialisation of the storesregnat
Add a new `init()` method to the `Store` class that is supposed to handle all the effectful initialisation needed to set-up the store. The constructor should remain side-effect free and just initialize the c++ data structure. The goal behind that is that we can create “dummy” instances of each store to query static properties about it (the parameters it accepts for example)
2020-09-16Fix the registration of storesregnat
2020-09-16Make the store plugins more introspectableregnat
Directly register the store classes rather than a function to build an instance of them. This gives the possibility to introspect static members of the class or choose different ways of instantiating them.
2020-08-04S3BinaryCacheStore: Fix size determinationEelco Dolstra
2020-08-04Revert "Remove putBytes"Eelco Dolstra
This reverts commit b8eea7e81af53905be7845dffc6d0a83ea8edc97.
2020-07-28Remove putBytesEelco Dolstra
istream->tellg() returns -1 so we can't get the number of bytes written. Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.
2020-07-13Make 'nix copy' to s3:// binary caches run in constant memoryEelco Dolstra
2020-07-13Remove istringstream_nocopyEelco Dolstra
2020-07-13Make 'nix copy' to file:// binary caches run in constant memoryEelco Dolstra
2020-07-13Fix 'nix verify --all' on a binary cache and add a testEelco Dolstra
2020-05-11Merge branch 'master' into errors-phase-2Ben Burdette
2020-05-03convert some printError calls to logErrorBen Burdette
2020-04-21remove 'format' from Error constructor callsBen Burdette
2020-04-08datatransfer.{cc,hh} -> filetransfer.{cc,hh}Nikola Knezevic
2020-04-08DataTransfer -> FileTransferNikola Knezevic
2020-04-08DownloadResult -> DataTransferResultNikola Knezevic
2020-04-08Rename src/lib/download.* to src/lib/datatransfer.*Nikola Knezevic
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-03-15Add a 5 seconds of timeouts to connect to S3.Chaker Benhamed
The default is 1000ms, but we can hit it a lot of we don't have direct link to AWS (e.g. using VPN).
2018-12-07s3: make scheme configurableDaiderd Jordan
This enables using for http for S3 request for debugging or implementations that don't have https configured. This is not a problem for binary caches since they should not contain sensitive information. Both package signatures and AWS auth already protect against tampering.
2018-11-01Disable the S3 content-encoding patchEelco Dolstra
Since we're not using multi-part uploads at the moment, we can drop this patch.
2018-10-30Remove redundant messageEelco Dolstra
2018-10-30S3BinaryCacheStore: Allow disabling multipart uploadsEelco Dolstra
The use of TransferManager has several issues, including that it doesn't allow setting a Content-Encoding without a patch, and it doesn't handle exceptions in worker threads (causing termination on memory allocation failure). Fixes #2493.
2018-10-30Fix broken uploadProgressCallback closureEelco Dolstra
Since the callback is global we can't refer to 'path' in it. This could cause a segfault or printing of arbitrary data.
2018-09-27S3BinaryCacheStore: Don't create bucketsEelco Dolstra
This meant that making a typo in an s3:// URI would cause a bucket to be created. Also it didn't handle eventual consistency very well. Now it's up to the user to create the bucket.