aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/http-binary-cache-store.cc
AgeCommit message (Collapse)Author
2023-02-07NarInfoDiskCache: Rename cacheExists -> upToDateCacheExistsRobert Hensing
This is slightly more accurate considering that an outdated record may exist in the persistent cache. Possibly-outdated records are quite relevant as they may be foreign keys to more recent information that we want to keep, but we will not return them here.
2022-05-30HttpBinaryCacheStore::getFile(): Don't throw an exceptionEelco Dolstra
This violates the noexcept specification. Fixes #6445.
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2021-09-22Make setDefault() typedEelco Dolstra
2020-12-20Overhaul store subclassingJohn Ericson
We embrace virtual the rest of the way, and get rid of the `assert(false)` 0-param constructors. We also list config base classes first, so the constructor order is always: 1. all the configs 2. all the stores Each in the same order
2020-10-06Remove static variable name clashesEelco Dolstra
This was useful for an experiment with building Nix as a single compilation unit. It's not very useful otherwise but also doesn't hurt...
2020-10-05Fix s3:// storeEelco Dolstra
Fixes https://github.com/NixOS/nixos-org-configurations/issues/123.
2020-09-21Move Callback into its own headerEelco Dolstra
This gets rid of the inclusion of <future> in util.hh, cutting compilation time by ~20s (CPU time). Issue #4045.
2020-09-16Add a name to the storesregnat
So that it can be printed by `nix describe-stores`
2020-09-16Document the new store hierarchyregnat
2020-09-16Properly filter the stores according to their declared uriSchemesregnat
When opening a store, only try the stores whose `uriSchemes()` include the current one
2020-09-16Correctly call all the parent contructors of the storesregnat
Using virtual inheritance means that only the default constructors of the parent classes will be called, which isn't what we want
2020-09-16Separate store configs from the implemsregnat
Rework the `Store` hierarchy so that there's now one hierarchy for the store configs and one for the implementations (where each implementation extends the corresponding config). So a class hierarchy like ``` StoreConfig-------->Store | | v v SubStoreConfig----->SubStore | | v v SubSubStoreConfig-->SubSubStore ``` (with virtual inheritance to prevent DDD). The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
2020-09-16Separate the instantiation and initialisation of the storesregnat
Add a new `init()` method to the `Store` class that is supposed to handle all the effectful initialisation needed to set-up the store. The constructor should remain side-effect free and just initialize the c++ data structure. The goal behind that is that we can create “dummy” instances of each store to query static properties about it (the parameters it accepts for example)
2020-09-16Fix the registration of storesregnat
2020-09-16Make the store plugins more introspectableregnat
Directly register the store classes rather than a function to build an instance of them. This gives the possibility to introspect static members of the class or choose different ways of instantiating them.
2020-09-01Allow HTTP binary cache to request absolute urisDomen Kožar
2020-07-13Make 'nix copy' to s3:// binary caches run in constant memoryEelco Dolstra
2020-07-13Make 'nix copy' to file:// binary caches run in constant memoryEelco Dolstra
2020-04-08datatransfer.{cc,hh} -> filetransfer.{cc,hh}Nikola Knezevic
2020-04-08DataTransfer -> FileTransferNikola Knezevic
2020-04-08Add upload methodNikola Knezevic
2020-04-08DownloadError -> DataTransferErrorNikola Knezevic
2020-04-08{get,make,new}Downloader -> DataTransferNikola Knezevic
2020-04-08enqueueDownload -> enqueueDataTransferNikola Knezevic
2020-04-08DownloadResult -> DataTransferResultNikola Knezevic
2020-04-08Rename src/lib/download.* to src/lib/datatransfer.*Nikola Knezevic
2020-04-08DownloadRequest -> DataTransferRequestNikola Knezevic
2020-03-24Misc changes from the flakes branchEelco Dolstra
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-09-03Add some noexceptsEelco Dolstra
This is to assert that callback functions should never throw (since the context in which they're called may not be able to handle the exception).
2019-09-03Ensure that Callback is called only onceEelco Dolstra
Also, make Callback movable but uncopyable.
2019-07-10HttpBinaryCacheStore: Use default number of retries for NARsEelco Dolstra
2019-07-10Revert "Fix 'error 9 while decompressing xz file'"Eelco Dolstra
This reverts commit 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91.
2019-06-24Add '--no-net' convenience flagEelco Dolstra
This flag * Disables substituters. * Sets the tarball-ttl to infinity (ensuring e.g. that the flake registry and any downloaded flakes are considered current). * Disables retrying downloads and sets the connection timeout to the minimum. (So it doesn't completely disable downloads at the moment.) (cherry picked from commit 8ea842260b4fd93315d35c5ba94b1ff99ab391d8)
2019-06-24Fix 'error 9 while decompressing xz file'Eelco Dolstra
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952. (cherry picked from commit a67cf5a3585c41dd9f219a2c7aa9cf67fa69520b)
2018-09-07If fallback is enabled, temporarily disable substituters after a failureEelco Dolstra
Otherwise, we just keep asking the substituter for other .narinfo files, which can take a very long time due to retries/timeouts.
2018-06-01Improve binary cache upload messagesEelco Dolstra
Don't say "download" when we mean "upload".
2018-05-30Make HttpBinaryCacheStore::narFromPath() run in constant memoryEelco Dolstra
This reduces memory consumption of nix copy --from https://cache.nixos.org --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79 from 176 MiB to 82 MiB. (The remaining memory is probably due to xz decompression overhead.) Issue https://github.com/NixOS/nix/issues/1681. Issue https://github.com/NixOS/nix/issues/1969.
2018-05-30Simplify the callback mechanismEelco Dolstra
2018-01-31CleanupEelco Dolstra
2018-01-26HttpBinaryCacheStore: Support upsertFile with PUT.Shea Levy
Some servers, such as Artifactory, allow uploading with PUT and BASIC auth. This allows nix copy to work to upload binaries to those servers. Worked on together with @adelbertc
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-05-16Improve progress indicatorEelco Dolstra
2017-03-15S3BinaryCacheStore: Set Content-TypeEelco Dolstra
This is necessary for serving log files to browsers.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-10-19Shut up clang warningEelco Dolstra
2016-09-20Fix "Promise already satisfied" errorEelco Dolstra
2016-09-16Make computeFSClosure() single-threaded againEelco Dolstra
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)