aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/http-binary-cache-store.cc
AgeCommit message (Collapse)Author
2017-05-16Improve progress indicatorEelco Dolstra
2017-03-15S3BinaryCacheStore: Set Content-TypeEelco Dolstra
This is necessary for serving log files to browsers.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-10-19Shut up clang warningEelco Dolstra
2016-09-20Fix "Promise already satisfied" errorEelco Dolstra
2016-09-16Make computeFSClosure() single-threaded againEelco Dolstra
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-14Enable HTTP/2 supportEelco Dolstra
The binary cache store can now use HTTP/2 to do lookups. This is much more efficient than HTTP/1.1 due to multiplexing: we can issue many requests in parallel over a single TCP connection. Thus it's no longer necessary to use a bunch of concurrent TCP connections (25 by default). For example, downloading 802 .narinfo files from https://cache.nixos.org/, using a single TCP connection, takes 11.8s with HTTP/1.1, but only 0.61s with HTTP/2. This did require a fairly substantial rewrite of the Downloader class to use the curl multi interface, because otherwise curl wouldn't be able to do multiplexing for us. As a bonus, we get connection reuse even with HTTP/1.1. All downloads are now handled by a single worker thread. Clients call Downloader::enqueueDownload() to tell the worker thread to start the download, getting a std::future to the result.
2016-08-30Increase the sleep time between download retriesEelco Dolstra
2016-08-10HttpBinaryCacheStore: Retry on transient HTTP errorsEelco Dolstra
This makes us more robust against 500 errors from CloudFront or S3 (assuming the 500 error isn't cached by CloudFront...).
2016-06-01HttpBinaryCacheStore: Fix caching of WantMassQueryEelco Dolstra
Also, test HttpBinaryCacheStore in addition to LocalBinaryCacheStore.
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-05-30Re-implement the WantMassQuery property of binary cachesEelco Dolstra
2016-05-30BinaryCacheStore: Remove buildPaths() / ensurePath()Eelco Dolstra
2016-04-29BinaryCacheStore: Make the signing key a parameterEelco Dolstra
2016-04-29Allow parameters in store URIsEelco Dolstra
This is to allow store-specific configuration, e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
2016-04-20Cache path info lookups in SQLiteEelco Dolstra
This re-implements the binary cache database in C++, allowing it to be used by other Store backends, in particular the S3 backend.
2016-04-15BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-03-30HttpBinaryCacheStore: Treat 403 errors as 404Eelco Dolstra
2016-03-24HttpBinaryCacheStore: Make thread-safeEelco Dolstra
2016-03-15Fix Darwin buildEelco Dolstra
http://hydra.nixos.org/build/33279996
2016-03-04Add option binary-cache-secret-key-file for signing binary cachesEelco Dolstra
2016-03-04BinaryCacheStore: Remove publicKeyFile argumentEelco Dolstra
The public key can be derived from the secret key, so there's no need for the user to supply it separately.
2016-03-03Add file missing from 201b48de60751979835037a4b4f78128ba3fb7b3Eelco Dolstra