aboutsummaryrefslogtreecommitdiff
path: root/src/nix-prefetch-url
AgeCommit message (Collapse)Author
2020-06-18Revert the `enum struct` changeJohn Ericson
Not a regular git revert as there have been many merges and things.
2020-06-18Merge branch 'enum-class' into no-hash-type-unknownJohn Ericson
2020-06-18Merge remote-tracking branch 'upstream/master' into enum-classJohn Ericson
2020-06-11Merge remote-tracking branch 'upstream/master' into errors-phase-2Ben Burdette
2020-06-08Move progress-bar.cc to libmainregnat
Needed so that we can include it as a logger in loggers.cc without adding a dependency on nix This also requires moving names.hh to libutil to prevent a circular dependency between libmain and libexpr
2020-06-02Remove `HashType::Unknown`John Ericson
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may also make `Hash` itself require a known hash type, encoraging people to use `std::optional<Hash>` instead.
2020-05-29Merge remote-tracking branch 'upstream/master' into errors-phase-2Ben Burdette
2020-05-28Merge branch 'master' of github.com:NixOS/nix into enum-classCarlo Nucera
2020-05-26Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethodCarlo Nucera
2020-05-11Merge branch 'master' into errors-phase-2Ben Burdette
2020-04-21remove 'format' from Error constructor callsBen Burdette
2020-04-08datatransfer.{cc,hh} -> filetransfer.{cc,hh}Nikola Knezevic
2020-04-08DataTransfer -> FileTransferNikola Knezevic
2020-04-08{get,make,new}Downloader -> DataTransferNikola Knezevic
2020-04-08Rename src/lib/download.* to src/lib/datatransfer.*Nikola Knezevic
2020-04-08DownloadRequest -> DataTransferRequestNikola Knezevic
2020-03-30Merge remote-tracking branch 'upstream/master' into enum-FileIngestionMethodJohn Ericson
2020-03-30Remove global -I flagsEelco Dolstra
(cherry picked from commit 2c692a3b144523bca68dd6de618124ba6c9bb332)
2020-03-29Replace some `bool recursive` with a new `FileIngestionMethod` enumJohn Ericson
2020-03-29Use `enum struct` and drop prefixesJohn Ericson
This does a few enums; the rest will be gotten in subsequent commits.
2020-03-24findAlongAttrPath(): Return positionEelco Dolstra
(cherry picked from commit 0b013a54dc570395bed887369f8dd622b8ce337b)
2020-01-05build: fix sandboxing on darwinDaiderd Jordan
Starting ba87b08f8529e4d9f8c58d8c625152058ceadb75 getEnv now returns an std::optional which means these getEnv() != "" conditions no longer happen if the variables are not defined.
2019-12-19Merge branch 'libarchive' of https://github.com/yorickvP/nixEelco Dolstra
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-12-07libarchive proof of conceptYorick van Pelt
2019-11-26Drop remaining uses of external "tar"Eelco Dolstra
Also, fetchGit now runs in O(1) memory since we pipe the output of 'git archive' directly into unpackTarball() (rather than first reading it all into memory).
2018-11-09nix-prefetch-url: Stop progress bar before printing resultsEelco Dolstra
2018-10-26nix-prefetch-url: progressbarWill Dietz
2018-10-26Merge all nix-* binaries into nixEelco Dolstra
These are all symlinks to 'nix' now, reducing the installed size by about ~1.7 MiB.
2018-07-12nix-prefetch-url: Download file in constant memoryEelco Dolstra
Before: $ command time nix-prefetch-url https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.6.tar.xz 1.19user 1.02system 0:41.96elapsed 5%CPU (0avgtext+0avgdata 182720maxresident)k After: 1.38user 1.05system 0:39.73elapsed 6%CPU (0avgtext+0avgdata 16204maxresident)k Note however that addToStore() can still take a lot of memory (e.g. RemoteStore::addToStore() is constant space, but LocalStore::addToStore() isn't; that's fixed by https://github.com/edolstra/nix/commit/c94b4fc7ee0c7b322a5f3c7ee784063b47a11d98 though). Fixes #1400.
2018-06-12Move EvalState from the stack to the heapEelco Dolstra
EvalState contains a few counters (e.g. nrValues) that increase quickly enough that they end up being interpreted as pointers by the garbage collector. Moving it to the heap makes them invisible to the garbage collector. This reduces the max RSS doing 100 evaluations of nixos.tests.firefox.x86_64-linux.drvPath from 455 MiB to 292 MiB. Note: ideally, allocations would be much further up in the 64-bit address space to reduce the odds of an integer being misinterpreted as a pointer. Maybe we can use some linker magic to move the .bss segment to a higher address.
2018-02-08Add plugins to make Nix more extensible.Shea Levy
All plugins in plugin-files will be dlopened, allowing them to statically construct instances of the various Register* types Nix supports.
2017-10-24nix: Respect -I, --arg, --argstrEelco Dolstra
Also, random cleanup to argument handling.
2017-09-18Don't remove Content-Encoding in fetchurl / nix-prefetch-urlEelco Dolstra
Fixes #1568.
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-03-15runProgram(): Distinguish between empty input and no inputEelco Dolstra
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-09-21printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra
2016-09-14Enable HTTP/2 supportEelco Dolstra
The binary cache store can now use HTTP/2 to do lookups. This is much more efficient than HTTP/1.1 due to multiplexing: we can issue many requests in parallel over a single TCP connection. Thus it's no longer necessary to use a bunch of concurrent TCP connections (25 by default). For example, downloading 802 .narinfo files from https://cache.nixos.org/, using a single TCP connection, takes 11.8s with HTTP/1.1, but only 0.61s with HTTP/2. This did require a fairly substantial rewrite of the Downloader class to use the curl multi interface, because otherwise curl wouldn't be able to do multiplexing for us. As a bonus, we get connection reuse even with HTTP/1.1. All downloads are now handled by a single worker thread. Clients call Downloader::enqueueDownload() to tell the worker thread to start the download, getting a std::future to the result.
2016-07-26makeFixedOutputPath(): Drop superfluous HashType argumentEelco Dolstra
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-04-15BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-02-29Add an HTTP binary cache storeEelco Dolstra
Allowing stuff like NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 or NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
2016-02-04Eliminate the "store" global variableEelco Dolstra
Also, move a few free-standing functions into StoreAPI and Derivation. Also, introduce a non-nullable smart pointer, ref<T>, which is just a wrapper around std::shared_ptr ensuring that the pointer is never null. (For reference-counted values, this is better than passing a "T&", because the latter doesn't maintain the refcount. Usually, the caller will have a shared_ptr keeping the value alive, but that's not always the case, e.g., when passing a reference to a std::thread via std::bind.)
2015-10-21Disable TLS verification for builtin fetchurlEelco Dolstra
This makes it consistent with the Nixpkgs fetchurl and makes it work in chroots. We don't need verification because the hash of the result is checked anyway.
2015-10-07Show progress during downloadsEelco Dolstra
2015-10-07nix-prefetch-url: Add --name optionEelco Dolstra
This allows overriding the name component of the resulting Nix store path, which is necessary if the base name of the URI contains "illegal" characters.
2015-10-07nix-prefetch-url -A: Use "name" attribute from Nix expressionEelco Dolstra
This is in particular useful for fetchFromGitHub et al., ensuring that the store path produced by nix-prefetch-url corresponds to what those functions expect.