aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2017-09-08Communicate the structured log FD to buildersEelco Dolstra
Since we may use a dedicated file descriptor in the future, this allows us to change it. So builders can do if [[ -n $NIX_LOG_FD ]]; then echo "@nix { message... }" >&$NIX_LOG_FD fi
2017-09-06Fix abort when the GC thread gets an exceptionEelco Dolstra
2017-09-05Add automatic garbage collectionEelco Dolstra
Nix can now automatically run the garbage collector during builds or while adding paths to the store. The option "min-free = <bytes>" specifies that Nix should run the garbage collector whenever free space in the Nix store drops below <bytes>. It will then delete garbage until "max-free" bytes are available. Garbage collection during builds is asynchronous; running builds are not paused and new builds are not blocked. However, there also is a synchronous GC run prior to the first build/substitution. Currently, no old GC roots are deleted (as in "nix-collect-garbage -d").
2017-09-05GC: Don't delete own temproots fileEelco Dolstra
Since file locks are per-process rather than per-file-descriptor, the garbage collector would always acquire a lock on its own temproots file and conclude that it's stale.
2017-09-01Abort curl downloads if there is no progress for 5 minutesEelco Dolstra
Maybe this will fix the curl hangs on macOS. (We could also use CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
2017-08-31Call queryMissing() prior to buildingEelco Dolstra
Without this, substitute info is fetched sequentially, which is superslow. In the old UI (e.g. nix-build), we call printMissing(), which calls queryMissing(), thereby preheating the binary cache cache. But the new UI doesn't do that.
2017-08-31Add an activity for binary cache queriesEelco Dolstra
2017-08-31Rename a few configuration optionsEelco Dolstra
In particular, drop the "build-" and "gc-" prefixes which are pointless. So now you can say nix build --no-sandbox instead of nix build --no-build-use-sandbox
2017-08-28Give activities a verbosity level againEelco Dolstra
And print them (separately from the progress bar) given sufficient -v flags.
2017-08-28Don't send progress messages to older clientsEelco Dolstra
2017-08-28Tunnel progress messages from the daemon to the clientEelco Dolstra
This makes the progress bar work for non-root users.
2017-08-25SimplifyEelco Dolstra
2017-08-25Allow derivations to update the build phaseEelco Dolstra
So the progress bar can show [1/0/1 built, 0.0 MiB DL] building hello-2.10 (configuring): checking whether pread is declared without a macro... yes
2017-08-25Allow activities to be nestedEelco Dolstra
In particular, this allows more relevant activities ("substituting X") to supersede inferior ones ("downloading X").
2017-08-25Restore activity metadataEelco Dolstra
This allows the progress bar to display "building perl-5.22.3" instead of "building /nix/store/<hash>-perl-5.22.3.drv".
2017-08-21Clean up JSON constructionEelco Dolstra
2017-08-21Allow builders to create activitiesEelco Dolstra
Actually, currently they can only create download activities. Thus, downloads by builtins.fetchurl show up in the progress bar.
2017-08-16nix optimise-store: Show how much space has been freedEelco Dolstra
2017-08-16nix optimise-store: AddEelco Dolstra
This replaces "nix-store --optimise". Main difference is that it has a progress indicator.
2017-08-16Progress indicator: CleanupEelco Dolstra
2017-08-16Progress indicator: More improvementsEelco Dolstra
2017-08-16Progress indicator: Show number of active itemsEelco Dolstra
2017-08-16Progress indicator: Unify "copying" and "substituting"Eelco Dolstra
They're the same thing after all. Example: $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped [0/1 built, 49/98 copied, 16.3/92.8 MiB DL, 55.8/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/0pl9li1jigcj2dany47hpmn0r3r48wc4nz48v5mqhh426lgz3bz6.nar.xz'
2017-08-16Improve substitution progress indicatorEelco Dolstra
E.g. $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped [0/1 built, 1/97/98 fetched, 65.8/92.8 MiB DL, 203.2/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/1czm9fk0svacy4h6a3fzkpafi4f7a9gml36kk8cq1igaghbspg3k.nar.xz'
2017-08-16nix copy: Improve progress indicatorEelco Dolstra
It now shows the amount of data copied: [8/1038 copied, 160.4/1590.9 MiB copied] copying path '...'
2017-08-16nix copy: Revive progress barEelco Dolstra
2017-08-16Disallow SSH host names starting with a dashEelco Dolstra
2017-08-10Style fixEelco Dolstra
2017-08-09Use /proc/self/fd to efficiently close all FDs on LinuxEelco Dolstra
Issue #1506.
2017-07-31Merge branch 'macOS' of https://github.com/davidak/nixEelco Dolstra
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30replace "OS X" with "macOS"davidak
2017-07-20FD_SETSIZE check: BuildError -> ErrorEelco Dolstra
BuildError denotes a permanent build failure, which is not the case here.
2017-07-18Do not try to fill fd_set with fd>=FD_SETSIZEDmitry Kalinkin
This is UB and causes buffer overflow and crash on linux.
2017-07-17Always use base-16 for hashed mirror lookupsEelco Dolstra
In particular, don't use base-64, which we don't support. (We do have base-32 redirects for hysterical reasons.) Also, add a test for the hashed mirror feature.
2017-07-17Make the hashes mirrors used by builtins.fetchurl configurableEelco Dolstra
In particular, this allows it to be disabled in our tests.
2017-07-17Fix accidental printErrorEelco Dolstra
2017-07-14nix path-info: Show download sizes for binary cache storesEelco Dolstra
E.g. $ nix path-info --json --store https://cache.nixos.org nixpkgs.thunderbird -S ... "downloadHash": "sha256:1jlixpzi225wwa0f4xdrwrqgi47ip1qpj9p06fyxxg07sfmyi4q0", "downloadSize": 43047620, "closureDownloadSize": 84745960 } ]
2017-07-14nix path-info: Don't barf on invalid pathsEelco Dolstra
Now you get [ { "path": "/nix/store/fzvliz4j5xzvnd0w5zgw2l0ksqh578yk-bla", "valid": false } ]
2017-07-11fetchTarball: Prevent concurrent downloads of the same fileEelco Dolstra
Fixes #849.
2017-07-07Merge pull request #1445 from matthewbauer/macos-skip-hardlinkEelco Dolstra
Don’t hardlink disallowed paths in OS X.
2017-07-06Don’t hardlink disallowed paths in OS X.Matthew Bauer
Fixes #1443
2017-07-04Add X32 to the seccomp filterEelco Dolstra
Fixes #1432.
2017-07-04Sort substituters by priorityEelco Dolstra
Fixes #1438.
2017-07-04getDefaultSubstituters(): Simplify initialisationEelco Dolstra
As shlevy pointed out, static variables in C++11 have thread-safe initialisation.
2017-07-04Add allow-new-privileges optionEelco Dolstra
This allows builds to call setuid binaries. This was previously possible until we started using seccomp. Turns out that seccomp by default disallows processes from acquiring new privileges. Generally, any use of setuid binaries (except those created by the builder itself) is by definition impure, but some people were relying on this ability for certain tests. Example: $ nix build '(with import <nixpkgs> {}; runCommand "foo" {} "/run/wrappers/bin/ping -c 1 8.8.8.8; exit 1")' --no-allow-new-privileges builder for ‘/nix/store/j0nd8kv85hd6r4kxgnwzvr0k65ykf6fv-foo.drv’ failed with exit code 1; last 2 log lines: cannot raise the capability into the Ambient set : Operation not permitted $ nix build '(with import <nixpkgs> {}; runCommand "foo" {} "/run/wrappers/bin/ping -c 1 8.8.8.8; exit 1")' --allow-new-privileges builder for ‘/nix/store/j0nd8kv85hd6r4kxgnwzvr0k65ykf6fv-foo.drv’ failed with exit code 1; last 6 log lines: PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=46 time=15.2 ms Fixes #1429.
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-07-03Replace a few bool flags with enumsEelco Dolstra
Functions like copyClosure() had 3 bool arguments, which creates a severe risk of mixing up arguments. Also, implement copyClosure() using copyPaths().
2017-06-20Call SetDllDirectory("") after sqlite3 init on cygwinDavid McFarland
Cygwin sqlite3 is patched to call SetDllDirectory("/usr/bin") on init, which affects the current process and is inherited by child processes. It causes DLLs to be loaded from /usr/bin/ before $PATH, which breaks all sorts of things. A typical failures would be header/lib version mismatches (e.g. openssl when running checkPhase on openssh). We'll just set it back to the default value. Note that this is a problem with the cygwin version of sqlite3 (currently 3.18.0). nixpkgs doesn't have the problematic patch.
2017-06-19Disable use of virtual hosting in aws-sdk-cppEelco Dolstra
Recently aws-sdk-cpp quietly switched to using S3 virtual host URIs (https://github.com/aws/aws-sdk-cpp/commit/69d9c53882), i.e. it sends requests to http://<bucket>.<region>.s3.amazonaws.com rather than http://<region>.s3.amazonaws.com/<bucket>. However this interacts badly with curl connection reuse. For example, if we do the following: 1) Check whether a bucket exists using GetBucketLocation. 2) If it doesn't, create it using CreateBucket. 3) Do operations on the bucket. then 3) will fail for a minute or so with a NoSuchBucket exception, presumably because the server being hit is a fallback for cases when buckets don't exist. Disabling the use of virtual hosts ensures that 3) succeeds immediately. (I don't know what S3's consistency guarantees are for bucket creation, but in practice buckets appear to be available immediately.)