aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2019-09-22Don't catch exceptions by valueEelco Dolstra
(cherry picked from commit 893be6f5e36abb58bbaa9c49055a5218114dd514)
2019-09-13std::uncaught_exception() -> std::uncaught_exceptions()Eelco Dolstra
The former is deprecated in C++17. Fixes a clang warning.
2019-09-04BinaryCacheStore: Add index-debug-info optionEelco Dolstra
This integrates the functionality of the index-debuginfo program in nixos-channel-scripts to maintain an index of DWARF debuginfo files in a format usable by dwarffs. Thus the debug info index is updated by Hydra rather than by the channel mirroring script. Example usage: $ nix copy --to 'file:///tmp/binary-cache?index-debug-info=true' /nix/store/vr9mhcch3fljzzkjld3kvkggvpq38cva-nix-2.2.2-debug $ cat /tmp/binary-cache/debuginfo/036b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug {"archive":"../nar/0313h2kdhk4v73xna9ysiksp2v8xrsk5xsw79mmwr3rg7byb4ka8.nar.xz","member":"lib/debug/.build-id/03/6b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug"} Fixes #3083.
2019-09-03Support allowSubstitutes attribute in structured attribute derivationsEelco Dolstra
Hopefully fixes #3081 (didn't test).
2019-09-03Add some noexceptsEelco Dolstra
This is to assert that callback functions should never throw (since the context in which they're called may not be able to handle the exception).
2019-09-03Ensure that Callback is called only onceEelco Dolstra
Also, make Callback movable but uncopyable.
2019-09-03Downloader: Remove a possible double call to CallbackEelco Dolstra
2019-08-29Merge pull request #3069 from matthewbauer/max-nameEelco Dolstra
Set maximum name length in Nix
2019-08-29Don't rely on st_blocksEelco Dolstra
It doesn't seem very reliable on ZFS.
2019-08-29CleanupEelco Dolstra
2019-08-28Set maximum name length in NixMatthew Bauer
Previously we allowed any length of name for Nix derivations. This is bad because different file systems have different max lengths. To make things predictable, I have picked a max. This was done by trying to build this derivation: derivation { name = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; builder = "/no-such-path"; system = "x86_64-linux"; } Take off one a and it will not lead to file name too long. That ends up being 212 a’s. An even smaller max could be picked if we want to support more file systems. Working backwards, this is why: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-${name}.drv.chroot > 255 - 32 - 1 - 4 - 7 = 211
2019-08-28Don't send certain setting overrides to the daemonEelco Dolstra
These are already handled separately. This fixes warnings like warning: ignoring the user-specified setting 'max-jobs', because it is a restricted setting and you are not a trusted user when using the -j flag.
2019-08-27Merge branch 'test-sandboxing' of https://github.com/matthewbauer/nixEelco Dolstra
2019-08-23Reset tmpDirInSandbox for unsandboxedMatthew Bauer
2019-08-08Merge pull request #3031 from grahamc/low-speed-limitEelco Dolstra
conf: stalled-download-timeout: make tunable
2019-08-08conf: stalled-download-timeout: make tunableGraham Christensen
Make curl's low speed limit configurable via stalled-download-timeout. Before, this limit was five minutes without receiving a single byte. This is much too long as if the remote end may not have even acknowledged the HTTP request.
2019-08-07Merge pull request #3030 from dtzWill/fix/missing-include-ocloexecEelco Dolstra
pathlocks: add include to fcntl.h for O_CLOEXEC
2019-08-07Merge pull request #2995 from tweag/post-build-hookEelco Dolstra
Add a post build hook
2019-08-07pathlocks: add include to fcntl.h for O_CLOEXECWill Dietz
2019-08-02nix-store --verify: Don't repair while holding the GC lockEelco Dolstra
2019-08-02SimplifyEelco Dolstra
With BSD locks we don't have to guard against reading our own temproots.
2019-08-02Use BSD instead of POSIX file locksEelco Dolstra
POSIX file locks are essentially incompatible with multithreading. BSD locks have much saner semantics. We need this now that there can be multiple concurrent LocalStore::buildPaths() invocations.
2019-08-02Add a test for auto-GCEelco Dolstra
This currently fails because we're using POSIX file locks. So when the garbage collector opens and closes its own temproots file, it causes the lock to be released and then deleted by another GC instance.
2019-08-02Add a post-build-hookregnat
Passing `--post-build-hook /foo/bar` to a nix-* command will cause `/foo/bar` to be executed after each build with the following environment variables set: DRV_PATH=/nix/store/drv-that-has-been-built.drv OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev This can be useful in particular to upload all the builded artifacts to the cache (including the ones that don't appear in the runtime closure of the final derivation or are built because of IFD). This new feature prints the stderr/stdout output to the `nix-build` and `nix build` client, and the output is printed in a Nix 2 compatible format: [nix]$ ./inst/bin/nix-build ./test.nix these derivations will be built: /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'... hello! bye! running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'... post-build-hook: + sleep 1 post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: + sleep 1 post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: + sleep 1 post-build-hook: + printf 'very important stuff' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix my-example-derivation> hello! my-example-derivation> bye! my-example-derivation (post)> + sleep 1 my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> + sleep 1 my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> + sleep 1 my-example-derivation (post)> + printf 'very important stuff' [1 built, 0.0 MiB DL] Co-authored-by: Graham Christensen <graham@grahamc.com> Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-07-30Don’t rely on EPERMMatthew Bauer
startProcess does not appear to send the exit code to the helper correctly. Not sure why this is, but it is probably safe to just fallback on all sandbox errors.
2019-07-30Disable findRuntimeRoots on darwin when running tests because lsof is slowBas van Dijk
See: https://github.com/NixOS/nix/issues/3011
2019-07-25Use sandbox fallback when cloning fails in builderMatthew Bauer
When sandbox-fallback = true (the default), the Nix builder will fall back to disabled sandbox mode when the kernel doesn’t allow users to set it up. This prevents hard errors from occuring in tricky places, especially the initial installer. To restore the previous behavior, users can set: sandbox-fallback = false in their /etc/nix/nix.conf configuration.
2019-07-25Disable CLONE_NEWUSER when it’s unavailableMatthew Bauer
Some kernels disable "unpriveleged user namespaces". This is unfortunate, but we can still use mount namespaces. Anyway, since each builder has its own nixbld user, we already have most of the benefits of user namespaces.
2019-07-13Merge pull request #2975 from matthewbauer/fix-nsswitch-issueEelco Dolstra
Don’t use entire /etc/nsswitch.conf file
2019-07-10Resume NAR downloadsEelco Dolstra
This is a much simpler fix to the 'error 9 while decompressing xz file' problem than 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91. We just do a ranged HTTP request starting after the data that we previously wrote into the sink. Fixes #2952, #379.
2019-07-10HttpBinaryCacheStore: Use default number of retries for NARsEelco Dolstra
2019-07-10Downloader: Use warn()Eelco Dolstra
2019-07-10Revert "Fix 'error 9 while decompressing xz file'"Eelco Dolstra
This reverts commit 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91.
2019-07-02Merge pull request #2779 from LnL7/build-exit-codesEelco Dolstra
build: add exit code for hash and check mismatches
2019-07-02Merge pull request #2974 from grahamc/invalid-nameEelco Dolstra
checkStoreName: give more precise/verbose error information
2019-07-02checkStoreName: give more precise/verbose error informationGraham Christensen
$ sudo ./inst/bin/nix-instantiate -E '"${./.git}"' error: The path name '.git' is invalid: it is illegal to start the name with a period. Path names are alphanumeric and can include the symbols +-._?= and must not begin with a period. Note: If '.git' is a source file and you cannot rename it on disk, builtins.path { name = ... } can be used to give it an alternative name.
2019-07-02build: replace 100 offset for build exit codesDaiderd Jordan
2019-07-02build: use binary mask for build status flagsDaiderd Jordan
If multiple builds with fail with different errors it will be reflected in the status code. eg. 103 => timeout + hash mismatch 105 => timeout + check mismatch 106 => hash mismatch + check mismatch 107 => timeout + hash mismatch + check mismatch
2019-07-02build: add exit code for hash and check mismatchesDaiderd Jordan
Makes it easier to identify the failure reason in other tooling, eg. differentiate between a non-deterministic --check vs a failed build. $ nix-build '<nix/fetchurl.nix>' --argstr url http://example.org --argstr sha256 0000000000000000000000000000000000000000000000000000 hash mismatch in fixed-output derivation '/nix/store/nzi9ck45rwlxzcwr25is7qlf3hs5xl83-example.org': wanted: sha256:0000000000000000000000000000000000000000000000000000 got: sha256:08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm $ echo $? 102 $ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" {} "date +%s > $out"' --check warning: rewriting hashes in '/nix/store/g3k47g0399fvjmbm0p0mnad74k4w8vkz-foo'; cross fingers error: derivation '/nix/store/mggc8dz13ackb49qca6m23zq4fpq132q-foo.drv' may not be deterministic: output '/nix/store/g3k47g0399fvjmbm0p0mnad74k4w8vkz-foo' differs $ echo $? 104
2019-06-28Fix `http2 = false` having no effect. Fixes #2971.Aniket Deshpande
Setting `http2 = false` in nix config (e.g. /etc/nix/nix.conf) had no effect, and `nix-env -vvvvv -i hello` still downloaded .nar packages using HTTP/2. In `src/libstore/download.cc`, the `CURL_HTTP_VERSION_2TLS` option was being explicitly set when `downloadSettings.enableHttp2` was `true`, but, `CURL_HTTP_VERSION_1_1` option was not being explicitly set when `downloadSettings.enableHttp2` was `false`. This may be because `https://curl.haxx.se/libcurl/c/libcurl-env.html` states: "You have to set this option if you want to use libcurl's HTTP/2 support." but, also, in the changelog, states: "DEFAULT Since curl 7.62.0: CURL_HTTP_VERSION_2TLS Before that: CURL_HTTP_VERSION_1_1" So, the default setting for `libcurl` is HTTP/2 for version >= 7.62.0. In this commit, option `CURLOPT_HTTP_VERSION` is explicitly set to `CURL_HTTP_VERSION_1_1` when `downloadSettings.enableHttp2` nix config setting is `false`. This can be tested by running `nix-env -vvvvv -i hello | grep HTTP`
2019-06-27Don’t use entire /etc/nsswitch.conf fileMatthew Bauer
The default nsswitch.conf(5) file in most distros can handle many different things including host name, user names, groups, etc. In Nix, we want to limit the amount of impurities that come from these things. As a result, we should only allow nss to be used for gethostbyname(3) and getservent(3). /cc @Ericson2314
2019-06-25nix-channel: Don't fetch binary-cache-urlEelco Dolstra
This has been ignored since the Perl->C++ rewrite.
2019-06-24Fix 32-bit overflow with --no-netEelco Dolstra
--no-net causes tarballTtl to be set to the largest 32-bit integer, which causes comparison like 'time + tarballTtl < other_time' to fail on 32-bit systems. So cast them to 64-bit first. https://hydra.nixos.org/build/95076624 (cherry picked from commit 29ccb2e9697ee2184012dd13854e487928ae4441)
2019-06-24Refactor downloadCached() interfaceEelco Dolstra
(cherry picked from commit df3f5a78d5ab0a1f2dc9d288b271b38a9b8b33b5)
2019-06-24Add '--no-net' convenience flagEelco Dolstra
This flag * Disables substituters. * Sets the tarball-ttl to infinity (ensuring e.g. that the flake registry and any downloaded flakes are considered current). * Disables retrying downloads and sets the connection timeout to the minimum. (So it doesn't completely disable downloads at the moment.) (cherry picked from commit 8ea842260b4fd93315d35c5ba94b1ff99ab391d8)
2019-06-24Fix 'error 9 while decompressing xz file'Eelco Dolstra
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952. (cherry picked from commit a67cf5a3585c41dd9f219a2c7aa9cf67fa69520b)
2019-06-24Downloader: Propagate exceptions from decompressionSink->finish()Eelco Dolstra
(cherry picked from commit 15fa70cd1b853f5e62662b99ccb9ef3da6cfadff)
2019-06-24CachedDownloadResult: Include store pathEelco Dolstra
Also, make fetchGit and fetchMercurial update allowedPaths properly. (Maybe the evaluator, rather than the caller of the evaluator, should apply toRealPath(), but that's a bigger change.) (cherry picked from commit 5c34d665386f4053d666b0899ecca0639e500fbd)
2019-06-24downloadCached: Return ETagEelco Dolstra
(cherry picked from commit 529add316c5356a8060c35f987643b7bf5c796dc)
2019-06-17Another attempt at getting pseudoterminals to work on macOSEelco Dolstra
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 00:52:55 +0000