aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2017-05-02LocalStoreAccessor: Fix handling of diverted storesEelco Dolstra
2017-05-02Replace $NIX_REMOTE_SYSTEMS with an option "builder-files"Eelco Dolstra
Also, to unify with hydra-queue-runner, allow it to be a list of files.
2017-05-02build-remote: Fix fallback to other machines when connecting failsEelco Dolstra
Opening an SSHStore or LegacySSHStore does not actually establish a connection, so the try/catch block here did nothing. Added a Store::connect() method to test whether a connection can be established.
2017-05-02Add an option for specifying remote buildersEelco Dolstra
This is useful for one-off situations where you want to specify a builder on the command line instead of having to mess with nix.machines. E.g. $ nix-build -A hello --argstr system x86_64-darwin \ --option builders 'root@macstadium1 x86_64-darwin' will perform the specified build on "macstadium1". It also removes the need for a separate nix.machines file since you can specify builders in nix.conf directly. (In fact nix.machines is yet another hack that predates the general nix.conf configuration file, IIRC.) Note: this option is supported by the daemon for trusted users. The fact that this allows trusted users to specify paths to SSH keys to which they don't normally have access is maybe a bit too much trust...
2017-05-02Factor out machines.conf parsingEelco Dolstra
This allows hydra-queue-runner to use it.
2017-05-02build-hook: If there are no machines defined, quit permanentlyEelco Dolstra
2017-05-02build-remote: Ugly hackery to get build logs to workEelco Dolstra
The build hook mechanism expects build log output to go to file descriptor 4, so do that.
2017-05-01build-remote: Don't require signaturesEelco Dolstra
This restores the old behaviour.
2017-05-01Support arbitrary store URIs in nix.machinesEelco Dolstra
For backwards compatibility, if the URI is just a hostname, ssh:// (i.e. LegacySSHStore) is prepended automatically. Also, all fields except the URI are now optional. For example, this is a valid nix.machines file: local?root=/tmp/nix This is useful for testing the remote build machinery since you don't have to mess around with ssh.
2017-05-01Implement LegacySSHStore::buildDerivation()Eelco Dolstra
This makes LegacySSHStore usable by build-remote and hydra-queue-runner.
2017-05-01Chomp log output from the build hookEelco Dolstra
2017-05-01Remove $NIX_BUILD_HOOK and $NIX_CURRENT_LOADEelco Dolstra
This is to simplify remote build configuration. These environment variables predate nix.conf. The build hook now has a sensible default (namely build-remote). The current load is kept in the Nix state directory now.
2017-05-01build-remote: Don't copy the .drv closureEelco Dolstra
Since build-remote uses buildDerivation() now, we don't need to copy the .drv file anymore. This greatly reduces the set of input paths copied to the remote side (e.g. from 392 to 51 store paths for GNU hello on x86_64-darwin).
2017-05-01Pass verbosity level to build hookEelco Dolstra
2017-05-01Reduce severity of EMLINK warningsEelco Dolstra
Fixes #1357.
2017-05-01Add a dummy Store::buildPaths() methodEelco Dolstra
This default implementation of buildPaths() does nothing if all requested paths are already valid, and throws an "unsupported operation" error otherwise. This fixes a regression introduced by c30330df6f67c81986dfb124631bc756c8e58c0d in binary cache and legacy SSH stores.
2017-04-28Fix brainfartEelco Dolstra
2017-04-26Add Store nesting to fix import-from-derivation within filterSourceShea Levy
2017-04-24Remove debug statementEelco Dolstra
2017-04-20Detect lsofEelco Dolstra
Also, don't use lsof on Linux since it's not needed. Fixes #1328.
2017-04-20Improve nix show-config --jsonEelco Dolstra
In particular, show descriptions. This could be used for manpage generation etc.
2017-04-20Setting: Remove "Tag" template argumentEelco Dolstra
2017-04-20Read per-user settings from ~/.config/nix/nix.confEelco Dolstra
2017-04-20binary-caches-parallel-connections -> http-connectionsEelco Dolstra
2017-04-20Reimplement trusted-substituters (aka trusted-binary-caches)Eelco Dolstra
2017-04-19Reimplement connect-timeoutEelco Dolstra
Fixes #1339.
2017-04-14Shut up some warningsEelco Dolstra
2017-04-14DohEelco Dolstra
2017-04-13Convert Settings to the new config systemEelco Dolstra
This makes all config options self-documenting. Unknown or unparseable config settings and --option flags now cause a warning.
2017-04-13Merge branch 'rework-options' of https://github.com/copumpkin/nixEelco Dolstra
2017-04-13Add a Config class to simplify adding configuration settingsEelco Dolstra
The typical use is to inherit Config and add Setting<T> members: class MyClass : private Config { Setting<int> foo{this, 123, "foo", "the number of foos to use"}; Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"}; MyClass() : Config(readConfigFile("/etc/my-app.conf")) { std::cout << foo << "\n"; // will print 123 unless overriden } }; Currently, this is used by Store and its subclasses for store parameters. You now get a warning if you specify a non-existant store parameter in a store URI.
2017-04-10Merge pull request #1316 from copumpkin/nix-retries-defaultEelco Dolstra
Default to 5 download retries
2017-04-10Add CURLE_WRITE_ERROR as a transient error conditionDan Peebles
We've observed it failing downloads in the wild and retrying the same URL a few moments later seemed to fix it.
2017-04-10Default to 5 download retriesDan Peebles
This should help certain downloaders that don't request anything special for the number of retries, like nix-channel.
2017-04-10Minor cleanupEelco Dolstra
Also, possible fix for #1310 on 32-bit systems.
2017-04-10Allow "auto" as a store URIEelco Dolstra
Using the empty string is likely to be ambiguous in some contexts.
2017-04-06Retry downloads on transient SSL errors tooDan Peebles
2017-04-06Implement RemoteStore::queryMissing()Eelco Dolstra
This provides a significant speedup, e.g. 64 s -> 12 s for nix-build --dry-run -I nixpkgs=channel:nixos-16.03 '<nixpkgs/nixos/tests/misc.nix>' -A test on a cold local and CloudFront cache. The alternative is to use lots of concurrent daemon connections but that seems wasteful.
2017-04-06Add a method to allow hydra-queue-runner to flush the path info cacheEelco Dolstra
2017-04-06nix-daemon: Disable path info cacheEelco Dolstra
This is useless because the client also caches path info, and can cause problems for long-running clients like hydra-queue-runner (i.e. it may return cached info about paths that have been garbage-collected).
2017-04-04Allow default sandbox paths to be overridenEelco Dolstra
E.g. you can now redirect /etc/resolv.conf to a different file.
2017-04-04Make /var/run/nscd/socket optionalEelco Dolstra
Not every distribution uses nscd.
2017-03-31Sandbox: Fix /dev/ptmx on recent kernelsEelco Dolstra
This fixes "No such file or directory" when opening /dev/ptmx (e.g. http://hydra.nixos.org/build/51094249). The reason appears to be some changes to /dev/ptmx / /dev/pts handling between Linux 4.4 and 4.9. See https://patchwork.kernel.org/patch/7832531/. The fix is to go back to mounting a proper /dev/pts instance inside the sandbox. Happily, this now works inside user namespaces, even for unprivileged users. So NIX_REMOTE=local?root=/tmp/nix nix-build \ '<nixpkgs/nixos/tests/misc.nix>' -A test works for non-root users. The downside is that the fix breaks sandbox builds on older kernels (probably pre-4.6), since mounting a devpts fails inside user namespaces for some reason I've never been able to figure out. Builds on those systems will fail with error: while setting up the build environment: mounting /dev/pts: Invalid argument Ah well.
2017-03-31Merge branch 'builtins.exec'Shea Levy
2017-03-31Retry curl error 16Eelco Dolstra
2017-03-30Add exec primop behind allow-unsafe-native-code-during-evaluation.Shea Levy
Execute a given program with the (optional) given arguments as the user running the evaluation, parsing stdout as an expression to be evaluated. There are many use cases for nix that would benefit from being able to run arbitrary code during evaluation, including but not limited to: * Automatic git fetching to get a sha256 from a git revision * git rev-parse HEAD * Automatic extraction of information from build specifications from other tools, particularly language-specific package managers like cabal or npm * Secrets decryption (e.g. with nixops) * Private repository fetching Ideally, we would add this functionality in a more principled way to nix, but in the mean time 'builtins.exec' can be used to get these tasks done. The primop is only available when the 'allow-unsafe-native-code-during-evaluation' nix option is true. That flag also enables the 'importNative' primop, which is strictly more powerful but less convenient (since it requires compiling a plugin against the running version of nix).
2017-03-28Ignore broken "Deriver: unknown-deriver" fields in .narinfoEelco Dolstra
These were generated by a legacy tool.
2017-03-21LocalFSStore::getBuildLog(): Handle corrupted logsEelco Dolstra
2017-03-21Only use cache.nixos.org when the store is /nix/storeEelco Dolstra
This is consistent with the behaviour of the old download-from-binary-cache substituter.
2017-03-21Require signatures by defaultEelco Dolstra
This corresponds to the NixOS default.