aboutsummaryrefslogtreecommitdiff
path: root/src/build-remote/build-remote.cc
AgeCommit message (Collapse)Author
2020-03-30Remove global -I flagsEelco Dolstra
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-11-13TypoEelco Dolstra
2019-03-06canBuildLocally: check for featuresvolth
It could happen that the local builder match the system but lacks some features. Now it results a failure. The fix gracefully excludes the local builder from the set of available builders for derivation which requires the feature, so the derivation is built on remote builders only (as though it has incompatible system, like ```aarch64-linux``` when local is x86)
2018-10-26Merge all nix-* binaries into nixEelco Dolstra
These are all symlinks to 'nix' now, reducing the installed size by about ~1.7 MiB.
2018-04-23Merge branch 'aarch64-armv7' of git://github.com/lheckemann/nixShea Levy
Support extra compatible architectures (#1916)
2018-03-20Hack to get SSH error messages from build-remoteEelco Dolstra
E.g. cannot build on 'ssh://mac1': cannot connect to 'mac1': bash: nix-store: command not found cannot build on 'ssh://mac2': cannot connect to 'mac2': Host key verification failed. cannot build on 'ssh://mac3': cannot connect to 'mac3': Received disconnect from 213... port 6001:2: Too many authentication failures Authentication failed.
2018-03-16build-remote: take extra-platforms into accountLinus Heckemann
2018-03-13build-remote: Don't substitute the build resultEelco Dolstra
2018-02-13Merge branch 'plugins' of https://github.com/shlevy/nixEelco Dolstra
2018-02-12Fix hang in build-remoteEelco Dolstra
2018-02-08Add plugins to make Nix more extensible.Shea Levy
All plugins in plugin-files will be dlopened, allowing them to statically construct instances of the various Register* types Nix supports.
2018-01-09nix.conf: builders-use-substitutesRenzo Carbonara
Fixes #937
2017-11-25build-remote: Fix missing log outputaszlig
The storeUri variable in the build-remote hook is declared very much to the start of the main function and a bunch of lines later, the same variable gets checked via hasPrefix() but it gets assigned *after* that check when the most suitable machine for the build was choosen. So I guess this was just a typo in d16fd2497374671c92cb877f9570d65783a7 and what we really want is to either checkd the prefix *after* assigning storeUri or use bestMachine->storeUri directly. I choose the latter, because the former could introduce even more regressions if the try block where the variable gets assigned terminates early. Nevertheless, the reason why the log output didn't work is because hasPrefix() checked for "ssh://" in front of storeUri, but if the storeUri isn't set correctly (or at all), we don't get the log file descriptor set up properly, leading to no log output. I've adjusted the remote-builds test to include a regression test for this, so that we can make sure we get a build output when using remote builds. In addition to that I've tested this with two of my build farms and the build logs are emitted correctly again. Signed-off-by: aszlig <aszlig@nix.build>
2017-10-24Allow shorter syntax for chroot storesEelco Dolstra
You can now say '--store /tmp/nix' instead of '--store local?root=/tmp/nix'.
2017-10-24More progress indicator improvementsEelco Dolstra
Fixes #1599.
2017-10-24Progress indicator: Show on what machine we're buildingEelco Dolstra
E.g. $ nix build nixpkgs.hello --builders 'root@wendy' [1/0/1 built] building hello-2.10 on ssh://root@wendy: checking for minix/config.h... no
2017-10-24build-remote: Work properly on a chrooted storeEelco Dolstra
2017-10-24Handle log messages from build-remoteEelco Dolstra
This makes the progress indicator show statuses like "connecting to 'root@machine'".
2017-10-23Pass all settings to build-remoteEelco Dolstra
This ensures that command line flags such as --builders get passed correctly.
2017-10-23build-remote: Put current load under the store state directoryEelco Dolstra
Fixes the error error: opening lock file '/nix/var/nix/current-load/main-lock': Permission denied when using a chroot store.
2017-07-31Merge branch 'macOS' of https://github.com/davidak/nixEelco Dolstra
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30replace "OS X" with "macOS"davidak
2017-07-03Replace a few bool flags with enumsEelco Dolstra
Functions like copyClosure() had 3 bool arguments, which creates a severe risk of mixing up arguments. Also, implement copyClosure() using copyPaths().
2017-05-08build-remote: Check remote build statusEelco Dolstra
2017-05-02build-remote: Fix fallback to other machines when connecting failsEelco Dolstra
Opening an SSHStore or LegacySSHStore does not actually establish a connection, so the try/catch block here did nothing. Added a Store::connect() method to test whether a connection can be established.
2017-05-02Add an option for specifying remote buildersEelco Dolstra
This is useful for one-off situations where you want to specify a builder on the command line instead of having to mess with nix.machines. E.g. $ nix-build -A hello --argstr system x86_64-darwin \ --option builders 'root@macstadium1 x86_64-darwin' will perform the specified build on "macstadium1". It also removes the need for a separate nix.machines file since you can specify builders in nix.conf directly. (In fact nix.machines is yet another hack that predates the general nix.conf configuration file, IIRC.) Note: this option is supported by the daemon for trusted users. The fact that this allows trusted users to specify paths to SSH keys to which they don't normally have access is maybe a bit too much trust...
2017-05-02Factor out machines.conf parsingEelco Dolstra
This allows hydra-queue-runner to use it.
2017-05-02build-hook: If there are no machines defined, quit permanentlyEelco Dolstra
2017-05-02build-remote: Ugly hackery to get build logs to workEelco Dolstra
The build hook mechanism expects build log output to go to file descriptor 4, so do that.
2017-05-01build-remote: Don't require signaturesEelco Dolstra
This restores the old behaviour.
2017-05-01Support arbitrary store URIs in nix.machinesEelco Dolstra
For backwards compatibility, if the URI is just a hostname, ssh:// (i.e. LegacySSHStore) is prepended automatically. Also, all fields except the URI are now optional. For example, this is a valid nix.machines file: local?root=/tmp/nix This is useful for testing the remote build machinery since you don't have to mess around with ssh.
2017-05-01Remove $NIX_BUILD_HOOK and $NIX_CURRENT_LOADEelco Dolstra
This is to simplify remote build configuration. These environment variables predate nix.conf. The build hook now has a sensible default (namely build-remote). The current load is kept in the Nix state directory now.
2017-05-01build-remote: Don't copy the .drv closureEelco Dolstra
Since build-remote uses buildDerivation() now, we don't need to copy the .drv file anymore. This greatly reduces the set of input paths copied to the remote side (e.g. from 392 to 51 store paths for GNU hello on x86_64-darwin).
2017-05-01Pass verbosity level to build hookEelco Dolstra
2017-03-16ssh:// -> ssh-ng://, legacy-ssh:// -> ssh://Eelco Dolstra
2017-03-16copyPaths(): Use queryValidPaths() to reduce SSH latencyEelco Dolstra
2017-03-03build-remote: Don't use a SSH masterEelco Dolstra
This is unnecessary because we make only one connection.
2017-03-03build-remote: Fix passing SSH keyEelco Dolstra
2017-03-03build-remote: Misc cleanupEelco Dolstra
2017-02-07Merge branch 'nix-copy-closure-c++' of https://github.com/shlevy/nixEelco Dolstra
2017-01-26openLockFile: Return an AutoCloseFDEelco Dolstra
2017-01-24Hopefully fix build on older GCCEelco Dolstra
http://hydra.nixos.org/build/46805140
2017-01-24Fix typoEelco Dolstra
2017-01-24build-remote: Use futimes instead of futimens on APPLEShea Levy
2017-01-20nix-copy-closure: Implement in C++.Shea Levy
Tests fail currently because the database is not given proper hashes in the VM
2017-01-10build-remote: Don't use C++ streams to read the conf fileShea Levy
2017-01-10build-remote: Use std::set for feature setsShea Levy
2016-11-10build-remote: replace strtoull with stoull to take advantage of C++ error ↵Shea Levy
handling