From 0d54671b7b3aa96ab45347e65352979d874346ea Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Wed, 21 Feb 2018 16:22:49 +0100 Subject: Manual: Update chapter on remote builds Alos add a command "nix ping-store" to make it easier to see if Nix can connect to a remote builder (e.g. 'nix ping-store --store ssh://mac'). --- doc/manual/advanced-topics/distributed-builds.xml | 211 +++++++++++++++------- 1 file changed, 144 insertions(+), 67 deletions(-) (limited to 'doc/manual/advanced-topics') diff --git a/doc/manual/advanced-topics/distributed-builds.xml b/doc/manual/advanced-topics/distributed-builds.xml index 1957e1105..20fd6a0cf 100644 --- a/doc/manual/advanced-topics/distributed-builds.xml +++ b/doc/manual/advanced-topics/distributed-builds.xml @@ -4,71 +4,109 @@ version="5.0" xml:id='chap-distributed-builds'> -Distributed Builds - -Nix supports distributed builds, where a local Nix installation can -forward Nix builds to other machines over the network. This allows -multiple builds to be performed in parallel (thus improving -performance) and allows Nix to perform multi-platform builds in a -semi-transparent way. For instance, if you perform a build for a -x86_64-darwin on an i686-linux -machine, Nix can automatically forward the build to a -x86_64-darwin machine, if available. - -You can enable distributed builds by setting the environment -variable NIX_BUILD_HOOK to point to a program that Nix -will call whenever it wants to build a derivation. The build hook -(typically a shell or Perl script) can decline the build, in which Nix -will perform it in the usual way if possible, or it can accept it, in -which case it is responsible for somehow getting the inputs of the -build to another machine, doing the build there, and getting the -results back. - -Remote machine configuration: -<filename>remote-systems.conf</filename> - -nix@mcflurry.labs.cs.uu.nl x86_64-darwin /home/nix/.ssh/id_quarterpounder_auto 2 -nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm -nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 -nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf - - - -Nix ships with a build hook that should be suitable for most -purposes. It uses ssh and -nix-copy-closure to copy the build inputs and -outputs and perform the remote build. To use it, you should set -NIX_BUILD_HOOK to -prefix/libexec/nix/build-remote. -You should also define a list of available build machines and point -the environment variable NIX_REMOTE_SYSTEMS to -it. NIX_REMOTE_SYSTEMS must be an absolute path. An -example configuration is shown in . Each line in the file specifies a machine, with the following -bits of information: +Remote Builds + +Nix supports remote builds, where a local Nix installation can +forward Nix builds to other machines. This allows multiple builds to +be performed in parallel and allows Nix to perform multi-platform +builds in a semi-transparent way. For instance, if you perform a +build for a x86_64-darwin on an +i686-linux machine, Nix can automatically forward +the build to a x86_64-darwin machine, if +available. + +To forward a build to a remote machine, it’s required that the +remote machine is accessible via SSH and that it has Nix +installed. You can test whether connecting to the remote Nix instance +works, e.g. + + +$ nix ping-store --store ssh://mac + + +will try to connect to the machine named mac. It is +possible to specify an SSH identity file as part of the remote store +URI, e.g. + + +$ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key + + +Since builds should be non-interactive, the key should not have a +passphrase. Alternatively, you can load identities ahead of time into +ssh-agent or gpg-agent. + +If you get the error + + +bash: nix-store: command not found +error: cannot connect to 'mac' + + +then you need to ensure that the PATH of +non-interactive login shells contains Nix. + +If you are building via the Nix daemon, it is the Nix +daemon user account (that is, root) that should +have SSH access to the remote machine. If you can’t or don’t want to +configure root to be able to access to remote +machine, you can use a private Nix store instead by passing +e.g. --store ~/my-nix. + +The list of remote machines can be specified on the command line +or in the Nix configuration file. The former is convenient for +testing. For example, the following command allows you to build a +derivation for x86_64-darwin on a Linux machine: + + +$ uname +Linux + +$ nix build \ + '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \ + --builders 'ssh://mac x86_64-darwin' +[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac + +$ cat ./result +Darwin + + +It is possible to specify multiple builders separated by a semicolon +or a newline, e.g. + + + --builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd' + + + +Each machine specification consists of the following elements, +separated by spaces. Only the first element is required. - The name of the remote machine, with optionally the - user under which the remote build should be performed. This is - actually passed as an argument to ssh, so it can - be an alias defined in your + The URI of the remote store in the format + ssh://[username@]hostname, + e.g. ssh://nix@mac or + ssh://mac. For backward compatibility, + ssh:// may be omitted. The hostname may be an + alias defined in your ~/.ssh/config. A comma-separated list of Nix platform type identifiers, such as x86_64-darwin. It is possible for a machine to support multiple platform types, e.g., - i686-linux,x86_64-linux. + i686-linux,x86_64-linux. If omitted, this + defaults to the local platform type. - The SSH private key to be used to log in to the - remote machine. Since builds should be non-interactive, this key - should not have a passphrase! + The SSH identity file to be used to log in to the + remote machine. If omitted, SSH will use its regular + identities. - The maximum number of builds that - build-remote will execute in parallel on the - machine. Typically this should be equal to the number of CPU cores. - For instance, the machine itchy in the example - will execute up to 8 builds in parallel. + The maximum number of builds that Nix will execute + in parallel on the machine. Typically this should be equal to the + number of CPU cores. For instance, the machine + itchy in the example will execute up to 8 builds + in parallel. The “speed factor”, indicating the relative speed of the machine. If there are multiple machines of the right type, Nix @@ -76,30 +114,69 @@ bits of information: A comma-separated list of supported features. If a derivation has the - requiredSystemFeatures attribute, then - build-remote will only perform the - derivation on a machine that has the specified features. For - instance, the attribute + requiredSystemFeatures attribute, then Nix will + only perform the derivation on a machine that has the specified + features. For instance, the attribute requiredSystemFeatures = [ "kvm" ]; will cause the build to be performed on a machine that has the - kvm feature (i.e., scratchy in - the example above). + kvm feature. A comma-separated list of mandatory features. A machine will only be used to build a derivation if all of the machine’s mandatory features appear in the - derivation’s requiredSystemFeatures attribute. - Thus, in the example, the machine poochie will - only do derivations that have - requiredSystemFeatures set to ["kvm" - "perf"] or ["perf"]. + derivation’s requiredSystemFeatures + attribute.. - +For example, the machine specification + + +nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm +nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 +nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark + + +specifies several machines that can perform +i686-linux builds. However, +poochie will only do builds that have the attribute + + +requiredSystemFeatures = [ "benchmark" ]; + + +or + + +requiredSystemFeatures = [ "benchmark" "kvm" ]; + + +itchy cannot do builds that require +kvm, but scratchy does support +such builds. For regular builds, itchy will be +preferred over scratchy because it has a higher +speed factor. + +Remote builders can also be configured in +nix.conf, e.g. + + +builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd + + +Finally, remote builders can be configured in a separate configuration +file included in via the syntax +@file. For example, + + +builders = @/etc/nix/machines + + +causes the list of machines in /etc/nix/machines +to be included. (This is the default.) -- cgit v1.2.3