aboutsummaryrefslogtreecommitdiff
path: root/doc/manual/advanced-topics/distributed-builds.xml
diff options
context:
space:
mode:
authorEelco Dolstra <eelco.dolstra@logicblox.com>2014-12-14 01:07:23 +0100
committerEelco Dolstra <eelco.dolstra@logicblox.com>2014-12-14 01:07:23 +0100
commit9c830394ecb42870c58603d7b4d9f620649ad969 (patch)
tree36e6567ec4cf0ee0364e2c01f89ea16b6ee0697e /doc/manual/advanced-topics/distributed-builds.xml
parent4c4e89165227d92f9e11fc18082e50006a04ffd0 (diff)
Rename files
Diffstat (limited to 'doc/manual/advanced-topics/distributed-builds.xml')
-rw-r--r--doc/manual/advanced-topics/distributed-builds.xml115
1 files changed, 115 insertions, 0 deletions
diff --git a/doc/manual/advanced-topics/distributed-builds.xml b/doc/manual/advanced-topics/distributed-builds.xml
new file mode 100644
index 000000000..70f396f81
--- /dev/null
+++ b/doc/manual/advanced-topics/distributed-builds.xml
@@ -0,0 +1,115 @@
+<chapter xmlns="http://docbook.org/ns/docbook"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns:xi="http://www.w3.org/2001/XInclude"
+ version="5.0"
+ xml:id='chap-distributed-builds'>
+
+<title>Distributed Builds</title>
+
+<para>Nix supports distributed builds, where a local Nix installation can
+forward Nix builds to other machines over the network. This allows
+multiple builds to be performed in parallel (thus improving
+performance) and allows Nix to perform multi-platform builds in a
+semi-transparent way. For instance, if you perform a build for a
+<literal>powerpc-darwin</literal> on an <literal>i686-linux</literal>
+machine, Nix can automatically forward the build to a
+<literal>powerpc-darwin</literal> machine, if available.</para>
+
+<para>You can enable distributed builds by setting the environment
+variable <envar>NIX_BUILD_HOOK</envar> to point to a program that Nix
+will call whenever it wants to build a derivation. The build hook
+(typically a shell or Perl script) can decline the build, in which Nix
+will perform it in the usual way if possible, or it can accept it, in
+which case it is responsible for somehow getting the inputs of the
+build to another machine, doing the build there, and getting the
+results back. The details of the build hook protocol are described in
+the documentation of the <link
+linkend="envar-build-hook"><envar>NIX_BUILD_HOOK</envar>
+variable</link>.</para>
+
+<example xml:id='ex-remote-systems'><title>Remote machine configuration:
+<filename>remote-systems.conf</filename></title>
+<programlisting>
+nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2
+nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
+nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
+nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf
+</programlisting>
+</example>
+
+<para>Nix ships with a build hook that should be suitable for most
+purposes. It uses <command>ssh</command> and
+<command>nix-copy-closure</command> to copy the build inputs and
+outputs and perform the remote build. To use it, you should set
+<envar>NIX_BUILD_HOOK</envar> to
+<filename><replaceable>prefix</replaceable>/libexec/nix/build-remote.pl</filename>.
+You should also define a list of available build machines and point
+the environment variable <envar>NIX_REMOTE_SYSTEMS</envar> to it. An
+example configuration is shown in <xref linkend='ex-remote-systems'
+/>. Each line in the file specifies a machine, with the following
+bits of information:
+
+<orderedlist>
+
+ <listitem><para>The name of the remote machine, with optionally the
+ user under which the remote build should be performed. This is
+ actually passed as an argument to <command>ssh</command>, so it can
+ be an alias defined in your
+ <filename>~/.ssh/config</filename>.</para></listitem>
+
+ <listitem><para>A comma-separated list of Nix platform type
+ identifiers, such as <literal>powerpc-darwin</literal>. It is
+ possible for a machine to support multiple platform types, e.g.,
+ <literal>i686-linux,x86_64-linux</literal>.</para></listitem>
+
+ <listitem><para>The SSH private key to be used to log in to the
+ remote machine. Since builds should be non-interactive, this key
+ should not have a passphrase!</para></listitem>
+
+ <listitem><para>The maximum number of builds that
+ <filename>build-remote.pl</filename> will execute in parallel on the
+ machine. Typically this should be equal to the number of CPU cores.
+ For instance, the machine <literal>itchy</literal> in the example
+ will execute up to 8 builds in parallel.</para></listitem>
+
+ <listitem><para>The “speed factor”, indicating the relative speed of
+ the machine. If there are multiple machines of the right type, Nix
+ will prefer the fastest, taking load into account.</para></listitem>
+
+ <listitem><para>A comma-separated list of <emphasis>supported
+ features</emphasis>. If a derivation has the
+ <varname>requiredSystemFeatures</varname> attribute, then
+ <filename>build-remote.pl</filename> will only perform the
+ derivation on a machine that has the specified features. For
+ instance, the attribute
+
+<programlisting>
+requiredSystemFeatures = [ "kvm" ];
+</programlisting>
+
+ will cause the build to be performed on a machine that has the
+ <literal>kvm</literal> feature (i.e., <literal>scratchy</literal> in
+ the example above).</para></listitem>
+
+ <listitem><para>A comma-separated list of <emphasis>mandatory
+ features</emphasis>. A machine will only be used to build a
+ derivation if all of the machine’s mandatory features appear in the
+ derivation’s <varname>requiredSystemFeatures</varname> attribute.
+ Thus, in the example, the machine <literal>poochie</literal> will
+ only do derivations that have
+ <varname>requiredSystemFeatures</varname> set to <literal>["kvm"
+ "perf"]</literal> or <literal>["perf"]</literal>.</para></listitem>
+
+</orderedlist>
+
+You should also set up the environment variable
+<envar>NIX_CURRENT_LOAD</envar> to point at a directory (e.g.,
+<filename>/var/run/nix/current-load</filename>) that
+<filename>build-remote.pl</filename> uses to remember how many builds
+it is currently executing remotely. It doesn't look at the actual
+load on the remote machine, so if you have multiple instances of Nix
+running, they should use the same <envar>NIX_CURRENT_LOAD</envar>
+file. Maybe in the future <filename>build-remote.pl</filename> will
+look at the actual remote load.</para>
+
+</chapter>