aboutsummaryrefslogtreecommitdiff
path: root/doc/manual/src/advanced-topics
diff options
context:
space:
mode:
authorEelco Dolstra <edolstra@gmail.com>2020-07-31 15:43:25 +0200
committerEelco Dolstra <edolstra@gmail.com>2020-07-31 15:43:25 +0200
commit1d0a7b54fa330b041a720932ee4e05dcad1d2d5c (patch)
tree48627a3530e4d6d58c612864b2e99afb11a0a902 /doc/manual/src/advanced-topics
parent0c94c176446bd9e9cb8c7e16fb7c6d88bb4e9a20 (diff)
Enable syntax highlighting
Diffstat (limited to 'doc/manual/src/advanced-topics')
-rw-r--r--doc/manual/src/advanced-topics/diff-hook.md118
-rw-r--r--doc/manual/src/advanced-topics/distributed-builds.md50
-rw-r--r--doc/manual/src/advanced-topics/post-build-hook.md68
3 files changed, 140 insertions, 96 deletions
diff --git a/doc/manual/src/advanced-topics/diff-hook.md b/doc/manual/src/advanced-topics/diff-hook.md
index e2234147f..7a2622b3d 100644
--- a/doc/manual/src/advanced-topics/diff-hook.md
+++ b/doc/manual/src/advanced-topics/diff-hook.md
@@ -7,17 +7,19 @@ for determining if the results are the same.
For purposes of demonstration, we'll use the following Nix file,
`deterministic.nix` for testing:
- let
- inherit (import <nixpkgs> {}) runCommand;
- in {
- stable = runCommand "stable" {} ''
- touch $out
- '';
-
- unstable = runCommand "unstable" {} ''
- echo $RANDOM > $out
- '';
- }
+```nix
+let
+ inherit (import <nixpkgs> {}) runCommand;
+in {
+ stable = runCommand "stable" {} ''
+ touch $out
+ '';
+
+ unstable = runCommand "unstable" {} ''
+ echo $RANDOM > $out
+ '';
+}
+```
Additionally, `nix.conf` contains:
@@ -26,10 +28,12 @@ Additionally, `nix.conf` contains:
where `/etc/nix/my-diff-hook` is an executable file containing:
- #!/bin/sh
- exec >&2
- echo "For derivation $3:"
- /run/current-system/sw/bin/diff -r "$1" "$2"
+```bash
+#!/bin/sh
+exec >&2
+echo "For derivation $3:"
+/run/current-system/sw/bin/diff -r "$1" "$2"
+```
The diff hook is executed by the same user and group who ran the build.
However, the diff hook does not have write access to the store path just
@@ -43,44 +47,55 @@ to the build command.
If the build passes and is deterministic, Nix will exit with a status
code of 0:
- $ nix-build ./deterministic.nix -A stable
- this derivation will be built:
- /nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv
- building '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'...
- /nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable
-
- $ nix-build ./deterministic.nix -A stable --check
- checking outputs of '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'...
- /nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable
+```console
+$ nix-build ./deterministic.nix -A stable
+this derivation will be built:
+ /nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv
+building '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'...
+/nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable
+
+$ nix-build ./deterministic.nix -A stable --check
+checking outputs of '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'...
+/nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable
+```
If the build is not deterministic, Nix will exit with a status code of
1:
- $ nix-build ./deterministic.nix -A unstable
- this derivation will be built:
- /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv
- building '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
- /nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable
-
- $ nix-build ./deterministic.nix -A unstable --check
- checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
- error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs
+```console
+$ nix-build ./deterministic.nix -A unstable
+this derivation will be built:
+ /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv
+building '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
+/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable
+
+$ nix-build ./deterministic.nix -A unstable --check
+checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
+error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may
+not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs
+```
In the Nix daemon's log, we will now see:
- For derivation /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv:
- 1c1
- < 8108
- ---
- > 30204
+```
+For derivation /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv:
+1c1
+< 8108
+---
+> 30204
+```
Using `--check` with `--keep-failed` will cause Nix to keep the second
build's output in a special, `.check` path:
- $ nix-build ./deterministic.nix -A unstable --check --keep-failed
- checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
- note: keeping build directory '/tmp/nix-build-unstable.drv-0'
- error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs from '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check'
+```console
+$ nix-build ./deterministic.nix -A unstable --check --keep-failed
+checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
+note: keeping build directory '/tmp/nix-build-unstable.drv-0'
+error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may
+not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs
+from '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check'
+```
In particular, notice the
`/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check` output. Nix
@@ -102,7 +117,8 @@ has copied the build results to that directory where you can examine it.
already. If the derivation has not been built Nix will fail with the
error:
- error: some outputs of '/nix/store/hzi1h60z2qf0nb85iwnpvrai3j2w7rr6-unstable.drv' are not valid, so checking is not possible
+ error: some outputs of '/nix/store/hzi1h60z2qf0nb85iwnpvrai3j2w7rr6-unstable.drv'
+ are not valid, so checking is not possible
Run the build without `--check`, and then try with `--check` again.
@@ -130,10 +146,12 @@ reproducibly:
An example output of this configuration:
- $ nix-build ./test.nix -A unstable
- this derivation will be built:
- /nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv
- building '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' (round 1/2)...
- building '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' (round 2/2)...
- output '/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable' of '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' differs from '/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable.check' from previous round
- /nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable
+```console
+$ nix-build ./test.nix -A unstable
+this derivation will be built:
+ /nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv
+building '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' (round 1/2)...
+building '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' (round 2/2)...
+output '/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable' of '/nix/store/ch6llwpr2h8c3jmnf3f2ghkhx59aa97f-unstable.drv' differs from '/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable.check' from previous round
+/nix/store/6xg356v9gl03hpbbg8gws77n19qanh02-unstable
+```
diff --git a/doc/manual/src/advanced-topics/distributed-builds.md b/doc/manual/src/advanced-topics/distributed-builds.md
index 76a5380bf..c6966a50b 100644
--- a/doc/manual/src/advanced-topics/distributed-builds.md
+++ b/doc/manual/src/advanced-topics/distributed-builds.md
@@ -11,12 +11,16 @@ To forward a build to a remote machine, it’s required that the remote
machine is accessible via SSH and that it has Nix installed. You can
test whether connecting to the remote Nix instance works, e.g.
- $ nix ping-store --store ssh://mac
+```console
+$ nix ping-store --store ssh://mac
+```
will try to connect to the machine named `mac`. It is possible to
specify an SSH identity file as part of the remote store URI, e.g.
- $ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key
+```console
+$ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key
+```
Since builds should be non-interactive, the key should not have a
passphrase. Alternatively, you can load identities ahead of time into
@@ -24,8 +28,10 @@ passphrase. Alternatively, you can load identities ahead of time into
If you get the error
- bash: nix-store: command not found
- error: cannot connect to 'mac'
+```console
+bash: nix-store: command not found
+error: cannot connect to 'mac'
+```
then you need to ensure that the `PATH` of non-interactive login shells
contains Nix.
@@ -43,21 +49,23 @@ the Nix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
- $ uname
- Linux
+```console
+$ uname
+Linux
- $ nix build \
- '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
- --builders 'ssh://mac x86_64-darwin'
- [1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
-
- $ cat ./result
- Darwin
+$ nix build \
+ '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
+ --builders 'ssh://mac x86_64-darwin'
+[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
+
+$ cat ./result
+Darwin
+```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
-```
+```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
@@ -91,8 +99,10 @@ default, set it to `-`.
the `requiredSystemFeatures` attribute, then Nix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
-
- requiredSystemFeatures = [ "kvm" ];
+
+ ```nix
+ requiredSystemFeatures = [ "kvm" ];
+ ```
will cause the build to be performed on a machine that has the `kvm`
feature.
@@ -111,11 +121,15 @@ For example, the machine specification
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
- requiredSystemFeatures = [ "benchmark" ];
+```nix
+requiredSystemFeatures = [ "benchmark" ];
+```
or
- requiredSystemFeatures = [ "benchmark" "kvm" ];
+```nix
+requiredSystemFeatures = [ "benchmark" "kvm" ];
+```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
diff --git a/doc/manual/src/advanced-topics/post-build-hook.md b/doc/manual/src/advanced-topics/post-build-hook.md
index 7b3ae58fb..bbdabed41 100644
--- a/doc/manual/src/advanced-topics/post-build-hook.md
+++ b/doc/manual/src/advanced-topics/post-build-hook.md
@@ -27,9 +27,11 @@ Use `nix-store --generate-binary-cache-key` to create our public and
private signing keys. We will sign paths with the private key, and
distribute the public key for verifying the authenticity of the paths.
- # nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public
- # cat /etc/nix/key.public
- example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
+```console
+# nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public
+# cat /etc/nix/key.public
+example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
+```
Then, add the public key and the cache URL to your `nix.conf`'s
`trusted-public-keys` and `substituters` options:
@@ -43,16 +45,18 @@ We will restart the Nix daemon in a later step.
Write the following script to `/etc/nix/upload-to-cache.sh`:
- #!/bin/sh
-
- set -eu
- set -f # disable globbing
- export IFS=' '
-
- echo "Signing paths" $OUT_PATHS
- nix sign-paths --key-file /etc/nix/key.private $OUT_PATHS
- echo "Uploading paths" $OUT_PATHS
- exec nix copy --to 's3://example-nix-cache' $OUT_PATHS
+```bash
+#!/bin/sh
+
+set -eu
+set -f # disable globbing
+export IFS=' '
+
+echo "Signing paths" $OUT_PATHS
+nix sign-paths --key-file /etc/nix/key.private $OUT_PATHS
+echo "Uploading paths" $OUT_PATHS
+exec nix copy --to 's3://example-nix-cache' $OUT_PATHS
+```
> **Note**
>
@@ -65,7 +69,9 @@ Write the following script to `/etc/nix/upload-to-cache.sh`:
Then make sure the hook program is executable by the `root` user:
- # chmod +x /etc/nix/upload-to-cache.sh
+```console
+# chmod +x /etc/nix/upload-to-cache.sh
+```
# Updating Nix Configuration
@@ -80,27 +86,33 @@ Then, restart the `nix-daemon`.
Build any derivation, for example:
- $ nix-build -E '(import <nixpkgs> {}).writeText "example" (builtins.toString builtins.currentTime)'
- this derivation will be built:
- /nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv
- building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'...
- running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
- post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
- post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
- /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+```console
+$ nix-build -E '(import <nixpkgs> {}).writeText "example" (builtins.toString builtins.currentTime)'
+this derivation will be built:
+ /nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv
+building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'...
+running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
+post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+/nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+```
Then delete the path from the store, and try substituting it from the
binary cache:
- $ rm ./result
- $ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+```console
+$ rm ./result
+$ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+```
Now, copy the path back from the cache:
- $ nix-store --realise /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
- copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'...
- warning: you did not specify '--add-root'; the result might be removed by the garbage collector
- /nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example
+```console
+$ nix-store --realise /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'...
+warning: you did not specify '--add-root'; the result might be removed by the garbage collector
+/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example
+```
# Conclusion