aboutsummaryrefslogtreecommitdiff
path: root/doc/manual/release-notes/rl-2.0.xml
diff options
context:
space:
mode:
authorEelco Dolstra <edolstra@gmail.com>2018-02-19 20:38:06 +0100
committerEelco Dolstra <edolstra@gmail.com>2018-02-19 20:38:53 +0100
commit70eb64147e073a0a7cf1d44af6815cfd4f8d507f (patch)
treedba5f62e4dc9d85dbed215075917ea2b6f8fd79a /doc/manual/release-notes/rl-2.0.xml
parenta6c0b773b72d4e30690e01f1f1dcffc28f2d9ea1 (diff)
Update release notes
Also add some examples to nix --help.
Diffstat (limited to 'doc/manual/release-notes/rl-2.0.xml')
-rw-r--r--doc/manual/release-notes/rl-2.0.xml983
1 files changed, 729 insertions, 254 deletions
diff --git a/doc/manual/release-notes/rl-2.0.xml b/doc/manual/release-notes/rl-2.0.xml
index effd2e39d..0ad17373e 100644
--- a/doc/manual/release-notes/rl-2.0.xml
+++ b/doc/manual/release-notes/rl-2.0.xml
@@ -6,165 +6,572 @@
<title>Release 2.0 (2018-02-??)</title>
-<para>This release has the following new features:</para>
+<para>The following incompatible changes have been made:</para>
<itemizedlist>
<listitem>
- <para>Start of new <command>nix</command> command line
- interface. This is a work in progress and the interface is subject
- to change.</para>
+ <para>The manifest-based substituter mechanism
+ (<command>download-using-manifests</command>) has been <link
+ xlink:href="https://github.com/NixOS/nix/commit/867967265b80946dfe1db72d40324b4f9af988ed">removed</link>. It
+ has been superseded by the binary cache substituter mechanism
+ since several years. As a result, the following programs have been
+ removed:
<itemizedlist>
+ <listitem><para><command>nix-pull</command></para></listitem>
+ <listitem><para><command>nix-generate-patches</command></para></listitem>
+ <listitem><para><command>bsdiff</command></para></listitem>
+ <listitem><para><command>bspatch</command></para></listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
- <listitem><para>Self-documenting: <option>--help</option> shows
- all available command-line arguments.</para></listitem>
-
- <listitem><para><option>--help-config</option> shows all
- configuration options.</para></listitem>
+ <listitem>
+ <para>The “copy from other stores” substituter mechanism
+ (<command>copy-from-other-stores</command> and the
+ <envar>NIX_OTHER_STORES</envar> environment variable) has been
+ removed. It was primarily used by the NixOS installer to copy
+ available paths from the installation medium. The replacement is
+ to use a chroot store as a substituter
+ (e.g. <literal>--substituters /mnt</literal>), or to build into a
+ chroot store (e.g. <literal>--store /mnt --substituter /</literal>).</para>
+ </listitem>
- <listitem><para><command>nix build</command>: Replacement for
- <command>nix-build</command>.</para></listitem>
+ <listitem>
+ <para>The command <command>nix-push</command> has been removed as
+ part of the effort to eliminate Nix's dependency on Perl. You can
+ use <command>nix copy</command> instead, e.g. <literal>nix copy
+ --to /tmp/my-binary-cache <replaceable>paths…</replaceable></literal></para>
+ </listitem>
- <listitem><para><command>nix ls-store</command> and <command>nix
- ls-nar</command> allow listing the contents of a store path or
- NAR file.</para></listitem>
+ <listitem>
+ <para>The “nested” log output feature (<option>--log-type
+ pretty</option>) has been removed. As a result,
+ <command>nix-log2xml</command> was also removed.</para>
+ </listitem>
- <listitem><para><command>nix cat-store</command> and
- <command>nix cat-nar</command> allow extracting a file from a
- store path or NAR file.</para></listitem>
+ <listitem>
+ <para>OpenSSL-based signing has been <link
+ xlink:href="https://github.com/NixOS/nix/commit/f435f8247553656774dd1b2c88e9de5d59cab203">removed</link>. This
+ feature was never well-supported. A better alternative is provided
+ by the <option>secret-key-files</option> and
+ <option>trusted-public-keys</option> options.</para>
+ </listitem>
- <listitem><para><command>nix verify</command> checks whether a
- store path is unmodified and/or is trusted.</para></listitem>
+ <listitem>
+ <para>Failed build caching has been <link
+ xlink:href="https://github.com/NixOS/nix/commit/8cffec84859cec8b610a2a22ab0c4d462a9351ff">removed</link>. This
+ feature was introduced to support the Hydra continuous build
+ system, but Hydra no longer uses it.</para>
+ </listitem>
- <listitem><para><command>nix copy-sigs</command> copies
- signatures from one store to another.</para></listitem>
+ <listitem>
+ <para><filename>nix-mode.el</filename> has been removed from
+ Nix. It is now <link
+ xlink:href="https://github.com/NixOS/nix-mode">a separate
+ repository</link> and can be installed through the MELPA package
+ repository.</para>
+ </listitem>
- <listitem><para><command>nix sign-paths</command> signs store
- paths.</para></listitem>
+</itemizedlist>
- <listitem><para><command>nix copy</command> copies paths between
- arbitrary Nix stores, generalising
- <command>nix-copy-closure</command> and
- <command>nix-push</command>.</para></listitem>
+<para>This release has the following new features:</para>
- <listitem><para><command>nix path-info</command> shows
- information about store paths.</para></listitem>
+<itemizedlist>
- <listitem><para><command>nix run</command> starts a shell in
- which the specified packages are available.</para></listitem>
+ <listitem>
+ <para>It introduces a new command named <command>nix</command>,
+ which is intended to eventually replace all
+ <command>nix-*</command> commands with a more consistent and
+ better designed user interface. It currently provides replacements
+ for some (but not all) of the functionality provided by
+ <command>nix-store</command>, <command>nix-build</command>,
+ <command>nix-shell -p</command>, <command>nix-env -qa</command>,
+ <command>nix-instantiate --eval</command>,
+ <command>nix-push</command> and
+ <command>nix-copy-closure</command>. It has the following major
+ features:</para>
- <listitem><para><command>nix log</command> shows the build log
- of a package or path. If the build log is not available locally,
- it will try to obtain it from a binary cache.</para></listitem>
+ <itemizedlist>
- <listitem><para><command>nix eval</command> replaces
- <command>nix-instantiate --eval</command>.</para></listitem>
+ <listitem>
+ <para>Unlike the legacy commands, it has a consistent way to
+ refer to packages and package-like argumements (like store
+ paths). For example, the following commands all copy the GNU
+ Hello package to a remote machine:
+
+ <screen>nix copy --to ssh://machine nixpkgs.hello</screen>
+ <screen>nix copy --to ssh://machine /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10</screen>
+ <screen>nix copy --to ssh://machine '(with import &lt;nixpkgs> {}; hello)'</screen>
+
+ By contrast, <command>nix-copy-closure</command> only accepted
+ store paths as arguments.</para>
+ </listitem>
+
+ <listitem>
+ <para>It is self-documenting: <option>--help</option> shows
+ all available command-line arguments. If
+ <option>--help</option> is given after a subcommand, it shows
+ examples for that subcommand. <command>nix
+ --help-config</command> shows all configuration
+ options.</para>
+ </listitem>
+
+ <listitem>
+ <para>It is much less verbose. By default, it displays a
+ single-line progress indicator that shows how many packages
+ are left to be built or downloaded, and (if there are running
+ builds) the most recent line of builder output. If a build
+ fails, it shows the last few lines of builder output. The full
+ build log can be retrieved using <command>nix
+ log</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para>It <link
+ xlink:href="https://github.com/NixOS/nix/commit/b8283773bd64d7da6859ed520ee19867742a03ba">provides</link>
+ all <filename>nix.conf</filename> configuration options as
+ command line flags. For example, instead of <literal>--option
+ http-connections 100</literal> you can write
+ <literal>--http-connections 100</literal>. Boolean options can
+ be written as
+ <literal>--<replaceable>foo</replaceable></literal> or
+ <literal>--no-<replaceable>foo</replaceable></literal>
+ (e.g. <option>--no-auto-optimise-store</option>).</para>
+ </listitem>
+
+ <listitem>
+ <para>Many subcommands have a <option>--json</option> flag to
+ write results to stdout in JSON format.</para>
+ </listitem>
- <listitem><para><command>nix dump-path</command> to get a NAR
- from a store path.</para></listitem>
+ </itemizedlist>
- <listitem><para><command>nix edit</command> opens the source
- code of a package in an editor.</para></listitem>
+ <warning><para>Please note that the <command>nix</command> command
+ is a work in progress and the interface is subject to
+ change.</para></warning>
- <listitem><para><command>nix search</command> replaces
- <command>nix-env -qa</command>. It searches the available
- packages for occurences of a search string in the attribute
- name, package name or description. It caches available packages
- to speed up searches.</para></listitem>
+ <para>It provides the following high-level (“porcelain”)
+ subcommands:</para>
- <listitem><para><command>nix why-depends</command> (d41c5eb13f4f3a37d80dbc6d3888644170c3b44a).</para></listitem>
+ <itemizedlist>
- <listitem><para><command>nix show-derivation</command> (e8d6ee7c1b90a2fe6d824f1a875acc56799ae6e2).</para></listitem>
+ <listitem>
+ <para><command>nix build</command> is a replacement for
+ <command>nix-build</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix run</command> executes a command in an
+ environment in which the specified packages are available. It
+ is (roughly) a replacement for <command>nix-shell
+ -p</command>. Unlike that command, it does not execute the
+ command in a shell, and has a flag (<command>-c</command>)
+ that specifies the unquoted command line to be
+ executed.</para>
+
+ <para>It is particularly useful in conjunction with chroot
+ stores, allowing Linux users who do not have permission to
+ install Nix in <command>/nix/store</command> to still use
+ binary substitutes that assume
+ <command>/nix/store</command>. For example,
+
+ <screen>nix run --store ~/my-nix nixpkgs.hello -c hello --greeting 'Hi everybody!'</screen>
+
+ downloads (or if not substitutes are available, builds) the
+ GNU Hello package into
+ <filename>~/my-nix/nix/store</filename>, then runs
+ <command>hello</command> in a mount namespace where
+ <filename>~/my-nix/nix/store</filename> is mounted onto
+ <command>/nix/store</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix search</command> replaces <command>nix-env
+ -qa</command>. It searches the available packages for
+ occurences of a search string in the attribute name, package
+ name or description. Unlike <command>nix-env -qa</command>, it
+ has a cache to speed up subsequent searches.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix copy</command> copies paths between
+ arbitrary Nix stores, generalising
+ <command>nix-copy-closure</command> and
+ <command>nix-push</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix repl</command> replaces the external
+ program <command>nix-repl</command>. It provides an
+ interactive environment for evaluating and building Nix
+ expressions. Note that it uses <literal>linenoise-ng</literal>
+ instead of GNU Readline.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix upgrade-nix</command> upgrades Nix to the
+ latest stable version. This requires that Nix is installed in
+ a profile. (Thus it won’t work on NixOS, or if it’s installed
+ outside of the Nix store.)</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix verify</command> checks whether store paths
+ are unmodified and/or “trusted” (see below). It replaces
+ <command>nix-store --verify</command> and <command>nix-store
+ --verify-path</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix log</command> shows the build log of a
+ package or path. If the build log is not available locally, it
+ will try to obtain it from the configured substituters (such
+ as <uri>cache.nixos.org</uri>, which now provides build
+ logs).</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix edit</command> opens the source code of a
+ package in your editor.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix eval</command> replaces
+ <command>nix-instantiate --eval</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command
+ xlink:href="https://github.com/NixOS/nix/commit/d41c5eb13f4f3a37d80dbc6d3888644170c3b44a">nix
+ why-depends</command> shows why one store path has another in
+ its closure. This is primarily useful to finding the causes of
+ closure bloat. For example,
+
+ <screen>nix why-depends nixpkgs.vlc nixpkgs.libdrm.dev</screen>
+
+ shows a chain of files and fragments of file contents that
+ cause the VLC package to have the “dev” output of
+ <literal>libdrm</literal> in its closure — an undesirable
+ situation.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix path-info</command> shows information about
+ store paths, replacing <command>nix-store -q</command>. A
+ useful feature is the option <option>--closure-size</option>
+ (<option>-S</option>). For example, the following command show
+ the closure sizes of every path in the current NixOS system
+ closure, sorted by size:
+
+ <screen>nix path-info -rS /run/current-system | sort -nk2</screen>
+
+ </para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix optimise-store</command> replaces
+ <command>nix-store --optimise</command>. The main difference
+ is that it has a progress indicator.</para>
+ </listitem>
- <listitem><para><command>nix add-to-store</command> (970366266b8df712f5f9cedb45af183ef5a8357f).</para></listitem>
+ </itemizedlist>
- <listitem><para><command>nix upgrade-nix</command> upgrades Nix
- to the latest stable version. This requires that Nix is
- installed in a profile. (Thus it won’t work on NixOS, or if it’s
- installed outside of the Nix store.)</para></listitem>
+ <para>A number of low-level (“plumbing”) commands are also
+ available:</para>
- <listitem><para>Progress indicator.</para></listitem>
+ <itemizedlist>
- <listitem><para>All options are available as flags now
- (b8283773bd64d7da6859ed520ee19867742a03ba).</para></listitem>
+ <listitem>
+ <para><command>nix ls-store</command> and <command>nix
+ ls-nar</command> list the contents of a store path or NAR
+ file. The former is primarily useful in conjunction with
+ remote stores, e.g.
+
+ <screen>nix ls-store --store https://cache.nixos.org/ -lR /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10</screen>
+
+ lists the contents of path in a binary cache.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix cat-store</command> and <command>nix
+ cat-nar</command> allow extracting a file from a store path or
+ NAR file.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix dump-path</command> writes the contents of
+ a store path to stdout in NAR format. This replaces
+ <command>nix-store --dump</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command
+ xlink:href="https://github.com/NixOS/nix/commit/e8d6ee7c1b90a2fe6d824f1a875acc56799ae6e2">nix
+ show-derivation</command> displays a store derivation in JSON
+ format. This is an alternative to
+ <command>pp-aterm</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command
+ xlink:href="https://github.com/NixOS/nix/commit/970366266b8df712f5f9cedb45af183ef5a8357f">nix
+ add-to-store</command> replaces <command>nix-store
+ --add</command>.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix sign-paths</command> signs store
+ paths. (TODO: add examples)</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix copy-sigs</command> copies signatures from
+ one store to another. (TODO: add examples and
+ tests)</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix show-config</command> shows all
+ configuration options and their current values.</para>
+ </listitem>
</itemizedlist>
</listitem>
<listitem>
- <para>The external program <command>nix-repl</command> has been
- integrated into Nix as <command>nix repl</command>.</para>
- </listitem>
+ <para>The store abstraction that Nix has had for a long time to
+ support store access via the Nix daemon has been extended
+ significantly. In particular, substituters (which used to be
+ external programs such as
+ <command>download-from-binary-cache</command>) are now subclasses
+ of the abstract <classname>Store</classname> class. This allows
+ many Nix commands to operate on such store types. For example,
+ <command>nix path-info</command> shows information about paths in
+ your local Nix store, while <command>nix path-info --store
+ https://cache.nixos.org/</command> shows information about paths
+ in the specified binary cache. Similarly,
+ <command>nix-copy-closure</command>, <command>nix-push</command>
+ and substitution are all instances of the general notion of
+ copying paths between different kinds of Nix stores.</para>
- <listitem>
- <para>If a fixed-output derivation produces a result with an
- incorrect hash, the output path will be moved to the location
- corresponding to the actual hash and registered as valid. Thus, a
- subsequent build of the fixed-output derivation with the correct
- hash is unnecessary.</para>
- </listitem>
+ <para>Stores are specified using an URI-like syntax,
+ e.g. <uri>https://cache.nixos.org/</uri> or
+ <uri>ssh://machine</uri>. The following store types are supported:
+
+ <itemizedlist>
+
+ <listitem>
+
+ <para><classname>LocalStore</classname> (stori URI
+ <literal>local</literal> or an absolute path) and the misnamed
+ <classname>RemoteStore</classname> (<literal>daemon</literal>)
+ provide access to a local Nix store, the latter via the Nix
+ daemon. You can use <literal>auto</literal> or the empty
+ string to auto-select a local or daemon store depending on
+ whether you have write permission to the Nix store. It is no
+ longer necessary to set the <envar>NIX_REMOTE</envar>
+ environment variable to use the Nix daemon.</para>
+
+ <para>As noted above, <classname>LocalStore</classname> now
+ supports chroot builds, allowing the “physical” location of
+ the Nix store
+ (e.g. <filename>/home/alice/nix/store</filename>) to differ
+ from its “logical” location (typically
+ <filename>/nix/store</filename>). This allows non-root users
+ to use Nix while still getting the benefits from prebuilt
+ binaries from <uri>cache.nixos.org</uri>.</para>
+
+ </listitem>
+
+ <listitem>
+
+ <para><classname>BinaryCacheStore</classname> is the abstract
+ superclass of all binary cache stores. It supports writing
+ build logs and NAR content listings in JSON format.</para>
+
+ </listitem>
+
+ <listitem>
+
+ <para><classname>HttpBinaryCacheStore</classname>
+ (<literal>http://</literal>, <literal>https://</literal>)
+ supports binary caches via HTTP or HTTPS. If the server
+ supports <literal>PUT</literal> requests, it supports
+ uploading store paths via commands such as <command>nix
+ copy</command>.</para>
+
+ </listitem>
+
+ <listitem>
+
+ <para><classname>LocalBinaryCacheStore</classname>
+ (<literal>file://</literal>) supports binary caches in the
+ local filesystem.</para>
+
+ </listitem>
+
+ <listitem>
+
+ <para><classname>S3BinaryCacheStore</classname>
+ (<literal>s3://</literal>) supports binary caches stored in
+ Amazon S3, if enabled at compile time.</para>
+
+ </listitem>
+
+ <listitem>
+
+ <para><classname>LegacySSHStore</classname> (<literal>ssh://</literal>)
+ is used to implement remote builds and
+ <command>nix-copy-closure</command>.</para>
+
+ </listitem>
+
+ <listitem>
+
+ <para><classname>SSHStore</classname>
+ (<literal>ssh-ng://</literal>) supports arbitrary Nix
+ operations on a remote machine via the same protocol used by
+ <command>nix-daemon</command>.</para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
- <listitem>
- <para>It is no longer necessary to set the
- <envar>NIX_REMOTE</envar> environment variable if you need to use
- the Nix daemon. Nix will use the daemon automatically if you don’t
- have write access to the Nix database.</para>
</listitem>
<listitem>
- <para>The Nix language now supports floating point numbers. They are
- based on regular C++ <literal>float</literal> and compatible with
- existing integers and number-related operations. Export and import to and
- from JSON and XML works, too.</para>
+
+ <para>Security has been improved in various ways:
+
+ <itemizedlist>
+
+ <listitem>
+ <para>Nix now stores signatures for local store
+ paths. When paths are copied between stores (e.g., copied from
+ a binary cache to a local store), signatures are
+ propagated.</para>
+
+ <para>Locally-built paths are signed automatically using the
+ secret keys specified by the <option>secret-key-files</option>
+ store option. Secret/public key pairs can be generated using
+ <command>nix-store
+ --generate-binary-cache-key</command>. (TODO: rename)</para>
+
+ <para>In addition, locally-built store paths are marked as
+ “ultimately trusted”, but this bit is not propagated when
+ paths are copied between stores.</para>
+ </listitem>
+
+ <listitem>
+ <para>Content-addressable store paths no longer require
+ signatures — they can be imported into a store by unprivileged
+ users even if they lack signatures.</para>
+ </listitem>
+
+ <listitem>
+ <para>The command <command>nix verify</command> checks whether
+ the specified paths are trusted, i.e., have a certain number
+ of trusted signatures, are ultimately trusted, or are
+ content-addressed.</para>
+ </listitem>
+
+ <listitem>
+ <para>Substitutions from binary caches <link
+ xlink:href="https://github.com/NixOS/nix/commit/ecbc3fedd3d5bdc5a0e1a0a51b29062f2874ac8b">now</link>
+ require signatures by default. This was already the case on
+ NixOS.</para>
+ </listitem>
+
+ <listitem>
+ <para>In Linux sandbox builds, we <link
+ xlink:href="https://github.com/NixOS/nix/commit/eba840c8a13b465ace90172ff76a0db2899ab11b">now</link>
+ use <filename>/build</filename> instead of
+ <filename>/tmp</filename> as the temporary build
+ directory. This fixes potential security problems when a build
+ accidentally stores its <envar>TMPDIR</envar> in some
+ security-sensitive place, such as an RPATH.</para>
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+
</listitem>
<listitem>
- <para><command>nix-shell</command> now sets the
- <varname>IN_NIX_SHELL</varname> environment variable during
- evaluation and in the shell itself. This can be used to perform
- different actions depending on whether you’re in a Nix shell or in
- a regular build. Nixpkgs provides
- <varname>lib.inNixShell</varname> to check this variable during
- evaluation. (bb36a1a3cf3fbe6bc9d0afcc5fa0f928bed03170)</para>
+ <para><emphasis>Pure evaluation mode</emphasis>. This is a variant
+ of the existing restricted evaluation mode. In pure mode, the Nix
+ evaluator forbids access to anything that could cause different
+ evaluations of the same command line arguments to produce a
+ different result. This includes builtin functions such as
+ <function>builtins.getEnv</function>, but more importantly,
+ <emphasis>all</emphasis> filesystem or network access unless a
+ content hash or commit hash is specified. For example, calls to
+ <function>builtins.fetchGit</function> are only allowed if a
+ <varname>rev</varname> attribute is specified.</para>
+
+ <para>The goal of this feature is to enable true reproducibility
+ and traceability of builds (including NixOS system configurations)
+ at the evaluation level. For example, in the future,
+ <command>nixos-rebuild</command> might build configurations from a
+ Nix expression in a Git repository in pure mode. That expression
+ might fetch other repositories such as Nixpkgs via
+ <function>builtins.fetchGit</function>. The commit hash of the
+ top-level repository then uniquely identifies a running system,
+ and, in conjunction with that repository, allows it to be
+ reproduced or modified.</para>
+
</listitem>
<listitem>
- <para>Internal: all <classname>Store</classname> classes are now
- thread-safe. <classname>RemoteStore</classname> supports multiple
- concurrent connections to the daemon. This is primarily useful in
- multi-threaded programs such as
- <command>hydra-queue-runner</command>.</para>
+ <para>There are several new features to support binary
+ reproducibility (i.e. to help ensure that multiple builds of the
+ same derivation produce exactly the same output). When
+ <option>enforce-determinism</option> is set to
+ <literal>false</literal>, it’s <link
+ xlink:href="https://github.com/NixOS/nix/commit/8bdf83f936adae6f2c907a6d2541e80d4120f051">no
+ longer</link> a fatal error if build rounds produce different
+ output. Also, a hook named <option>diff-hook</option> is <link
+ xlink:href="https://github.com/NixOS/nix/commit/9a313469a4bdea2d1e8df24d16289dc2a172a169w">provided</link>
+ to allow you to run tools such as <command>diffoscope</command>
+ when build rounds produce different output.</para>
</listitem>
<listitem>
- <para>The dependency on Perl has been removed. As a result, some
- (obsolete) programs have been removed: <command>nix-push</command>
- (replaced by <command>nix copy</command>),
- <command>nix-pull</command> (obsoleted by binary caches),
- <command>nix-generate-patches</command>,
- <command>bsdiff</command>, <command>bspatch</command>.</para>
+ <para>Configuring remote builds is a lot easier now. Provided you
+ are not using the Nix daemon, you can now just specify a remote
+ build machine on the command line, e.g. <literal>--option builders
+ 'ssh://my-mac x86_64-darwin'</literal>. The environment variable
+ <envar>NIX_BUILD_HOOK</envar> has been removed and is no longer
+ needed. The environment variable <envar>NIX_REMOTE_SYSTEMS</envar>
+ is still supported for compatibility, but it is also possible to
+ specify builders in <command>nix.conf</command> by setting the
+ option <literal>builders =
+ @<replaceable>path</replaceable></literal>.</para>
</listitem>
<listitem>
- <para>Improved store abstraction. Substituters
- eliminated. BinaryCacheStore, LocalBinaryCacheStore,
- HttpBinaryCacheStore, S3BinaryCacheStore (compile-time
- optional), SSHStore. Add docs + examples?
- </para>
+ <para>If a fixed-output derivation produces a result with an
+ incorrect hash, the output path is moved to the location
+ corresponding to the actual hash and registered as valid. Thus, a
+ subsequent build of the fixed-output derivation with the correct
+ hash is unnecessary.</para>
</listitem>
<listitem>
- <para>Nix now stores signatures for local store
- paths. Locally-built paths are now signed automatically using the
- secret keys specified by the <option>secret-key-files</option>
- store option.</para>
-
- <para>In addition, store paths that have been built locally are
- marked as “ultimately trusted”, and content-addressable store
- paths carry a “content-addressability assertion” that allow them
- to be trusted without any signatures.</para>
+ <para><command>nix-shell</command> <link
+ xlink:href="https://github.com/NixOS/nix/commit/ea59f39326c8e9dc42dfed4bcbf597fbce58797c">now</link>
+ sets the <varname>IN_NIX_SHELL</varname> environment variable
+ during evaluation and in the shell itself. This can be used to
+ perform different actions depending on whether you’re in a Nix
+ shell or in a regular build. Nixpkgs provides
+ <varname>lib.inNixShell</varname> to check this variable during
+ evaluation.</para>
</listitem>
<listitem>
@@ -179,7 +586,8 @@
<uri>https://nixos.org/channels/<replaceable>channel-name</replaceable>/nixexprs.tar.xz</uri>. For
example, <literal>nix-build channel:nixos-15.09 -A hello</literal>
will build the GNU Hello package from the
- <literal>nixos-15.09</literal> channel.</para>
+ <literal>nixos-15.09</literal> channel. In the future, this may
+ use Git to fetch updates more efficiently.</para>
</listitem>
<listitem>
@@ -189,45 +597,119 @@
</listitem>
<listitem>
- <para><function>builtins.fetchGit</function>.
- (38539b943a060d9cdfc24d6e5d997c0885b8aa2f)</para>
+ <para>Networking has been improved:
+
+ <itemizedlist>
+
+ <listitem>
+ <para>HTTP/2 is now supported. This makes binary cache lookups
+ <link
+ xlink:href="https://github.com/NixOS/nix/commit/90ad02bf626b885a5dd8967894e2eafc953bdf92">much
+ more efficient</link>.</para>
+ </listitem>
+
+ <listitem>
+ <para>We now retry downloads on many HTTP errors, making
+ binary caches substituters more resilient to temporary
+ failures.</para>
+ </listitem>
+
+ <listitem>
+ <para>HTTP credentials can now be configured via the standard
+ <filename>netrc</filename> mechanism.</para>
+ </listitem>
+
+ <listitem>
+ <para>If S3 support is enabled at compile time,
+ <uri>s3://</uri> URIs are <link
+ xlink:href="https://github.com/NixOS/nix/commit/9ff9c3f2f80ba4108e9c945bbfda2c64735f987b">supported</link>
+ in all places where Nix allows URIs.</para>
+ </listitem>
+
+ <listitem>
+ <para>Brotli compression is now supported. In particular,
+ <uri>cache.nixos.org</uri> build logs are now compressed using
+ Brotli.</para>
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+
</listitem>
<listitem>
- <para><literal>&lt;nix/fetchurl.nix&gt;</literal> now uses the
- content-addressable tarball cache at
- <uri>http://tarballs.nixos.org/</uri>, just like
- <function>fetchurl</function> in
- Nixpkgs. (f2682e6e18a76ecbfb8a12c17e3a0ca15c084197)</para>
+ <para><command>nix-env</command> <link
+ xlink:href="https://github.com/NixOS/nix/commit/b0cb11722626e906a73f10dd9a0c9eea29faf43a">now</link>
+ ignores packages with bad derivation names (in particular those
+ starting with a digit or containing a dot).</para>
</listitem>
<listitem>
- <para>Chroot Nix stores: allow the “physical” location of the Nix
- store (e.g. <filename>/home/alice/nix/store</filename>) to differ
- from its “logical” location (typically
- <filename>/nix/store</filename>). This allows non-root users to
- use Nix while still getting the benefits from prebuilt binaries
- from
- <uri>cache.nixos.org</uri>. (4494000e04122f24558e1436e66d20d89028b4bd,
- 3eb621750848e0e6b30e5a79f76afbb096bb6c8a)</para>
+ <para>Many configuration options have been renamed, either because
+ they were unnecessarily verbose
+ (e.g. <option>build-use-sandbox</option> is now just
+ <option>sandbox</option>) or to reflect generalised behaviour
+ (e.g. <option>binary-caches</option> is now
+ <option>substituters</option> because it allows arbitrary store
+ URIs). The old names are still supported for compatibility.</para>
</listitem>
<listitem>
- <para>On Linux, builds are now executed in a user
- namespace with uid 1000 and gid 100.</para>
+ <para>The <option>max-jobs</option> option can <link
+ xlink:href="https://github.com/NixOS/nix/commit/7251d048fa812d2551b7003bc9f13a8f5d4c95a5">now</link>
+ be set to <literal>auto</literal> to use the number of CPUs in the
+ system.</para>
</listitem>
<listitem>
- <para><function>builtins.fetchurl</function> and
- <function>builtins.fetchTarball</function> now support
- <varname>sha256</varname> and <varname>name</varname>
- attributes.</para>
+ <para>Hashes can <link
+ xlink:href="https://github.com/NixOS/nix/commit/c0015e87af70f539f24d2aa2bc224a9d8b84276b">now</link>
+ be specified in base-64 format, in addition to base-16 and the
+ non-standard base-32.</para>
</listitem>
<listitem>
- <para><literal>HttpBinaryCacheStore</literal> (the replacement of
- <command>download-from-binary-cache</command>) now retries
- automatically on certain HTTP error codes.</para>
+ <para><command>nix-shell</command> now uses
+ <varname>bashInteractive</varname> from Nixpkgs, rather than the
+ <command>bash</command> command that happens to be in the caller’s
+ <envar>PATH</envar>. This is especially important on macOS where
+ the <command>bash</command> provided by the system is seriously
+ outdated and cannot execute <literal>stdenv</literal>’s setup
+ script.</para>
+ </listitem>
+
+ <listitem>
+ <para>Nix can now automatically trigger a garbage collection if
+ free disk space drops below a certain level during a build. This
+ is configured using the <option>min-free</option> and
+ <option>max-free</option> options.</para>
+ </listitem>
+
+ <listitem>
+ <para><command>nix-store -q --roots</command> and
+ <command>nix-store --gc --print-roots</command> now show temporary
+ and in-memory roots.</para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Nix can now be extended with plugins. See the documentation of
+ the <option>plugin-files</option> option for more details.
+ </para>
+ </listitem>
+
+</itemizedlist>
+
+<para>The Nix language has the following new features:
+
+<itemizedlist>
+
+ <listitem>
+ <para>It supports floating point numbers. They are based on the
+ C++ <literal>float</literal> type and are supported by the
+ existing numerical operators. Export and import to and from JSON
+ and XML works, too.</para>
</listitem>
<listitem>
@@ -245,194 +727,187 @@ configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev"
add docs.</para>
</listitem>
- <listitem>
- <para>Support for HTTP/2. This makes binary cache lookups much
- more efficient. (90ad02bf626b885a5dd8967894e2eafc953bdf92)</para>
- </listitem>
+</itemizedlist>
- <listitem>
- <para>The <option>build-sandbox-paths</option> configuration
- option can now specify optional paths by appending a
- <literal>?</literal>, e.g. <literal>/dev/nvidiactl?</literal> will
- bind-mount <varname>/dev/nvidiactl</varname> only if it
- exists.</para>
- </listitem>
+</para>
- <listitem>
- <para>More support for testing build reproducibility: when
- <option>enforce-determinism</option> is set to
- <literal>false</literal>, it’s no longer a fatal error build
- rounds produce different output
- (8bdf83f936adae6f2c907a6d2541e80d4120f051); add a hook to run
- diffoscope when build rounds produce different output
- (9a313469a4bdea2d1e8df24d16289dc2a172a169w).</para>
- </listitem>
+<para>The following builtin functions are new or extended:
- <listitem>
- <para>Kill builds as soon as stdout/stderr is closed. This fixes a
- bug that allowed builds to hang Nix indefinitely (regardless of
- timeouts). (21948deed99a3295e4d5666e027a6ca42dc00b40)</para>
- </listitem>
+<itemizedlist>
<listitem>
- <para>Add support for passing structured data to builders. TODO:
- document. (6de33a9c675b187437a2e1abbcb290981a89ecb1)</para>
- </listitem>
+ <para><function
+ xlink:href="https://github.com/NixOS/nix/commit/38539b943a060d9cdfc24d6e5d997c0885b8aa2f">builtins.fetchGit</function>
+ allows Git repositories to be fetched at evaluation time. Thus it
+ differs from the <function>fetchgit</function> function in
+ Nixpkgs, which fetches at build time and cannot be used to fetch
+ Nix expressions during evaluation. A typical use case is to import
+ external NixOS modules from your configuration, e.g.
- <listitem>
- <para><varname>exportReferencesGraph</varname>: Export more
- complete info in JSON
- format. (c2b0d8749f7e77afc1c4b3e8dd36b7ee9720af4a)</para>
- </listitem>
+ <programlisting>imports = [ (builtins.fetchGit https://github.com/edolstra/dwarffs + "/module.nix") ];</programlisting>
- <listitem>
- <para>Support for
- netrc. (e6e74f987f0fa284d220432d426eb965269a97d6,
- 302386f775eea309679654e5ea7c972fb6e7b9af)</para>
+ </para>
</listitem>
<listitem>
- <para>Support <uri>s3://</uri> URIs in all places where Nix allows
- URIs. (9ff9c3f2f80ba4108e9c945bbfda2c64735f987b)</para>
+ <para>Similarly, <function>builtins.fetchMercurial</function>
+ allows you to fetch Mercurial repositories.</para>
</listitem>
<listitem>
- <para>The <option>build-max-jobs</option> option can be set to
- <literal>auto</literal> to use the number of CPUs in the
- system. (7251d048fa812d2551b7003bc9f13a8f5d4c95a5)</para>
+ <para><function>builtins.path</function> generalises
+ <function>builtins.filterSource</function> and path literals
+ (e.g. <literal>./foo</literal>). It allows specifying a store path
+ name that differs from the source path name
+ (e.g. <literal>builtins.path { path = ./foo; name = "bar";
+ }</literal>) and also supports filtering out unwanted
+ files.</para>
</listitem>
<listitem>
- <para>Add support for Brotli compression.
- <uri>cache.nixos.org</uri> compresses build logs using
- Brotli.</para>
+ <para><function>builtins.fetchurl</function> and
+ <function>builtins.fetchTarball</function> now support
+ <varname>sha256</varname> and <varname>name</varname>
+ attributes.</para>
</listitem>
<listitem>
- <para>Substitutions from binary caches now require signatures by
- default. This was already the case on
- NixOS. (ecbc3fedd3d5bdc5a0e1a0a51b29062f2874ac8b)</para>
+ <para><function
+ xlink:href="https://github.com/NixOS/nix/commit/b8867a0239b1930a16f9ef3f7f3e864b01416dff">builtins.split</function>
+ splits a string using a POSIX extended regular expression as the
+ separator.</para>
</listitem>
<listitem>
- <para><command>nix-env</command> now ignores packages with bad
- derivation names (in particular those starting with a digit or
- containing a
- dot). (b0cb11722626e906a73f10dd9a0c9eea29faf43a)</para>
+ <para><function
+ xlink:href="https://github.com/NixOS/nix/commit/26d92017d3b36cff940dcb7d1611c42232edb81a">builtins.partition</function>
+ partitions the elements of a list into two lists, depending on a
+ Boolean predicate.</para>
</listitem>
<listitem>
- <para>Renamed various configuration options. (TODO: in progress)</para>
+ <para><literal>&lt;nix/fetchurl.nix&gt;</literal> now uses the
+ content-addressable tarball cache at
+ <uri>http://tarballs.nixos.org/</uri>, just like
+ <function>fetchurl</function> in
+ Nixpkgs. (f2682e6e18a76ecbfb8a12c17e3a0ca15c084197)</para>
</listitem>
<listitem>
- <para>Remote machines can now be specified on the command
- line. TODO:
- document. (1a68710d4dff609bbaf61db3e17a2573f0aadf17)</para>
+ <para>In restricted and pure evaluation mode, builtin functions
+ that download from the network (such as
+ <function>fetchGit</function>) are permitted to fetch underneath a
+ list of URI prefixes specified in the option
+ <option>allowed-uris</option>.</para>
</listitem>
- <listitem>
- <para>In Linux sandbox builds, we now use
- <filename>/build</filename> instead of <filename>/tmp</filename>
- as the temporary build directory. This fixes potential security
- problems when a build accidentally stores its
- <envar>TMPDIR</envar> in some critical place, such as an
- RPATH. (eba840c8a13b465ace90172ff76a0db2899ab11b)</para>
- </listitem>
+</itemizedlist>
- <listitem>
- <para>In Linux sandbox builds, we now provide a default
- <filename>/bin/sh</filename> (namely <filename>ash</filename> from
- BusyBox). (a2d92bb20e82a0957067ede60e91fab256948b41)</para>
- </listitem>
+</para>
- <listitem>
- <para>Make all configuration options available as command line
- flags (b8283773bd64d7da6859ed520ee19867742a03ba).</para>
- </listitem>
+<para>The Nix build environment has the following changes:
- <listitem>
- <para>Support base-64
- hashes. (c0015e87af70f539f24d2aa2bc224a9d8b84276b)</para>
- </listitem>
+<itemizedlist>
<listitem>
- <para><command>nix-shell</command> now uses
- <varname>bashInteractive</varname> from Nixpkgs, rather than the
- <command>bash</command> command that happens to be in the caller’s
- <envar>PATH</envar>. This is especially important on macOS where
- the <command>bash</command> provided by the system is seriously
- outdated and cannot execute <literal>stdenv</literal>’s setup
- script.</para>
+ <para>Values such as Booleans, integers, (nested) lists and
+ attribute sets can <link
+ xlink:href="https://github.com/NixOS/nix/commit/6de33a9c675b187437a2e1abbcb290981a89ecb1">now</link>
+ be passed to builders in a non-lossy way. If the special attribute
+ <varname>__structuredAttrs</varname> is set to
+ <literal>true</literal>, the other derivation attributes are
+ serialised in JSON format and made available to the builder via
+ the file <envar>.attrs.json</envar> in the builder’s temporary
+ directory. This obviates the need for
+ <varname>passAsFile</varname> since JSON files have no size
+ restrictions, unlike process environments.</para>
+
+ <para><link
+ xlink:href="https://github.com/NixOS/nix/commit/2d5b1b24bf70a498e4c0b378704cfdb6471cc699">As
+ a convenience to Bash builders</link>, Nix writes a script named
+ <envar>.attrs.sh</envar> to the builder’s directory that
+ initialises shell variables corresponding to all attributes that
+ are representable in Bash. This includes non-nested (associative)
+ arrays. For example, the attribute <literal>hardening.format =
+ true</literal> ends up as the Bash associative array element
+ <literal>${hardening[format]}</literal>.</para>
+ </listitem>
+
+ <listitem>
+ <para>Builders can <link
+ xlink:href="https://github.com/NixOS/nix/commit/88e6bb76de5564b3217be9688677d1c89101b2a3">now</link>
+ communicate what build phase they are in by writing messages to
+ the file descriptor specified in <envar>NIX_LOG_FD</envar>. The
+ current phase is shown by the <command>nix</command> progress
+ indicator.
+ </para>
</listitem>
<listitem>
- <para>New builtin functions: <function>builtins.split</function>
- (b8867a0239b1930a16f9ef3f7f3e864b01416dff),
- <function>builtins.partition</function>.</para>
+ <para>In Linux sandbox builds, we <link
+ xlink:href="https://github.com/NixOS/nix/commit/a2d92bb20e82a0957067ede60e91fab256948b41">now</link>
+ provide a default <filename>/bin/sh</filename> (namely
+ <filename>ash</filename> from BusyBox).</para>
</listitem>
<listitem>
- <para>Automatic garbage collection.</para>
+ <para>In structured attribute mode,
+ <varname>exportReferencesGraph</varname> <link
+ xlink:href="https://github.com/NixOS/nix/commit/c2b0d8749f7e77afc1c4b3e8dd36b7ee9720af4a">exports</link>
+ extended information about closures in JSON format. In particular,
+ it includes the sizes and hashes of paths. This is primarily
+ useful for NixOS image builders.</para>
</listitem>
<listitem>
- <para><command>nix-store -q --roots</command> and
- <command>nix-store --gc --print-roots</command> now show temporary
- and in-memory roots.</para>
+ <para>Builds are <link
+ xlink:href="https://github.com/NixOS/nix/commit/21948deed99a3295e4d5666e027a6ca42dc00b40">now</link>
+ killed as soon as Nix receives EOF on the builder’s stdout or
+ stderr. This fixes a bug that allowed builds to hang Nix
+ indefinitely, regardless of
+ timeouts.</para>
</listitem>
<listitem>
- <para>Builders can now communicate what build phase they are in by
- writing messages to the file descriptor specified in
- <envar>NIX_LOG_FD</envar>. (88e6bb76de5564b3217be9688677d1c89101b2a3)
- </para>
+ <para>The <option>sandbox-paths</option> configuration
+ option can now specify optional paths by appending a
+ <literal>?</literal>, e.g. <literal>/dev/nvidiactl?</literal> will
+ bind-mount <varname>/dev/nvidiactl</varname> only if it
+ exists.</para>
</listitem>
<listitem>
- <para>
- Nix can now be extended with plugins. See the documentation of
- the 'plugin-files' option for more details.
- </para>
+ <para>On Linux, builds are now executed in a user
+ namespace with uid 1000 and gid 100.</para>
</listitem>
</itemizedlist>
-<para>Some features were removed:</para>
-
-<itemizedlist>
-
- <listitem>
- <para>“Nested” log output. As a result,
- <command>nix-log2xml</command> was also removed.</para>
- </listitem>
+</para>
- <listitem>
- <para>OpenSSL-based signing. (f435f8247553656774dd1b2c88e9de5d59cab203)</para>
- </listitem>
+<para>A number of significant internal changes were made:
- <listitem>
- <para>Caching of failed
- builds. (8cffec84859cec8b610a2a22ab0c4d462a9351ff)</para>
- </listitem>
+<itemizedlist>
<listitem>
- <para><filename>nix-mode.el</filename> has been removed from
- Nix. It is now a separate repository in
- <uri>https://github.com/NixOS/nix-mode</uri> and can be installed
- through the MELPA package repository.</para>
+ <para>Nix no longer depends on Perl and all Perl components have
+ been rewritten in C++ or removed. The Perl bindings that used to
+ be part of Nix have been moved to a separate package,
+ <literal>nix-perl</literal>.</para>
</listitem>
<listitem>
- <para>In restricted evaluation mode
- (<option>--restrict-eval</option>), builtin functions that
- download from the network (such as <function>fetchGit</function>)
- are permitted to fetch underneath the list of URI prefixes
- specified in the option <option>allowed-uris</option>.</para>
+ <para>All <classname>Store</classname> classes are now
+ thread-safe. <classname>RemoteStore</classname> supports multiple
+ concurrent connections to the daemon. This is primarily useful in
+ multi-threaded programs such as
+ <command>hydra-queue-runner</command>.</para>
</listitem>
</itemizedlist>
+</para>
+
<para>This release has contributions from TBD.</para>
</section>