diff options
| author | John Ericson <John.Ericson@Obsidian.Systems> | 2021-04-27 19:06:58 +0000 |
|---|---|---|
| committer | John Ericson <John.Ericson@Obsidian.Systems> | 2021-04-27 19:06:58 +0000 |
| commit | e023c985d58094041e74ff59a51757bc75687ca7 (patch) | |
| tree | 8865872040ac8752c8349b73fa71b82e80dc2584 /doc | |
| parent | d3cfc14e3a370116e5715d5de5f64ed34dd2f912 (diff) | |
| parent | 906adadacd2d1c98346a2f42c0b42a32d2806d94 (diff) | |
Merge remote-tracking branch 'upstream/master' into auto-uid-allocation
Diffstat (limited to 'doc')
22 files changed, 788 insertions, 1338 deletions
diff --git a/doc/manual/command-ref/conf-file.xml b/doc/manual/command-ref/conf-file.xml deleted file mode 100644 index d0f1b09ca..000000000 --- a/doc/manual/command-ref/conf-file.xml +++ /dev/null @@ -1,1236 +0,0 @@ -<?xml version="1.0" encoding="utf-8"?> -<refentry xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - xml:id="sec-conf-file" - version="5"> - -<refmeta> - <refentrytitle>nix.conf</refentrytitle> - <manvolnum>5</manvolnum> - <refmiscinfo class="source">Nix</refmiscinfo> - <refmiscinfo class="version"><xi:include href="../version.txt" parse="text"/></refmiscinfo> -</refmeta> - -<refnamediv> - <refname>nix.conf</refname> - <refpurpose>Nix configuration file</refpurpose> -</refnamediv> - -<refsection><title>Description</title> - -<para>By default Nix reads settings from the following places:</para> - -<para>The system-wide configuration file -<filename><replaceable>sysconfdir</replaceable>/nix/nix.conf</filename> -(i.e. <filename>/etc/nix/nix.conf</filename> on most systems), or -<filename>$NIX_CONF_DIR/nix.conf</filename> if -<envar>NIX_CONF_DIR</envar> is set. Values loaded in this file are not forwarded to the Nix daemon. The -client assumes that the daemon has already loaded them. -</para> - -<para>User-specific configuration files:</para> - -<para> - If <envar>NIX_USER_CONF_FILES</envar> is set, then each path separated by - <literal>:</literal> will be loaded in reverse order. -</para> - -<para> - Otherwise it will look for <filename>nix/nix.conf</filename> files in - <envar>XDG_CONFIG_DIRS</envar> and <envar>XDG_CONFIG_HOME</envar>. - - The default location is <filename>$HOME/.config/nix.conf</filename> if - those environment variables are unset. -</para> - -<para>The configuration files consist of -<literal><replaceable>name</replaceable> = -<replaceable>value</replaceable></literal> pairs, one per line. Other -files can be included with a line like <literal>include -<replaceable>path</replaceable></literal>, where -<replaceable>path</replaceable> is interpreted relative to the current -conf file and a missing file is an error unless -<literal>!include</literal> is used instead. -Comments start with a <literal>#</literal> character. Here is an -example configuration file:</para> - -<programlisting> -keep-outputs = true # Nice for developers -keep-derivations = true # Idem -</programlisting> - -<para>You can override settings on the command line using the -<option>--option</option> flag, e.g. <literal>--option keep-outputs -false</literal>.</para> - -<para>The following settings are currently available: - -<variablelist> - - - <varlistentry xml:id="conf-allowed-uris"><term><literal>allowed-uris</literal></term> - - <listitem> - - <para>A list of URI prefixes to which access is allowed in - restricted evaluation mode. For example, when set to - <literal>https://github.com/NixOS</literal>, builtin functions - such as <function>fetchGit</function> are allowed to access - <literal>https://github.com/NixOS/patchelf.git</literal>.</para> - - </listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-allow-import-from-derivation"><term><literal>allow-import-from-derivation</literal></term> - - <listitem><para>By default, Nix allows you to <function>import</function> from a derivation, - allowing building at evaluation time. With this option set to false, Nix will throw an error - when evaluating an expression that uses this feature, allowing users to ensure their evaluation - will not require any builds to take place.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-allow-new-privileges"><term><literal>allow-new-privileges</literal></term> - - <listitem><para>(Linux-specific.) By default, builders on Linux - cannot acquire new privileges by calling setuid/setgid programs or - programs that have file capabilities. For example, programs such - as <command>sudo</command> or <command>ping</command> will - fail. (Note that in sandbox builds, no such programs are available - unless you bind-mount them into the sandbox via the - <option>sandbox-paths</option> option.) You can allow the - use of such programs by enabling this option. This is impure and - usually undesirable, but may be useful in certain scenarios - (e.g. to spin up containers or set up userspace network interfaces - in tests).</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-allowed-users"><term><literal>allowed-users</literal></term> - - <listitem> - - <para>A list of names of users (separated by whitespace) that - are allowed to connect to the Nix daemon. As with the - <option>trusted-users</option> option, you can specify groups by - prefixing them with <literal>@</literal>. Also, you can allow - all users by specifying <literal>*</literal>. The default is - <literal>*</literal>.</para> - - <para>Note that trusted users are always allowed to connect.</para> - - </listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-auto-optimise-store"><term><literal>auto-optimise-store</literal></term> - - <listitem><para>If set to <literal>true</literal>, Nix - automatically detects files in the store that have identical - contents, and replaces them with hard links to a single copy. - This saves disk space. If set to <literal>false</literal> (the - default), you can still run <command>nix-store - --optimise</command> to get rid of duplicate - files.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-builders"> - <term><literal>builders</literal></term> - <listitem> - <para>A list of machines on which to perform builds. <phrase - condition="manual">See <xref linkend="chap-distributed-builds" - /> for details.</phrase></para> - </listitem> - </varlistentry> - - - <varlistentry xml:id="conf-builders-use-substitutes"><term><literal>builders-use-substitutes</literal></term> - - <listitem><para>If set to <literal>true</literal>, Nix will instruct - remote build machines to use their own binary substitutes if available. In - practical terms, this means that remote hosts will fetch as many build - dependencies as possible from their own substitutes (e.g, from - <literal>cache.nixos.org</literal>), instead of waiting for this host to - upload them all. This can drastically reduce build times if the network - connection between this computer and the remote build host is slow. Defaults - to <literal>false</literal>.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-build-users-group"><term><literal>build-users-group</literal></term> - - <listitem><para>This options specifies the Unix group containing - the Nix build user accounts. In multi-user Nix installations, - builds should not be performed by the Nix account since that would - allow users to arbitrarily modify the Nix store and database by - supplying specially crafted builders; and they cannot be performed - by the calling user since that would allow him/her to influence - the build result.</para> - - <para>Therefore, if this option is non-empty and specifies a valid - group, builds will be performed under the user accounts that are a - member of the group specified here (as listed in - <filename>/etc/group</filename>). Those user accounts should not - be used for any other purpose!</para> - - <para>Nix will never run two builds under the same user account at - the same time. This is to prevent an obvious security hole: a - malicious user writing a Nix expression that modifies the build - result of a legitimate Nix expression being built by another user. - Therefore it is good to have as many Nix build user accounts as - you can spare. (Remember: uids are cheap.)</para> - - <para>The build users should have permission to create files in - the Nix store, but not delete them. Therefore, - <filename>/nix/store</filename> should be owned by the Nix - account, its group should be the group specified here, and its - mode should be <literal>1775</literal>.</para> - - <para>If the build users group is empty, builds will be performed - under the uid of the Nix process (that is, the uid of the caller - if <envar>NIX_REMOTE</envar> is empty, the uid under which the Nix - daemon runs if <envar>NIX_REMOTE</envar> is - <literal>daemon</literal>). Obviously, this should not be used in - multi-user settings with untrusted users.</para> - - </listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-compress-build-log"><term><literal>compress-build-log</literal></term> - - <listitem><para>If set to <literal>true</literal> (the default), - build logs written to <filename>/nix/var/log/nix/drvs</filename> - will be compressed on the fly using bzip2. Otherwise, they will - not be compressed.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-connect-timeout"><term><literal>connect-timeout</literal></term> - - <listitem> - - <para>The timeout (in seconds) for establishing connections in - the binary cache substituter. It corresponds to - <command>curl</command>’s <option>--connect-timeout</option> - option.</para> - - </listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-cores"><term><literal>cores</literal></term> - - <listitem><para>Sets the value of the - <envar>NIX_BUILD_CORES</envar> environment variable in the - invocation of builders. Builders can use this variable at their - discretion to control the maximum amount of parallelism. For - instance, in Nixpkgs, if the derivation attribute - <varname>enableParallelBuilding</varname> is set to - <literal>true</literal>, the builder passes the - <option>-j<replaceable>N</replaceable></option> flag to GNU Make. - It can be overridden using the <option - linkend='opt-cores'>--cores</option> command line switch and - defaults to <literal>1</literal>. The value <literal>0</literal> - means that the builder should use all available CPU cores in the - system.</para> - - <para>See also <xref linkend="chap-tuning-cores-and-jobs" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-diff-hook"><term><literal>diff-hook</literal></term> - <listitem> - <para> - Absolute path to an executable capable of diffing build results. - The hook executes if <xref linkend="conf-run-diff-hook" /> is - true, and the output of a build is known to not be the same. - This program is not executed to determine if two results are the - same. - </para> - - <para> - The diff hook is executed by the same user and group who ran the - build. However, the diff hook does not have write access to the - store path just built. - </para> - - <para>The diff hook program receives three parameters:</para> - - <orderedlist> - <listitem> - <para> - A path to the previous build's results - </para> - </listitem> - - <listitem> - <para> - A path to the current build's results - </para> - </listitem> - - <listitem> - <para> - The path to the build's derivation - </para> - </listitem> - - <listitem> - <para> - The path to the build's scratch directory. This directory - will exist only if the build was run with - <option>--keep-failed</option>. - </para> - </listitem> - </orderedlist> - - <para> - The stderr and stdout output from the diff hook will not be - displayed to the user. Instead, it will print to the nix-daemon's - log. - </para> - - <para>When using the Nix daemon, <literal>diff-hook</literal> must - be set in the <filename>nix.conf</filename> configuration file, and - cannot be passed at the command line. - </para> - </listitem> - </varlistentry> - - <varlistentry xml:id="conf-enforce-determinism"> - <term><literal>enforce-determinism</literal></term> - - <listitem><para>See <xref linkend="conf-repeat" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-extra-sandbox-paths"> - <term><literal>extra-sandbox-paths</literal></term> - - <listitem><para>A list of additional paths appended to - <option>sandbox-paths</option>. Useful if you want to extend - its default value.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-extra-platforms"><term><literal>extra-platforms</literal></term> - - <listitem><para>Platforms other than the native one which - this machine is capable of building for. This can be useful for - supporting additional architectures on compatible machines: - i686-linux can be built on x86_64-linux machines (and the default - for this setting reflects this); armv7 is backwards-compatible with - armv6 and armv5tel; some aarch64 machines can also natively run - 32-bit ARM code; and qemu-user may be used to support non-native - platforms (though this may be slow and buggy). Most values for this - are not enabled by default because build systems will often - misdetect the target platform and generate incompatible code, so you - may wish to cross-check the results of using this option against - proper natively-built versions of your - derivations.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-extra-substituters"><term><literal>extra-substituters</literal></term> - - <listitem><para>Additional binary caches appended to those - specified in <option>substituters</option>. When used by - unprivileged users, untrusted substituters (i.e. those not listed - in <option>trusted-substituters</option>) are silently - ignored.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-fallback"><term><literal>fallback</literal></term> - - <listitem><para>If set to <literal>true</literal>, Nix will fall - back to building from source if a binary substitute fails. This - is equivalent to the <option>--fallback</option> flag. The - default is <literal>false</literal>.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-fsync-metadata"><term><literal>fsync-metadata</literal></term> - - <listitem><para>If set to <literal>true</literal>, changes to the - Nix store metadata (in <filename>/nix/var/nix/db</filename>) are - synchronously flushed to disk. This improves robustness in case - of system crashes, but reduces performance. The default is - <literal>true</literal>.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-hashed-mirrors"><term><literal>hashed-mirrors</literal></term> - - <listitem><para>A list of web servers used by - <function>builtins.fetchurl</function> to obtain files by hash. - Given a hash type <replaceable>ht</replaceable> and a base-16 hash - <replaceable>h</replaceable>, Nix will try to download the file - from - <literal>hashed-mirror/<replaceable>ht</replaceable>/<replaceable>h</replaceable></literal>. - This allows files to be downloaded even if they have disappeared - from their original URI. For example, given the hashed mirror - <literal>http://tarballs.example.com/</literal>, when building the - derivation - -<programlisting> -builtins.fetchurl { - url = "https://example.org/foo-1.2.3.tar.xz"; - sha256 = "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"; -} -</programlisting> - - Nix will attempt to download this file from - <literal>http://tarballs.example.com/sha256/2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae</literal> - first. If it is not available there, if will try the original URI.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-http-connections"><term><literal>http-connections</literal></term> - - <listitem><para>The maximum number of parallel TCP connections - used to fetch files from binary caches and by other downloads. It - defaults to 25. 0 means no limit.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-keep-build-log"><term><literal>keep-build-log</literal></term> - - <listitem><para>If set to <literal>true</literal> (the default), - Nix will write the build log of a derivation (i.e. the standard - output and error of its builder) to the directory - <filename>/nix/var/log/nix/drvs</filename>. The build log can be - retrieved using the command <command>nix-store -l - <replaceable>path</replaceable></command>.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-keep-derivations"><term><literal>keep-derivations</literal></term> - - <listitem><para>If <literal>true</literal> (default), the garbage - collector will keep the derivations from which non-garbage store - paths were built. If <literal>false</literal>, they will be - deleted unless explicitly registered as a root (or reachable from - other roots).</para> - - <para>Keeping derivation around is useful for querying and - traceability (e.g., it allows you to ask with what dependencies or - options a store path was built), so by default this option is on. - Turn it off to save a bit of disk space (or a lot if - <literal>keep-outputs</literal> is also turned on).</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-keep-env-derivations"><term><literal>keep-env-derivations</literal></term> - - <listitem><para>If <literal>false</literal> (default), derivations - are not stored in Nix user environments. That is, the derivations of - any build-time-only dependencies may be garbage-collected.</para> - - <para>If <literal>true</literal>, when you add a Nix derivation to - a user environment, the path of the derivation is stored in the - user environment. Thus, the derivation will not be - garbage-collected until the user environment generation is deleted - (<command>nix-env --delete-generations</command>). To prevent - build-time-only dependencies from being collected, you should also - turn on <literal>keep-outputs</literal>.</para> - - <para>The difference between this option and - <literal>keep-derivations</literal> is that this one is - “sticky”: it applies to any user environment created while this - option was enabled, while <literal>keep-derivations</literal> - only applies at the moment the garbage collector is - run.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-keep-outputs"><term><literal>keep-outputs</literal></term> - - <listitem><para>If <literal>true</literal>, the garbage collector - will keep the outputs of non-garbage derivations. If - <literal>false</literal> (default), outputs will be deleted unless - they are GC roots themselves (or reachable from other roots).</para> - - <para>In general, outputs must be registered as roots separately. - However, even if the output of a derivation is registered as a - root, the collector will still delete store paths that are used - only at build time (e.g., the C compiler, or source tarballs - downloaded from the network). To prevent it from doing so, set - this option to <literal>true</literal>.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-max-build-log-size"><term><literal>max-build-log-size</literal></term> - - <listitem> - - <para>This option defines the maximum number of bytes that a - builder can write to its stdout/stderr. If the builder exceeds - this limit, it’s killed. A value of <literal>0</literal> (the - default) means that there is no limit.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-max-free"><term><literal>max-free</literal></term> - - <listitem><para>When a garbage collection is triggered by the - <literal>min-free</literal> option, it stops as soon as - <literal>max-free</literal> bytes are available. The default is - infinity (i.e. delete all garbage).</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-max-jobs"><term><literal>max-jobs</literal></term> - - <listitem><para>This option defines the maximum number of jobs - that Nix will try to build in parallel. The default is - <literal>1</literal>. The special value <literal>auto</literal> - causes Nix to use the number of CPUs in your system. <literal>0</literal> - is useful when using remote builders to prevent any local builds (except for - <literal>preferLocalBuild</literal> derivation attribute which executes locally - regardless). It can be - overridden using the <option - linkend='opt-max-jobs'>--max-jobs</option> (<option>-j</option>) - command line switch.</para> - - <para>See also <xref linkend="chap-tuning-cores-and-jobs" />.</para> - </listitem> - </varlistentry> - - <varlistentry xml:id="conf-max-silent-time"><term><literal>max-silent-time</literal></term> - - <listitem> - - <para>This option defines the maximum number of seconds that a - builder can go without producing any data on standard output or - standard error. This is useful (for instance in an automated - build system) to catch builds that are stuck in an infinite - loop, or to catch remote builds that are hanging due to network - problems. It can be overridden using the <option - linkend="opt-max-silent-time">--max-silent-time</option> command - line switch.</para> - - <para>The value <literal>0</literal> means that there is no - timeout. This is also the default.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-min-free"><term><literal>min-free</literal></term> - - <listitem> - <para>When free disk space in <filename>/nix/store</filename> - drops below <literal>min-free</literal> during a build, Nix - performs a garbage-collection until <literal>max-free</literal> - bytes are available or there is no more garbage. A value of - <literal>0</literal> (the default) disables this feature.</para> - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-narinfo-cache-negative-ttl"><term><literal>narinfo-cache-negative-ttl</literal></term> - - <listitem> - - <para>The TTL in seconds for negative lookups. If a store path is - queried from a substituter but was not found, there will be a - negative lookup cached in the local disk cache database for the - specified duration.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-narinfo-cache-positive-ttl"><term><literal>narinfo-cache-positive-ttl</literal></term> - - <listitem> - - <para>The TTL in seconds for positive lookups. If a store path is - queried from a substituter, the result of the query will be cached - in the local disk cache database including some of the NAR - metadata. The default TTL is a month, setting a shorter TTL for - positive lookups can be useful for binary caches that have - frequent garbage collection, in which case having a more frequent - cache invalidation would prevent trying to pull the path again and - failing with a hash mismatch if the build isn't reproducible. - </para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-netrc-file"><term><literal>netrc-file</literal></term> - - <listitem><para>If set to an absolute path to a <filename>netrc</filename> - file, Nix will use the HTTP authentication credentials in this file when - trying to download from a remote host through HTTP or HTTPS. Defaults to - <filename>$NIX_CONF_DIR/netrc</filename>.</para> - - <para>The <filename>netrc</filename> file consists of a list of - accounts in the following format: - -<screen> -machine <replaceable>my-machine</replaceable> -login <replaceable>my-username</replaceable> -password <replaceable>my-password</replaceable> -</screen> - - For the exact syntax, see <link - xlink:href="https://ec.haxx.se/usingcurl-netrc.html">the - <literal>curl</literal> documentation.</link></para> - - <note><para>This must be an absolute path, and <literal>~</literal> - is not resolved. For example, <filename>~/.netrc</filename> won't - resolve to your home directory's <filename>.netrc</filename>.</para></note> - </listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-plugin-files"> - <term><literal>plugin-files</literal></term> - <listitem> - <para> - A list of plugin files to be loaded by Nix. Each of these - files will be dlopened by Nix, allowing them to affect - execution through static initialization. In particular, these - plugins may construct static instances of RegisterPrimOp to - add new primops or constants to the expression language, - RegisterStoreImplementation to add new store implementations, - RegisterCommand to add new subcommands to the - <literal>nix</literal> command, and RegisterSetting to add new - nix config settings. See the constructors for those types for - more details. - </para> - <para> - Since these files are loaded into the same address space as - Nix itself, they must be DSOs compatible with the instance of - Nix running at the time (i.e. compiled against the same - headers, not linked to any incompatible libraries). They - should not be linked to any Nix libs directly, as those will - be available already at load time. - </para> - <para> - If an entry in the list is a directory, all files in the - directory are loaded as plugins (non-recursively). - </para> - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-pre-build-hook"><term><literal>pre-build-hook</literal></term> - - <listitem> - - - <para>If set, the path to a program that can set extra - derivation-specific settings for this system. This is used for settings - that can't be captured by the derivation model itself and are too variable - between different versions of the same system to be hard-coded into nix. - </para> - - <para>The hook is passed the derivation path and, if sandboxes are enabled, - the sandbox directory. It can then modify the sandbox and send a series of - commands to modify various settings to stdout. The currently recognized - commands are:</para> - - <variablelist> - <varlistentry xml:id="extra-sandbox-paths"> - <term><literal>extra-sandbox-paths</literal></term> - - <listitem> - - <para>Pass a list of files and directories to be included in the - sandbox for this build. One entry per line, terminated by an empty - line. Entries have the same format as - <literal>sandbox-paths</literal>.</para> - - </listitem> - - </varlistentry> - </variablelist> - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-post-build-hook"> - <term><literal>post-build-hook</literal></term> - <listitem> - <para>Optional. The path to a program to execute after each build.</para> - - <para>This option is only settable in the global - <filename>nix.conf</filename>, or on the command line by trusted - users.</para> - - <para>When using the nix-daemon, the daemon executes the hook as - <literal>root</literal>. If the nix-daemon is not involved, the - hook runs as the user executing the nix-build.</para> - - <itemizedlist> - <listitem><para>The hook executes after an evaluation-time build.</para></listitem> - <listitem><para>The hook does not execute on substituted paths.</para></listitem> - <listitem><para>The hook's output always goes to the user's terminal.</para></listitem> - <listitem><para>If the hook fails, the build succeeds but no further builds execute.</para></listitem> - <listitem><para>The hook executes synchronously, and blocks other builds from progressing while it runs.</para></listitem> - </itemizedlist> - - <para>The program executes with no arguments. The program's environment - contains the following environment variables:</para> - - <variablelist> - <varlistentry> - <term><envar>DRV_PATH</envar></term> - <listitem> - <para>The derivation for the built paths.</para> - <para>Example: - <literal>/nix/store/5nihn1a7pa8b25l9zafqaqibznlvvp3f-bash-4.4-p23.drv</literal> - </para> - </listitem> - </varlistentry> - - <varlistentry> - <term><envar>OUT_PATHS</envar></term> - <listitem> - <para>Output paths of the built derivation, separated by a space character.</para> - <para>Example: - <literal>/nix/store/zf5lbh336mnzf1nlswdn11g4n2m8zh3g-bash-4.4-p23-dev - /nix/store/rjxwxwv1fpn9wa2x5ssk5phzwlcv4mna-bash-4.4-p23-doc - /nix/store/6bqvbzjkcp9695dq0dpl5y43nvy37pq1-bash-4.4-p23-info - /nix/store/r7fng3kk3vlpdlh2idnrbn37vh4imlj2-bash-4.4-p23-man - /nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23</literal>. - </para> - </listitem> - </varlistentry> - </variablelist> - - <para>See <xref linkend="chap-post-build-hook" /> for an example - implementation.</para> - - </listitem> - </varlistentry> - - <varlistentry xml:id="conf-repeat"><term><literal>repeat</literal></term> - - <listitem><para>How many times to repeat builds to check whether - they are deterministic. The default value is 0. If the value is - non-zero, every build is repeated the specified number of - times. If the contents of any of the runs differs from the - previous ones and <xref linkend="conf-enforce-determinism" /> is - true, the build is rejected and the resulting store paths are not - registered as “valid” in Nix’s database.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-require-sigs"><term><literal>require-sigs</literal></term> - - <listitem><para>If set to <literal>true</literal> (the default), - any non-content-addressed path added or copied to the Nix store - (e.g. when substituting from a binary cache) must have a valid - signature, that is, be signed using one of the keys listed in - <option>trusted-public-keys</option> or - <option>secret-key-files</option>. Set to <literal>false</literal> - to disable signature checking.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-restrict-eval"><term><literal>restrict-eval</literal></term> - - <listitem> - - <para>If set to <literal>true</literal>, the Nix evaluator will - not allow access to any files outside of the Nix search path (as - set via the <envar>NIX_PATH</envar> environment variable or the - <option>-I</option> option), or to URIs outside of - <option>allowed-uri</option>. The default is - <literal>false</literal>.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-run-diff-hook"><term><literal>run-diff-hook</literal></term> - <listitem> - <para> - If true, enable the execution of <xref linkend="conf-diff-hook" />. - </para> - - <para> - When using the Nix daemon, <literal>run-diff-hook</literal> must - be set in the <filename>nix.conf</filename> configuration file, - and cannot be passed at the command line. - </para> - </listitem> - </varlistentry> - - <varlistentry xml:id="conf-sandbox"><term><literal>sandbox</literal></term> - - <listitem><para>If set to <literal>true</literal>, builds will be - performed in a <emphasis>sandboxed environment</emphasis>, i.e., - they’re isolated from the normal file system hierarchy and will - only see their dependencies in the Nix store, the temporary build - directory, private versions of <filename>/proc</filename>, - <filename>/dev</filename>, <filename>/dev/shm</filename> and - <filename>/dev/pts</filename> (on Linux), and the paths configured with the - <link linkend='conf-sandbox-paths'><literal>sandbox-paths</literal> - option</link>. This is useful to prevent undeclared dependencies - on files in directories such as <filename>/usr/bin</filename>. In - addition, on Linux, builds run in private PID, mount, network, IPC - and UTS namespaces to isolate them from other processes in the - system (except that fixed-output derivations do not run in private - network namespace to ensure they can access the network).</para> - - <para>Currently, sandboxing only work on Linux and macOS. The use - of a sandbox requires that Nix is run as root (so you should use - the <link linkend='conf-build-users-group'>“build users” - feature</link> to perform the actual builds under different users - than root).</para> - - <para>If this option is set to <literal>relaxed</literal>, then - fixed-output derivations and derivations that have the - <varname>__noChroot</varname> attribute set to - <literal>true</literal> do not run in sandboxes.</para> - - <para>The default is <literal>true</literal> on Linux and - <literal>false</literal> on all other platforms.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-sandbox-dev-shm-size"><term><literal>sandbox-dev-shm-size</literal></term> - - <listitem><para>This option determines the maximum size of the - <literal>tmpfs</literal> filesystem mounted on - <filename>/dev/shm</filename> in Linux sandboxes. For the format, - see the description of the <option>size</option> option of - <literal>tmpfs</literal> in - <citerefentry><refentrytitle>mount</refentrytitle><manvolnum>8</manvolnum></citerefentry>. The - default is <literal>50%</literal>.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-sandbox-paths"> - <term><literal>sandbox-paths</literal></term> - - <listitem><para>A list of paths bind-mounted into Nix sandbox - environments. You can use the syntax - <literal><replaceable>target</replaceable>=<replaceable>source</replaceable></literal> - to mount a path in a different location in the sandbox; for - instance, <literal>/bin=/nix-bin</literal> will mount the path - <literal>/nix-bin</literal> as <literal>/bin</literal> inside the - sandbox. If <replaceable>source</replaceable> is followed by - <literal>?</literal>, then it is not an error if - <replaceable>source</replaceable> does not exist; for example, - <literal>/dev/nvidiactl?</literal> specifies that - <filename>/dev/nvidiactl</filename> will only be mounted in the - sandbox if it exists in the host filesystem.</para> - - <para>Depending on how Nix was built, the default value for this option - may be empty or provide <filename>/bin/sh</filename> as a - bind-mount of <command>bash</command>.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-secret-key-files"><term><literal>secret-key-files</literal></term> - - <listitem><para>A whitespace-separated list of files containing - secret (private) keys. These are used to sign locally-built - paths. They can be generated using <command>nix-store - --generate-binary-cache-key</command>. The corresponding public - key can be distributed to other users, who can add it to - <option>trusted-public-keys</option> in their - <filename>nix.conf</filename>.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-show-trace"><term><literal>show-trace</literal></term> - - <listitem><para>Causes Nix to print out a stack trace in case of Nix - expression evaluation errors.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-substitute"><term><literal>substitute</literal></term> - - <listitem><para>If set to <literal>true</literal> (default), Nix - will use binary substitutes if available. This option can be - disabled to force building from source.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-stalled-download-timeout"><term><literal>stalled-download-timeout</literal></term> - <listitem> - <para>The timeout (in seconds) for receiving data from servers - during download. Nix cancels idle downloads after this timeout's - duration.</para> - </listitem> - </varlistentry> - - <varlistentry xml:id="conf-substituters"><term><literal>substituters</literal></term> - - <listitem><para>A list of URLs of substituters, separated by - whitespace. The default is - <literal>https://cache.nixos.org</literal>.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-system"><term><literal>system</literal></term> - - <listitem><para>This option specifies the canonical Nix system - name of the current installation, such as - <literal>i686-linux</literal> or - <literal>x86_64-darwin</literal>. Nix can only build derivations - whose <literal>system</literal> attribute equals the value - specified here. In general, it never makes sense to modify this - value from its default, since you can use it to ‘lie’ about the - platform you are building on (e.g., perform a Mac OS build on a - Linux machine; the result would obviously be wrong). It only - makes sense if the Nix binaries can run on multiple platforms, - e.g., ‘universal binaries’ that run on <literal>x86_64-linux</literal> and - <literal>i686-linux</literal>.</para> - - <para>It defaults to the canonical Nix system name detected by - <filename>configure</filename> at build time.</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="conf-system-features"><term><literal>system-features</literal></term> - - <listitem><para>A set of system “features” supported by this - machine, e.g. <literal>kvm</literal>. Derivations can express a - dependency on such features through the derivation attribute - <varname>requiredSystemFeatures</varname>. For example, the - attribute - -<programlisting> -requiredSystemFeatures = [ "kvm" ]; -</programlisting> - - ensures that the derivation can only be built on a machine with - the <literal>kvm</literal> feature.</para> - - <para>This setting by default includes <literal>kvm</literal> if - <filename>/dev/kvm</filename> is accessible, and the - pseudo-features <literal>nixos-test</literal>, - <literal>benchmark</literal> and <literal>big-parallel</literal> - that are used in Nixpkgs to route builds to specific - machines.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-tarball-ttl"><term><literal>tarball-ttl</literal></term> - - <listitem> - <para>Default: <literal>3600</literal> seconds.</para> - - <para>The number of seconds a downloaded tarball is considered - fresh. If the cached tarball is stale, Nix will check whether - it is still up to date using the ETag header. Nix will download - a new version if the ETag header is unsupported, or the - cached ETag doesn't match. - </para> - - <para>Setting the TTL to <literal>0</literal> forces Nix to always - check if the tarball is up to date.</para> - - <para>Nix caches tarballs in - <filename>$XDG_CACHE_HOME/nix/tarballs</filename>.</para> - - <para>Files fetched via <envar>NIX_PATH</envar>, - <function>fetchGit</function>, <function>fetchMercurial</function>, - <function>fetchTarball</function>, and <function>fetchurl</function> - respect this TTL. - </para> - </listitem> - </varlistentry> - - <varlistentry xml:id="conf-timeout"><term><literal>timeout</literal></term> - - <listitem> - - <para>This option defines the maximum number of seconds that a - builder can run. This is useful (for instance in an automated - build system) to catch builds that are stuck in an infinite loop - but keep writing to their standard output or standard error. It - can be overridden using the <option - linkend="opt-timeout">--timeout</option> command line - switch.</para> - - <para>The value <literal>0</literal> means that there is no - timeout. This is also the default.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-trace-function-calls"><term><literal>trace-function-calls</literal></term> - - <listitem> - - <para>Default: <literal>false</literal>.</para> - - <para>If set to <literal>true</literal>, the Nix evaluator will - trace every function call. Nix will print a log message at the - "vomit" level for every function entrance and function exit.</para> - - <informalexample><screen> -function-trace entered undefined position at 1565795816999559622 -function-trace exited undefined position at 1565795816999581277 -function-trace entered /nix/store/.../example.nix:226:41 at 1565795253249935150 -function-trace exited /nix/store/.../example.nix:226:41 at 1565795253249941684 -</screen></informalexample> - - <para>The <literal>undefined position</literal> means the function - call is a builtin.</para> - - <para>Use the <literal>contrib/stack-collapse.py</literal> script - distributed with the Nix source code to convert the trace logs - in to a format suitable for <command>flamegraph.pl</command>.</para> - - </listitem> - - </varlistentry> - - <varlistentry xml:id="conf-trusted-public-keys"><term><literal>trusted-public-keys</literal></term> - - <listitem><para>A whitespace-separated list of public keys. When - paths are copied from another Nix store (such as a binary cache), - they must be signed with one of these keys. For example: - <literal>cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= - hydra.nixos.org-1:CNHJZBh9K4tP3EKF6FkkgeVYsS3ohTl+oS0Qa8bezVs=</literal>.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-trusted-substituters"><term><literal>trusted-substituters</literal></term> - - <listitem><para>A list of URLs of substituters, separated by - whitespace. These are not used by default, but can be enabled by - users of the Nix daemon by specifying <literal>--option - substituters <replaceable>urls</replaceable></literal> on the - command line. Unprivileged users are only allowed to pass a - subset of the URLs listed in <literal>substituters</literal> and - <literal>trusted-substituters</literal>.</para></listitem> - - </varlistentry> - - <varlistentry xml:id="conf-trusted-users"><term><literal>trusted-users</literal></term> - - <listitem> - - <para>A list of names of users (separated by whitespace) that - have additional rights when connecting to the Nix daemon, such - as the ability to specify additional binary caches, or to import - unsigned NARs. You can also specify groups by prefixing them - with <literal>@</literal>; for instance, - <literal>@wheel</literal> means all users in the - <literal>wheel</literal> group. The default is - <literal>root</literal>.</para> - - <warning><para>Adding a user to <option>trusted-users</option> - is essentially equivalent to giving that user root access to the - system. For example, the user can set - <option>sandbox-paths</option> and thereby obtain read access to - directories that are otherwise inacessible to - them.</para></warning> - - </listitem> - - </varlistentry> - -</variablelist> -</para> - -<refsection> - <title>Deprecated Settings</title> - -<para> - -<variablelist> - - <varlistentry xml:id="conf-binary-caches"> - <term><literal>binary-caches</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>binary-caches</literal> is now an alias to - <xref linkend="conf-substituters" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-binary-cache-public-keys"> - <term><literal>binary-cache-public-keys</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>binary-cache-public-keys</literal> is now an alias to - <xref linkend="conf-trusted-public-keys" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-compress-log"> - <term><literal>build-compress-log</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-compress-log</literal> is now an alias to - <xref linkend="conf-compress-build-log" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-cores"> - <term><literal>build-cores</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-cores</literal> is now an alias to - <xref linkend="conf-cores" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-extra-chroot-dirs"> - <term><literal>build-extra-chroot-dirs</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-extra-chroot-dirs</literal> is now an alias to - <xref linkend="conf-extra-sandbox-paths" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-extra-sandbox-paths"> - <term><literal>build-extra-sandbox-paths</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-extra-sandbox-paths</literal> is now an alias to - <xref linkend="conf-extra-sandbox-paths" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-fallback"> - <term><literal>build-fallback</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-fallback</literal> is now an alias to - <xref linkend="conf-fallback" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-max-jobs"> - <term><literal>build-max-jobs</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-max-jobs</literal> is now an alias to - <xref linkend="conf-max-jobs" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-max-log-size"> - <term><literal>build-max-log-size</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-max-log-size</literal> is now an alias to - <xref linkend="conf-max-build-log-size" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-max-silent-time"> - <term><literal>build-max-silent-time</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-max-silent-time</literal> is now an alias to - <xref linkend="conf-max-silent-time" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-repeat"> - <term><literal>build-repeat</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-repeat</literal> is now an alias to - <xref linkend="conf-repeat" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-timeout"> - <term><literal>build-timeout</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-timeout</literal> is now an alias to - <xref linkend="conf-timeout" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-use-chroot"> - <term><literal>build-use-chroot</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-use-chroot</literal> is now an alias to - <xref linkend="conf-sandbox" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-use-sandbox"> - <term><literal>build-use-sandbox</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-use-sandbox</literal> is now an alias to - <xref linkend="conf-sandbox" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-build-use-substitutes"> - <term><literal>build-use-substitutes</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>build-use-substitutes</literal> is now an alias to - <xref linkend="conf-substitute" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-gc-keep-derivations"> - <term><literal>gc-keep-derivations</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>gc-keep-derivations</literal> is now an alias to - <xref linkend="conf-keep-derivations" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-gc-keep-outputs"> - <term><literal>gc-keep-outputs</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>gc-keep-outputs</literal> is now an alias to - <xref linkend="conf-keep-outputs" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-env-keep-derivations"> - <term><literal>env-keep-derivations</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>env-keep-derivations</literal> is now an alias to - <xref linkend="conf-keep-env-derivations" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-extra-binary-caches"> - <term><literal>extra-binary-caches</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>extra-binary-caches</literal> is now an alias to - <xref linkend="conf-extra-substituters" />.</para></listitem> - </varlistentry> - - <varlistentry xml:id="conf-trusted-binary-caches"> - <term><literal>trusted-binary-caches</literal></term> - - <listitem><para><emphasis>Deprecated:</emphasis> - <literal>trusted-binary-caches</literal> is now an alias to - <xref linkend="conf-trusted-substituters" />.</para></listitem> - </varlistentry> -</variablelist> -</para> -</refsection> - -</refsection> - -</refentry> diff --git a/doc/manual/generate-manpage.nix b/doc/manual/generate-manpage.nix index db266750a..964b57086 100644 --- a/doc/manual/generate-manpage.nix +++ b/doc/manual/generate-manpage.nix @@ -1,55 +1,95 @@ +command: + with builtins; with import ./utils.nix; let showCommand = - { command, section, def }: - "${section} Name\n\n" + { command, def, filename }: + '' + **Warning**: This program is **experimental** and its interface is subject to change. + '' + + "# Name\n\n" + "`${command}` - ${def.description}\n\n" - + "${section} Synopsis\n\n" + + "# Synopsis\n\n" + showSynopsis { inherit command; args = def.args; } + + (if def.commands or {} != {} + then + let + categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues def.commands))); + listCommands = cmds: + concatStrings (map (name: + "* [`${command} ${name}`](./${appendName filename name}.md) - ${cmds.${name}.description}\n") + (attrNames cmds)); + in + "where *subcommand* is one of the following:\n\n" + # FIXME: group by category + + (if length categories > 1 + then + concatStrings (map + (cat: + "**${toString cat.description}:**\n\n" + + listCommands (filterAttrs (n: v: v.category == cat) def.commands) + + "\n" + ) categories) + + "\n" + else + listCommands def.commands + + "\n") + else "") + (if def ? doc - then "${section} Description\n\n" + def.doc + "\n\n" + then def.doc + "\n\n" else "") - + (let s = showFlags def.flags; in + + (let s = showOptions def.flags; in if s != "" - then "${section} Flags\n\n${s}" + then "# Options\n\n${s}" else "") - + (if def.examples or [] != [] - then - "${section} Examples\n\n" - + concatStrings (map ({ description, command }: "${description}\n\n```console\n${command}\n```\n\n") def.examples) - else "") - + (if def.commands or [] != [] - then concatStrings ( - map (name: - "# Subcommand `${command} ${name}`\n\n" - + showCommand { command = command + " " + name; section = "##"; def = def.commands.${name}; }) - (attrNames def.commands)) - else ""); - - showFlags = flags: - concatStrings - (map (longName: - let flag = flags.${longName}; in - if flag.category or "" != "config" - then - " - `--${longName}`" - + (if flag ? shortName then " / `${flag.shortName}`" else "") - + (if flag ? labels then " " + (concatStringsSep " " (map (s: "*${s}*") flag.labels)) else "") - + " \n" - + " " + flag.description + "\n\n" - else "") - (attrNames flags)); + ; + + appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name; + + showOptions = flags: + let + categories = sort builtins.lessThan (unique (map (cmd: cmd.category) (attrValues flags))); + in + concatStrings (map + (cat: + (if cat != "" + then "**${cat}:**\n\n" + else "") + + concatStrings + (map (longName: + let + flag = flags.${longName}; + in + " - `--${longName}`" + + (if flag ? shortName then " / `-${flag.shortName}`" else "") + + (if flag ? labels then " " + (concatStringsSep " " (map (s: "*${s}*") flag.labels)) else "") + + " \n" + + " " + flag.description + "\n\n" + ) (attrNames (filterAttrs (n: v: v.category == cat) flags)))) + categories); showSynopsis = { command, args }: - "`${command}` [*flags*...] ${concatStringsSep " " + "`${command}` [*option*...] ${concatStringsSep " " (map (arg: "*${arg.label}*" + (if arg ? arity then "" else "...")) args)}\n\n"; -in + processCommand = { command, def, filename }: + [ { name = filename + ".md"; value = showCommand { inherit command def filename; }; inherit command; } ] + ++ concatMap + (name: processCommand { + filename = appendName filename name; + command = command + " " + name; + def = def.commands.${name}; + }) + (attrNames def.commands or {}); -command: +in -showCommand { command = "nix"; section = "#"; def = command; } +let + manpages = processCommand { filename = "nix"; command = "nix"; def = command; }; + summary = concatStrings (map (manpage: " - [${manpage.command}](command-ref/new-cli/${manpage.name})\n") manpages); +in +(listToAttrs manpages) // { "SUMMARY.md" = summary; } diff --git a/doc/manual/local.mk b/doc/manual/local.mk index 7d9a1a3e8..81a7755e8 100644 --- a/doc/manual/local.mk +++ b/doc/manual/local.mk @@ -4,7 +4,7 @@ MANUAL_SRCS := $(call rwildcard, $(d)/src, *.md) # Generate man pages. man-pages := $(foreach n, \ - nix-env.1 nix-build.1 nix-shell.1 nix-store.1 nix-instantiate.1 nix.1 \ + nix-env.1 nix-build.1 nix-shell.1 nix-store.1 nix-instantiate.1 \ nix-collect-garbage.1 \ nix-prefetch-url.1 nix-channel.1 \ nix-hash.1 nix-copy-closure.1 \ @@ -13,9 +13,14 @@ man-pages := $(foreach n, \ clean-files += $(d)/*.1 $(d)/*.5 $(d)/*.8 -dist-files += $(man-pages) +# Provide a dummy environment for nix, so that it will not access files outside the macOS sandbox. +dummy-env = env -i \ + HOME=/dummy \ + NIX_CONF_DIR=/dummy \ + NIX_SSL_CERT_FILE=/dummy/no-ca-bundle.crt \ + NIX_STATE_DIR=/dummy -nix-eval = $(bindir)/nix eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw --expr +nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw $(d)/%.1: $(d)/src/command-ref/%.md @printf "Title: %s\n\n" "$$(basename $@ .1)" > $^.tmp @@ -35,37 +40,51 @@ $(d)/nix.conf.5: $(d)/src/command-ref/conf-file.md $(trace-gen) lowdown -sT man $^.tmp -o $@ @rm $^.tmp -$(d)/src/command-ref/nix.md: $(d)/nix.json $(d)/generate-manpage.nix $(bindir)/nix - $(trace-gen) $(nix-eval) 'import doc/manual/generate-manpage.nix (builtins.fromJSON (builtins.readFile $<))' > $@.tmp +$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/command-ref/new-cli + $(trace-gen) cat doc/manual/src/SUMMARY.md.in | while IFS= read line; do if [[ $$line = @manpages@ ]]; then cat doc/manual/src/command-ref/new-cli/SUMMARY.md; else echo "$$line"; fi; done > $@.tmp @mv $@.tmp $@ +$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/generate-manpage.nix $(bindir)/nix + @rm -rf $@ + $(trace-gen) $(nix-eval) --write-to $@ --expr 'import doc/manual/generate-manpage.nix (builtins.fromJSON (builtins.readFile $<))' + $(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/generate-options.nix $(d)/src/command-ref/conf-file-prefix.md $(bindir)/nix @cat doc/manual/src/command-ref/conf-file-prefix.md > $@.tmp - $(trace-gen) $(nix-eval) 'import doc/manual/generate-options.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp + $(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-options.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp @mv $@.tmp $@ $(d)/nix.json: $(bindir)/nix - $(trace-gen) $(bindir)/nix __dump-args > $@.tmp + $(trace-gen) $(dummy-env) $(bindir)/nix __dump-args > $@.tmp @mv $@.tmp $@ $(d)/conf-file.json: $(bindir)/nix - $(trace-gen) env -i NIX_CONF_DIR=/dummy HOME=/dummy NIX_SSL_CERT_FILE=/dummy/no-ca-bundle.crt $(bindir)/nix show-config --json --experimental-features nix-command > $@.tmp + $(trace-gen) $(dummy-env) $(bindir)/nix show-config --json --experimental-features nix-command > $@.tmp @mv $@.tmp $@ $(d)/src/expressions/builtins.md: $(d)/builtins.json $(d)/generate-builtins.nix $(d)/src/expressions/builtins-prefix.md $(bindir)/nix @cat doc/manual/src/expressions/builtins-prefix.md > $@.tmp - $(trace-gen) $(nix-eval) 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp + $(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp @mv $@.tmp $@ $(d)/builtins.json: $(bindir)/nix - $(trace-gen) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-builtins > $@.tmp - mv $@.tmp $@ + $(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-builtins > $@.tmp + @mv $@.tmp $@ # Generate the HTML manual. install: $(docdir)/manual/index.html -$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/custom.css $(d)/src/command-ref/nix.md $(d)/src/command-ref/conf-file.md $(d)/src/expressions/builtins.md - $(trace-gen) mdbook build doc/manual -d $(docdir)/manual +# Generate 'nix' manpages. +install: $(d)/src/command-ref/new-cli + $(trace-gen) for i in doc/manual/src/command-ref/new-cli/*.md; do \ + name=$$(basename $$i .md); \ + if [[ $$name = SUMMARY ]]; then continue; fi; \ + printf "Title: %s\n\n" "$$name" > $$i.tmp; \ + cat $$i >> $$i.tmp; \ + lowdown -sT man $$i.tmp -o $(mandir)/man1/$$name.1; \ + done + +$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/command-ref/conf-file.md $(d)/src/expressions/builtins.md + $(trace-gen) RUST_LOG=warn mdbook build doc/manual -d $(docdir)/manual @cp doc/manual/highlight.pack.js $(docdir)/manual/highlight.js endif diff --git a/doc/manual/src/SUMMARY.md b/doc/manual/src/SUMMARY.md.in index 8281f683f..448fee803 100644 --- a/doc/manual/src/SUMMARY.md +++ b/doc/manual/src/SUMMARY.md.in @@ -62,11 +62,13 @@ - [nix-instantiate](command-ref/nix-instantiate.md) - [nix-prefetch-url](command-ref/nix-prefetch-url.md) - [Experimental Commands](command-ref/experimental-commands.md) - - [nix](command-ref/nix.md) +@manpages@ - [Files](command-ref/files.md) - [nix.conf](command-ref/conf-file.md) - [Glossary](glossary.md) -- [Hacking](hacking.md) +- [Contributing](contributing/contributing.md) + - [Hacking](contributing/hacking.md) + - [CLI guideline](contributing/cli-guideline.md) - [Release Notes](release-notes/release-notes.md) - [Release 2.3 (2019-09-04)](release-notes/rl-2.3.md) - [Release 2.2 (2019-01-11)](release-notes/rl-2.2.md) diff --git a/doc/manual/src/advanced-topics/distributed-builds.md b/doc/manual/src/advanced-topics/distributed-builds.md index c6966a50b..580b36736 100644 --- a/doc/manual/src/advanced-topics/distributed-builds.md +++ b/doc/manual/src/advanced-topics/distributed-builds.md @@ -37,7 +37,7 @@ then you need to ensure that the `PATH` of non-interactive login shells contains Nix. > **Warning** -> +> > If you are building via the Nix daemon, it is the Nix daemon user > account (that is, `root`) that should have SSH access to the remote > machine. If you can’t or don’t want to configure `root` to be able to @@ -52,7 +52,7 @@ example, the following command allows you to build a derivation for ```console $ uname Linux - + $ nix build \ '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \ --builders 'ssh://mac x86_64-darwin' @@ -103,7 +103,7 @@ default, set it to `-`. ```nix requiredSystemFeatures = [ "kvm" ]; ``` - + will cause the build to be performed on a machine that has the `kvm` feature. @@ -112,6 +112,10 @@ default, set it to `-`. features appear in the derivation’s `requiredSystemFeatures` attribute.. +8. The (base64-encoded) public host key of the remote machine. If omitted, SSH + will use its regular known-hosts file. Specifically, the field is calculated + via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`. + For example, the machine specification nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm diff --git a/doc/manual/src/advanced-topics/post-build-hook.md b/doc/manual/src/advanced-topics/post-build-hook.md index bbdabed41..fcb52d878 100644 --- a/doc/manual/src/advanced-topics/post-build-hook.md +++ b/doc/manual/src/advanced-topics/post-build-hook.md @@ -53,7 +53,7 @@ set -f # disable globbing export IFS=' ' echo "Signing paths" $OUT_PATHS -nix sign-paths --key-file /etc/nix/key.private $OUT_PATHS +nix store sign --key-file /etc/nix/key.private $OUT_PATHS echo "Uploading paths" $OUT_PATHS exec nix copy --to 's3://example-nix-cache' $OUT_PATHS ``` @@ -63,7 +63,7 @@ exec nix copy --to 's3://example-nix-cache' $OUT_PATHS > The `$OUT_PATHS` variable is a space-separated list of Nix store > paths. In this case, we expect and want the shell to perform word > splitting to make each output path its own argument to `nix -> sign-paths`. Nix guarantees the paths will not contain any spaces, +> store sign`. Nix guarantees the paths will not contain any spaces, > however a store path might contain glob characters. The `set -f` > disables globbing in the shell. diff --git a/doc/manual/src/command-ref/conf-file-prefix.md b/doc/manual/src/command-ref/conf-file-prefix.md index 9987393d2..3140170ab 100644 --- a/doc/manual/src/command-ref/conf-file-prefix.md +++ b/doc/manual/src/command-ref/conf-file-prefix.md @@ -19,19 +19,33 @@ By default Nix reads settings from the following places: and `XDG_CONFIG_HOME`. If these are unset, it will look in `$HOME/.config/nix.conf`. -The configuration files consist of `name = -value` pairs, one per line. Other files can be included with a line like -`include -path`, where *path* is interpreted relative to the current conf file and -a missing file is an error unless `!include` is used instead. Comments + - If `NIX_CONFIG` is set, its contents is treated as the contents of + a configuration file. + +The configuration files consist of `name = value` pairs, one per +line. Other files can be included with a line like `include path`, +where *path* is interpreted relative to the current conf file and a +missing file is an error unless `!include` is used instead. Comments start with a `#` character. Here is an example configuration file: keep-outputs = true # Nice for developers keep-derivations = true # Idem -You can override settings on the command line using the `--option` flag, -e.g. `--option keep-outputs -false`. +You can override settings on the command line using the `--option` +flag, e.g. `--option keep-outputs false`. Every configuration setting +also has a corresponding command line flag, e.g. `--max-jobs 16`; for +Boolean settings, there are two flags to enable or disable the setting +(e.g. `--keep-failed` and `--no-keep-failed`). + +A configuration setting usually overrides any previous value. However, +you can prefix the name of the setting by `extra-` to *append* to the +previous value. For instance, + + substituters = a b + extra-substituters = c d + +defines the `substituters` setting to be `a b c d`. This is also +available as a command line flag (e.g. `--extra-substituters`). The following settings are currently available: diff --git a/doc/manual/src/command-ref/env-common.md b/doc/manual/src/command-ref/env-common.md index 03016dba7..c670d82b8 100644 --- a/doc/manual/src/command-ref/env-common.md +++ b/doc/manual/src/command-ref/env-common.md @@ -81,6 +81,11 @@ Most Nix commands interpret the following environment variables: Overrides the location of the system Nix configuration directory (default `prefix/etc/nix`). + - `NIX_CONFIG` + Applies settings from Nix configuration from the environment. + The content is treated as if it was read from a Nix configuration file. + Settings are separated by the newline character. + - `NIX_USER_CONF_FILES` Overrides the location of the user Nix configuration files to load from (defaults to the XDG spec locations). The variable is treated diff --git a/doc/manual/src/command-ref/nix-hash.md b/doc/manual/src/command-ref/nix-hash.md index 7ed82cdfc..de0459b9e 100644 --- a/doc/manual/src/command-ref/nix-hash.md +++ b/doc/manual/src/command-ref/nix-hash.md @@ -45,7 +45,7 @@ md5sum`. - `--type` *hashAlgo* Use the specified cryptographic hash algorithm, which can be one of - `md5`, `sha1`, and `sha256`. + `md5`, `sha1`, `sha256`, and `sha512`. - `--to-base16` Don’t hash anything, but convert the base-32 hash representation diff --git a/doc/manual/src/command-ref/nix-prefetch-url.md b/doc/manual/src/command-ref/nix-prefetch-url.md index 78c612cd4..59ab89b29 100644 --- a/doc/manual/src/command-ref/nix-prefetch-url.md +++ b/doc/manual/src/command-ref/nix-prefetch-url.md @@ -39,7 +39,7 @@ Nix store is also printed. - `--type` *hashAlgo* Use the specified cryptographic hash algorithm, which can be one of - `md5`, `sha1`, and `sha256`. + `md5`, `sha1`, `sha256`, and `sha512`. - `--print-path` Print the store path of the downloaded file on standard output. diff --git a/doc/manual/src/command-ref/nix-shell.md b/doc/manual/src/command-ref/nix-shell.md index 45a5ff08c..54812a49f 100644 --- a/doc/manual/src/command-ref/nix-shell.md +++ b/doc/manual/src/command-ref/nix-shell.md @@ -32,7 +32,7 @@ URL of a tarball that will be downloaded and unpacked to a temporary location. The tarball must include a single top-level directory containing at least a file named `default.nix`. -If the derivation defines the variable `shellHook`, it will be evaluated +If the derivation defines the variable `shellHook`, it will be run after `$stdenv/setup` has been sourced. Since this hook is not executed by regular Nix builds, it allows you to perform initialisation specific to `nix-shell`. For example, the derivation attribute @@ -41,10 +41,12 @@ to `nix-shell`. For example, the derivation attribute shellHook = '' echo "Hello shell" + export SOME_API_TOKEN="$(cat ~/.config/some-app/api-token)" ''; ``` -will cause `nix-shell` to print `Hello shell`. +will cause `nix-shell` to print `Hello shell` and set the `SOME_API_TOKEN` +environment variable to a user-configured value. # Options @@ -76,8 +78,8 @@ All options not listed here are passed to `nix-store cleared before the interactive shell is started, so you get an environment that more closely corresponds to the “real” Nix build. A few variables, in particular `HOME`, `USER` and `DISPLAY`, are - retained. Note that `~/.bashrc` and (depending on your Bash - installation) `/etc/bashrc` are still sourced, so any variables set + retained. Note that (depending on your Bash + installation) `/etc/bashrc` is still sourced, so any variables set there will affect the interactive shell. - `--packages` / `-p` *packages*… @@ -230,22 +232,23 @@ terraform apply > in a nix-shell shebang. Finally, using the merging of multiple nix-shell shebangs the following -Haskell script uses a specific branch of Nixpkgs/NixOS (the 18.03 stable +Haskell script uses a specific branch of Nixpkgs/NixOS (the 20.03 stable branch): ```haskell #! /usr/bin/env nix-shell -#! nix-shell -i runghc -p "haskellPackages.ghcWithPackages (ps: [ps.HTTP ps.tagsoup])" -#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-18.03.tar.gz +#! nix-shell -i runghc -p "haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])" +#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.03.tar.gz -import Network.HTTP +import Network.Curl.Download import Text.HTML.TagSoup +import Data.Either +import Data.ByteString.Char8 (unpack) -- Fetch nixos.org and print all hrefs. main = do - resp <- Network.HTTP.simpleHTTP (getRequest "http://nixos.org/") - body <- getResponseBody resp - let tags = filter (isTagOpenName "a") $ parseTags body + resp <- openURI "https://nixos.org/" + let tags = filter (isTagOpenName "a") $ parseTags $ unpack $ fromRight undefined resp let tags' = map (fromAttrib "href") tags mapM_ putStrLn $ filter (/= "") tags' ``` diff --git a/doc/manual/src/command-ref/nix-store.md b/doc/manual/src/command-ref/nix-store.md index 827adbd05..49d06f31e 100644 --- a/doc/manual/src/command-ref/nix-store.md +++ b/doc/manual/src/command-ref/nix-store.md @@ -79,7 +79,7 @@ paths. Realisation is a somewhat overloaded term: system). If the path is already valid, we are done immediately. Otherwise, the path and any missing paths in its closure may be produced through substitutes. If there are no (successful) - subsitutes, realisation fails. + substitutes, realisation fails. The output path of each derivation is printed on standard output. (For non-derivations argument, the argument itself is printed.) @@ -226,7 +226,7 @@ control what gets deleted and in what order: or TiB units. The behaviour of the collector is also influenced by the -`keep-outputs` and `keep-derivations` variables in the Nix +`keep-outputs` and `keep-derivations` settings in the Nix configuration file. By default, the collector prints the total number of freed bytes when it diff --git a/doc/manual/src/command-ref/opt-common.md b/doc/manual/src/command-ref/opt-common.md index 9650f53f8..bc8eb6796 100644 --- a/doc/manual/src/command-ref/opt-common.md +++ b/doc/manual/src/command-ref/opt-common.md @@ -134,15 +134,6 @@ Most Nix commands accept the following command-line options: failure in obtaining the substitutes to lead to a full build from source (with the related consumption of resources). - - `--no-build-hook` - Disables the build hook mechanism. This allows to ignore remote - builders if they are setup on the machine. - - It's useful in cases where the bandwidth between the client and the - remote builder is too low. In that case it can take more time to - upload the sources to the remote builder and fetch back the result - than to do the computation locally. - - `--readonly-mode` When this option is used, no attempt is made to open the Nix database. Most Nix operations do need database access, so those diff --git a/doc/manual/src/contributing/cli-guideline.md b/doc/manual/src/contributing/cli-guideline.md new file mode 100644 index 000000000..0132867c8 --- /dev/null +++ b/doc/manual/src/contributing/cli-guideline.md @@ -0,0 +1,589 @@ +# CLI guideline + +## Goals + +Purpose of this document is to provide a clear direction to **help design +delightful command line** experience. This document contain guidelines to +follow to ensure a consistent and approachable user experience. + +## Overview + +`nix` command provides a single entry to a number of sub-commands that help +**developers and system administrators** in the life-cycle of a software +project. We particularly need to pay special attention to help and assist new +users of Nix. + +# Naming the `COMMANDS` + +Words matter. Naming is an important part of the usability. Users will be +interacting with Nix on a regular basis so we should **name things for ease of +understanding**. + +We recommend following the [Principle of Least +Astonishment](https://en.wikipedia.org/wiki/Principle_of_least_astonishment). +This means that you should **never use acronyms or abbreviations** unless they +are commonly used in other tools (e.g. `nix init`). And if the command name is +too long (> 10-12 characters) then shortening it makes sense (e.g. +“prioritization” → “priority”). + +Commands should **follow a noun-verb dialogue**. Although noun-verb formatting +seems backwards from a speaking perspective (i.e. `nix store copy` vs. `nix +copy store`) it allows us to organize commands the same way users think about +completing an action (the group first, then the command). + +## Naming rules + +Rules are there to guide you by limiting your options. But not everything can +fit the rules all the time. In those cases document the exceptions in [Appendix +1: Commands naming exceptions](#appendix-1-commands-naming-exceptions) and +provide reason. The rules want to force a Nix developer to look, not just at +the command at hand, but also the command in a full context alongside other +`nix` commands. + +```shell +$ nix [<GROUP>] <COMMAND> [<ARGUMENTS>] [<OPTIONS>] +``` + +- `GROUP`, `COMMAND`, `ARGUMENTS` and `OPTIONS` should be lowercase and in a + singular form. +- `GROUP` should be a **NOUN**. +- `COMMAND` should be a **VERB**. +- `ARGUMENTS` and `OPTIONS` are discussed in [*Input* section](#input). + +## Classification + +Some commands are more important, some less. While we want all of our commands +to be perfect we can only spend limited amount of time testing and improving +them. + +This classification tries to separate commands in 3 categories in terms of +their importance in regards to the new users. Users who are likely to be +impacted the most by bad user experience. + +- **Main commands** + + Commands used for our main use cases and most likely used by new users. We + expect attention to details, such as: + + - Proper use of [colors](#colors), [emojis](#special-unicode-characters) + and [aligning of text](#text-alignment). + - [Autocomplete](#shell-completion) of options. + - Show [next possible steps](#next-steps). + - Showing some [“tips”](#educate-the-user) when running logs running tasks + (eg. building / downloading) in order to teach users interesting bits of + Nix ecosystem. + - [Help pages](#help-is-essential) to be as good as we can write them + pointing to external documentation and tutorials for more. + + Examples of such commands: `nix init`, `nix develop`, `nix build`, `nix run`, + ... + +- **Infrequently used commands** + + From infrequently used commands we expect less attention to details, but + still some: + + - Proper use of [colors](#colors), [emojis](#special-unicode-characters) + and [aligning of text](#text-alignment). + - [Autocomplete](#shell-completion) of options. + + Examples of such commands: `nix doctor`, `nix edit`, `nix eval`, ... + +- **Utility and scripting commands** + + Commands that expose certain internal functionality of `nix`, mostly used by + other scripts. + + - [Autocomplete](#shell-completion) of options. + + Examples of such commands: `nix store copy`, `nix hash base16`, `nix store + ping`, ... + + +# Help is essential + +Help should be built into your command line so that new users can gradually +discover new features when they need them. + +## Looking for help + +Since there is no standard way how user will look for help we rely on ways help +is provided by commonly used tools. As a guide for this we took `git` and +whenever in doubt look at it as a preferred direction. + +The rules are: + +- Help is shown by using `--help` or `help` command (eg `nix` `--``help` or + `nix help`). +- For non-COMMANDs (eg. `nix` `--``help` and `nix store` `--``help`) we **show + a summary** of most common use cases. Summary is presented on the STDOUT + without any use of PAGER. +- For COMMANDs (eg. `nix init` `--``help` or `nix help init`) we display the + man page of that command. By default the PAGER is used (as in `git`). +- At the end of either summary or man page there should be an URL pointing to + an online version of more detailed documentation. +- The structure of summaries and man pages should be the same as in `git`. + +## Anticipate where help is needed + +Even better then requiring the user to search for help is to anticipate and +predict when user might need it. Either because the lack of discoverability, +typo in the input or simply taking the opportunity to teach the user of +interesting - but less visible - details. + +### Shell completion + +This type of help is most common and almost expected by users. We need to +**provide the best shell completion** for `bash`, `zsh` and `fish`. + +Completion needs to be **context aware**, this mean when a user types: + +```shell +$ nix build n<TAB> +``` + +we need to display a list of flakes starting with `n`. + +### Wrong input + +As we all know we humans make mistakes, all the time. When a typo - intentional +or unintentional - is made, we should prompt for closest possible options or +point to the documentation which would educate user to not make the same +errors. Here are few examples: + +In first example we prompt the user for typing wrong command name: + + +```shell +$ nix int +------------------------------------------------------------------------ + Error! Command `int` not found. +------------------------------------------------------------------------ + Did you mean: + |> nix init + |> nix input +``` + +Sometimes users will make mistake either because of a typo or simply because of +lack of discoverability. Our handling of this cases needs to be context +sensitive. + + +```shell +$ nix init --template=template#pyton +------------------------------------------------------------------------ + Error! Template `template#pyton` not found. +------------------------------------------------------------------------ +Initializing Nix project at `/path/to/here`. + Select a template for you new project: + |> template#pyton + template#python-pip + template#python-poetry +``` + +### Next steps + +It can be invaluable to newcomers to show what a possible next steps and what +is the usual development workflow with Nix. For example: + + +```shell +$ nix init --template=template#python +Initializing project `template#python` + in `/home/USER/dev/new-project` + + Next steps + |> nix develop -- to enter development environment + |> nix build -- to build your project +``` + +### Educate the user + +We should take any opportunity to **educate users**, but at the same time we +must **be very very careful to not annoy users**. There is a thin line between +being helpful and being annoying. + +An example of educating users might be to provide *Tips* in places where they +are waiting. + +```shell +$ nix build + Started building my-project 1.2.3 + Downloaded python3.8-poetry 1.2.3 in 5.3 seconds + Downloaded python3.8-requests 1.2.3 in 5.3 seconds +------------------------------------------------------------------------ + Press `v` to increase logs verbosity + |> `?` to see other options +------------------------------------------------------------------------ + Learn something new with every build... + |> See last logs of a build with `nix log --last` command. +------------------------------------------------------------------------ + Evaluated my-project 1.2.3 in 14.43 seconds +Downloading [12 / 200] + |> firefox 1.2.3 [#########> ] 10Mb/s | 2min left + Building [2 / 20] + |> glibc 1.2.3 -> buildPhase: <last log line> +------------------------------------------------------------------------ +``` + +Now **Learn** part of the output is where you educate users. You should only +show it when you know that a build will take some time and not annoy users of +the builds that take only few seconds. + +Every feature like this should go though a intensive review and testing to +collect as much a feedback as possible and to fine tune every little detail. If +done right this can be an awesome features beginners and advance users will +love, but if not done perfectly it will annoy users and leave bad impression. + +# Input + +Input to a command is provided via `ARGUMENTS` and `OPTIONS`. + +`ARGUMENTS` represent a required input for a function. When choosing to use +`ARGUMENT` over function please be aware of the downsides that come with it: + +- User will need to remember the order of `ARGUMENTS`. This is not a problem if + there is only one `ARGUMENT`. +- With `OPTIONS` it is possible to provide much better auto completion. +- With `OPTIONS` it is possible to provide much better error message. +- Using `OPTIONS` it will mean there is a little bit more typing. + +We don’t discourage the use of `ARGUMENTS`, but simply want to make every +developer consider the downsides and choose wisely. + +## Naming the `OPTIONS` + +Then only naming convention - apart from the ones mentioned in Naming the +`COMMANDS` section is how flags are named. + +Flags are a type of `OPTION` that represent an option that can be turned ON of +OFF. We can say **flags are boolean type of** `**OPTION**`. + +Here are few examples of flag `OPTIONS`: + +- `--colors` vs. `--no-colors` (showing colors in the output) +- `--emojis` vs. `--no-emojis` (showing emojis in the output) + +## Prompt when input not provided + +For *main commands* (as [per classification](#classification)) we want command +to improve the discoverability of possible input. A new user will most likely +not know which `ARGUMENTS` and `OPTIONS` are required or which values are +possible for those options. + +In cases, the user might not provide the input or they provide wrong input, +rather then show the error, prompt a user with an option to find and select +correct input (see examples). + +Prompting is of course not required when TTY is not attached to STDIN. This +would mean that scripts wont need to handle prompt, but rather handle errors. + +A place to use prompt and provide user with interactive select + + +```shell +$ nix init +Initializing Nix project at `/path/to/here`. + Select a template for you new project: + |> py + template#python-pip + template#python-poetry + [ Showing 2 templates from 1345 templates ] +``` + +Another great place to add prompts are **confirmation dialogues for dangerous +actions**. For example when adding new substitutor via `OPTIONS` or via +`flake.nix` we should prompt - for the first time - and let user review what is +going to happen. + + +```shell +$ nix build --option substitutors https://cache.example.org +------------------------------------------------------------------------ + Warning! A security related question need to be answered. +------------------------------------------------------------------------ + The following substitutors will be used to in `my-project`: + - https://cache.example.org + + Do you allow `my-project` to use above mentioned substitutors? + [y/N] |> y +``` + +# Output + +Terminal output can be quite limiting in many ways. Which should forces us to +think about the experience even more. As with every design the output is a +compromise between being terse and being verbose, between showing help to +beginners and annoying advance users. For this it is important that we know +what are the priorities. + +Nix command line should be first and foremost written with beginners in mind. +But users wont stay beginners for long and what was once useful might quickly +become annoying. There is no golden rule that we can give in this guideline +that would make it easier how to draw a line and find best compromise. + +What we would encourage is to **build prototypes**, do some **user testing** +and collect **feedback**. Then repeat the cycle few times. + +First design the *happy path* and only after your iron it out, continue to work +on **edge cases** (handling and displaying errors, changes of the output by +certain `OPTIONS`, etc…) + +## Follow best practices + +Needless to say we Nix must be a good citizen and follow best practices in +command line. + +In short: **STDOUT is for output, STDERR is for (human) messaging.** + +STDOUT and STDERR provide a way for you to output messages to the user while +also allowing them to redirect content to a file. For example: + +```shell +$ nix build > build.txt +------------------------------------------------------------------------ + Error! Atrribute `bin` missing at (1:94) from string. +------------------------------------------------------------------------ + + 1| with import <nixpkgs> { }; (pkgs.runCommandCC or pkgs.runCommand) "shell" { buildInputs = [ (surge.bin) ]; } "" +``` + +Because this warning is on STDERR, it doesn’t end up in the file. + +But not everything on STDERR is an error though. For example, you can run `nix +build` and collect logs in a file while still seeing the progress. + +``` +$ nix build > build.txt + Evaluated 1234 files in 1.2 seconds + Downloaded python3.8-poetry 1.2.3 in 5.3 seconds + Downloaded python3.8-requests 1.2.3 in 5.3 seconds +------------------------------------------------------------------------ + Press `v` to increase logs verbosity + |> `?` to see other options +------------------------------------------------------------------------ + Learn something new with every build... + |> See last logs of a build with `nix log --last` command. +------------------------------------------------------------------------ + Evaluated my-project 1.2.3 in 14.43 seconds +Downloading [12 / 200] + |> firefox 1.2.3 [#########> ] 10Mb/s | 2min left + Building [2 / 20] + |> glibc 1.2.3 -> buildPhase: <last log line> +------------------------------------------------------------------------ +``` + +## Errors (WIP) + +**TODO**: Once we have implementation for the *happy path* then we will think +how to present errors. + +## Not only for humans + +Terse, machine-readable output formats can also be useful but shouldn’t get in +the way of making beautiful CLI output. When needed, commands should offer a +`--json` flag to allow users to easily parse and script the CLI. + +When TTY is not detected on STDOUT we should remove all design elements (no +colors, no emojis and using ASCII instead of Unicode symbols). The same should +happen when TTY is not detected on STDERR. We should not display progress / +status section, but only print warnings and errors. + +## Dialog with the user + +CLIs don't always make it clear when an action has taken place. For every +action a user performs, your CLI should provide an equal and appropriate +reaction, clearly highlighting the what just happened. For example: + +```shell +$ nix build + Downloaded python3.8-poetry 1.2.3 in 5.3 seconds + Downloaded python3.8-requests 1.2.3 in 5.3 seconds +... + Success! You have successfully built my-project. +$ +``` + +Above command clearly states that command successfully completed. And in case +of `nix build`, which is a command that might take some time to complete, it is +equally important to also show that a command started. + +## Text alignment + +Text alignment is the number one design element that will present all of the +Nix commands as a family and not as separate tools glued together. + +The format we should follow is: + +```shell +$ nix COMMAND + VERB_1 NOUN and other words + VERB__1 NOUN and other words + |> Some details +``` + +Few rules that we can extract from above example: + +- Each line should start at least with one space. +- First word should be a VERB and must be aligned to the right. +- Second word should be a NOUN and must be aligned to the left. +- If you can not find a good VERB / NOUN pair, don’t worry make it as + understandable to the user as possible. +- More details of each line can be provided by `|>` character which is serving + as the first word when aligning the text + +Don’t forget you should also test your terminal output with colors and emojis +off (`--no-colors --no-emojis`). + +## Dim / Bright + +After comparing few terminals with different color schemes we would **recommend +to avoid using dimmed text**. The difference from the rest of the text is very +little in many terminal and color scheme combinations. Sometimes the difference +is not even notable, therefore relying on it wouldn’t make much sense. + +**The bright text is much better supported** across terminals and color +schemes. Most of the time the difference is perceived as if the bright text +would be bold. + +## Colors + +Humans are already conditioned by society to attach certain meaning to certain +colors. While the meaning is not universal, a simple collection of colors is +used to represent basic emotions. + +Colors that can be used in output + +- Red = error, danger, stop +- Green = success, good +- Yellow/Orange = proceed with caution, warning, in progress +- Blue/Magenta = stability, calm + +While colors are nice, when command line is used by machines (in automation +scripts) you want to remove the colors. There should be a global `--no-colors` +option that would remove the colors. + +## Special (Unicode) characters + +Most of the terminal have good support for Unicode characters and you should +use them in your output by default. But always have a backup solution that is +implemented only with ASCII characters and will be used when `--ascii` option +is going to be passed in. Please make sure that you test your output also +without Unicode characters + +More they showing all the different Unicode characters it is important to +**establish common set of characters** that we use for certain situations. + +## Emojis + +Emojis help channel emotions even better than text, colors and special +characters. + +We recommend **keeping the set of emojis to a minimum**. This will enable each +emoji to stand out more. + +As not everybody is happy about emojis we should provide an `--no-emojis` +option to disable them. Please make sure that you test your output also without +emojis. + +## Tables + +All commands that are listing certain data can be implemented in some sort of a +table. It’s important that each row of your output is a single ‘entry’ of data. +Never output table borders. It’s noisy and a huge pain for parsing using other +tools such as `grep`. + +Be mindful of the screen width. Only show a few columns by default with the +table header, for more the table can be manipulated by the following options: + +- `--no-headers`: Show column headers by default but allow to hide them. +- `--columns`: Comma-separated list of column names to add. +- `--sort`: Allow sorting by column. Allow inverse and multi-column sort as well. + +## Interactive output + +Interactive output was selected to be able to strike the balance between +beginners and advance users. While the default output will target beginners it +can, with a few key strokes, be changed into and advance introspection tool. + +### Progress + +For longer running commands we should provide and overview of the progress. +This is shown best in `nix build` example: + +```shell +$ nix build + Started building my-project 1.2.3 + Downloaded python3.8-poetry 1.2.3 in 5.3 seconds + Downloaded python3.8-requests 1.2.3 in 5.3 seconds +------------------------------------------------------------------------ + Press `v` to increase logs verbosity + |> `?` to see other options +------------------------------------------------------------------------ + Learn something new with every build... + |> See last logs of a build with `nix log --last` command. +------------------------------------------------------------------------ + Evaluated my-project 1.2.3 in 14.43 seconds +Downloading [12 / 200] + |> firefox 1.2.3 [#########> ] 10Mb/s | 2min left + Building [2 / 20] + |> glibc 1.2.3 -> buildPhase: <last log line> +------------------------------------------------------------------------ +``` + +### Search + +Use a `fzf` like fuzzy search when there are multiple options to choose from. + +```shell +$ nix init +Initializing Nix project at `/path/to/here`. + Select a template for you new project: + |> py + template#python-pip + template#python-poetry + [ Showing 2 templates from 1345 templates ] +``` + +### Prompt + +In some situations we need to prompt the user and inform the user about what is +going to happen. + +```shell +$ nix build --option substitutors https://cache.example.org +------------------------------------------------------------------------ + Warning! A security related question need to be answered. +------------------------------------------------------------------------ + The following substitutors will be used to in `my-project`: + - https://cache.example.org + + Do you allow `my-project` to use above mentioned substitutors? + [y/N] |> y +``` + +## Verbosity + +There are many ways that you can control verbosity. + +Verbosity levels are: + +- `ERROR` (level 0) +- `WARN` (level 1) +- `NOTICE` (level 2) +- `INFO` (level 3) +- `TALKATIVE` (level 4) +- `CHATTY` (level 5) +- `DEBUG` (level 6) +- `VOMIT` (level 7) + +The default level that the command starts is `ERROR`. The simplest way to +increase the verbosity by stacking `-v` option (eg: `-vvv == level 3 == INFO`). +There are also two shortcuts, `--debug` to run in `DEBUG` verbosity level and +`--quiet` to run in `ERROR` verbosity level. + +---------- + +# Appendix 1: Commands naming exceptions + +`nix init` and `nix repl` are well established diff --git a/doc/manual/src/contributing/contributing.md b/doc/manual/src/contributing/contributing.md new file mode 100644 index 000000000..854139a31 --- /dev/null +++ b/doc/manual/src/contributing/contributing.md @@ -0,0 +1 @@ +# Contributing diff --git a/doc/manual/src/hacking.md b/doc/manual/src/contributing/hacking.md index 2a1e55e5b..2a1e55e5b 100644 --- a/doc/manual/src/hacking.md +++ b/doc/manual/src/contributing/hacking.md diff --git a/doc/manual/src/expressions/language-operators.md b/doc/manual/src/expressions/language-operators.md index 1d787ffe3..b7fd6f4c6 100644 --- a/doc/manual/src/expressions/language-operators.md +++ b/doc/manual/src/expressions/language-operators.md @@ -25,5 +25,4 @@ order of precedence (from strongest to weakest binding). | Inequality | *e1* `!=` *e2* | none | Inequality. | 11 | | Logical AND | *e1* `&&` *e2* | left | Logical AND. | 12 | | Logical OR | *e1* `\|\|` *e2* | left | Logical OR. | 13 | -| Logical Implication | *e1* `->` *e2* | none | Logical implication (equivalent to `!e1 \|\| - e2`). | 14 | +| Logical Implication | *e1* `->` *e2* | none | Logical implication (equivalent to `!e1 \|\| e2`). | 14 | diff --git a/doc/manual/src/installation/installing-binary.md b/doc/manual/src/installation/installing-binary.md index 8b8d1d738..ae7fd458b 100644 --- a/doc/manual/src/installation/installing-binary.md +++ b/doc/manual/src/installation/installing-binary.md @@ -195,7 +195,7 @@ If you are comfortable navigating these tradeoffs, you can encrypt the volume with something along the lines of: ```console -alice$ diskutil apfs enableFileVault /nix -user disk +$ diskutil apfs enableFileVault /nix -user disk ``` ## Symlink the Nix store to a custom location @@ -234,13 +234,13 @@ as a helpful reference if you run into trouble. without a reboot: ```console - alice$ /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util -B + $ /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util -B ``` 3. Create the new APFS volume with diskutil: ```console - alice$ sudo diskutil apfs addVolume diskX APFS 'Nix Store' -mountpoint /nix + $ sudo diskutil apfs addVolume diskX APFS 'Nix Store' -mountpoint /nix ``` 4. Using `vifs`, add the new mount to `/etc/fstab`. If it doesn't @@ -280,10 +280,10 @@ it somewhere (e.g. in `/tmp`), and then run the script named `install` inside the binary tarball: ```console -alice$ cd /tmp -alice$ tar xfj nix-1.8-x86_64-darwin.tar.bz2 -alice$ cd nix-1.8-x86_64-darwin -alice$ ./install +$ cd /tmp +$ tar xfj nix-1.8-x86_64-darwin.tar.bz2 +$ cd nix-1.8-x86_64-darwin +$ ./install ``` If you need to edit the multi-user installation script to use different diff --git a/doc/manual/src/installation/prerequisites-source.md b/doc/manual/src/installation/prerequisites-source.md index 69b7c5a5e..12758c5e1 100644 --- a/doc/manual/src/installation/prerequisites-source.md +++ b/doc/manual/src/installation/prerequisites-source.md @@ -30,7 +30,7 @@ have bzip2 installed, including development headers and libraries. If your distribution does not provide these, you can obtain bzip2 from - <https://web.archive.org/web/20180624184756/http://www.bzip.org/>. + <https://sourceware.org/bzip2/>. - `liblzma`, which is provided by XZ Utils. If your distribution does not provide this, you can get it from <https://tukaani.org/xz/>. @@ -69,3 +69,6 @@ `--disable-seccomp-sandboxing` option to the `configure` script (Not recommended unless your system doesn't support `libseccomp`). To get the library, visit <https://github.com/seccomp/libseccomp>. + + - Niels Lohmann's [JSON library](https://github.com/nlohmann/json). + diff --git a/doc/manual/src/introduction.md b/doc/manual/src/introduction.md index f01fe7b38..d68445c95 100644 --- a/doc/manual/src/introduction.md +++ b/doc/manual/src/introduction.md @@ -165,10 +165,10 @@ You’re then dropped into a shell where you can edit, build and test the package: ```console -[nix-shell]$ tar xf $src +[nix-shell]$ unpackPhase [nix-shell]$ cd pan-* -[nix-shell]$ ./configure -[nix-shell]$ make +[nix-shell]$ configurePhase +[nix-shell]$ buildPhase [nix-shell]$ ./pan/gui/pan ``` diff --git a/doc/manual/src/release-notes/rl-2.4.md b/doc/manual/src/release-notes/rl-2.4.md new file mode 100644 index 000000000..f7ab9f6ad --- /dev/null +++ b/doc/manual/src/release-notes/rl-2.4.md @@ -0,0 +1,8 @@ +# Release 2.4 (202X-XX-XX) + + - It is now an error to modify the `plugin-files` setting via a + command-line flag that appears after the first non-flag argument + to any command, including a subcommand to `nix`. For example, + `nix-instantiate default.nix --plugin-files ""` must now become + `nix-instantiate --plugin-files "" default.nix`. + - Plugins that add new `nix` subcommands are now actually respected. diff --git a/doc/manual/utils.nix b/doc/manual/utils.nix index 50150bf3e..d4b18472f 100644 --- a/doc/manual/utils.nix +++ b/doc/manual/utils.nix @@ -1,7 +1,15 @@ with builtins; -{ +rec { splitLines = s: filter (x: !isList x) (split "\n" s); concatStrings = concatStringsSep ""; + + # FIXME: O(n^2) + unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) []; + + nameValuePair = name: value: { inherit name value; }; + + filterAttrs = pred: set: + listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set)); } |