Merge pull request #8141 from tweag/user-files-doc

Document user files of nix

8 files changed, 336 insertions, 70 deletions

diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in
index 1d5613354..606aecd8f 100644
--- a/doc/manual/src/SUMMARY.md.in
+++ b/doc/manual/src/SUMMARY.md.in
@@ -92,6 +92,11 @@
 {{#include ./command-ref/new-cli/SUMMARY.md}}
   - [Files](command-ref/files.md)
     - [nix.conf](command-ref/conf-file.md)
+    - [Profiles](command-ref/files/profiles.md)
+      - [manifest.nix](command-ref/files/manifest.nix.md)
+      - [manifest.json](command-ref/files/manifest.json.md)
+    - [Channels](command-ref/files/channels.md)
+    - [Default Nix expression](command-ref/files/default-nix-expression.md)
 - [Architecture](architecture/architecture.md)
 - [Glossary](glossary.md)
 - [Contributing](contributing/contributing.md)
diff --git a/doc/manual/src/command-ref/files/channels.md b/doc/manual/src/command-ref/files/channels.md
new file mode 100644
index 000000000..7b1f27128
--- /dev/null
+++ b/doc/manual/src/command-ref/files/channels.md
@@ -0,0 +1,26 @@
+## Channels
+
+A directory containing symlinks to Nix channels, managed by [`nix-channel`]:
+
+- `$XDG_STATE_HOME/nix/profiles/channels` for regular users
+- `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root`
+
+[`nix-channel`] uses a [profile](@docroot@/command-ref/files/profiles.md) to store channels.
+This profile contains symlinks to the contents of those channels.
+
+## Subscribed channels
+
+The list of subscribed channels is stored in
+
+- `~/.nix-channels`
+- `$XDG_STATE_HOME/nix/channels` if [`use-xdg-base-directories`] is set to `true`
+
+in the following format:
+
+```
+<url> <name>
+...
+```
+
+[`nix-channel`]: @docroot@/command-ref/nix-channel.md
+[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
diff --git a/doc/manual/src/command-ref/files/default-nix-expression.md b/doc/manual/src/command-ref/files/default-nix-expression.md
new file mode 100644
index 000000000..620f7035c
--- /dev/null
+++ b/doc/manual/src/command-ref/files/default-nix-expression.md
@@ -0,0 +1,52 @@
+## Default Nix expression
+
+The source for the default [Nix expressions](@docroot@/language/index.md) used by [`nix-env`]:
+
+- `~/.nix-defexpr`
+- `$XDG_STATE_HOME/nix/defexpr` if [`use-xdg-base-directories`] is set to `true`.
+
+It is loaded as follows:
+
+- If the default expression is a file, it is loaded as a Nix expression.
+- If the default expression is a directory containing a `default.nix` file, that `default.nix` file is loaded as a Nix expression.
+- If the default expression is a directory without a `default.nix` file, then its contents (both files and subdirectories) are loaded as Nix expressions.
+  The expressions are combined into a single attribute set, each expression under an attribute with the same name as the original file or subdirectory.
+  Subdirectories without a `default.nix` file are traversed recursively in search of more Nix expressions, but the names of these intermediate directories are not added to the attribute paths of the default Nix expression.
+
+Then, the resulting expression is interpreted like this:
+
+- If the expression is an attribute set, it is used as the default Nix expression.
+- If the expression is a function, an empty set is passed as argument and the return value is used as the default Nix expression.
+
+
+For example, if the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to
+
+```nix
+{
+  foo = import ~/.nix-defexpr/foo.nix;
+  bar = import ~/.nix-defexpr/bar.nix;
+}
+```
+
+The file [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) is always ignored.
+
+The command [`nix-channel`] places a symlink to the user's current [channels profile](@docroot@/command-ref/files/channels.md) in this directory.
+This makes all subscribed channels available as attributes in the default expression.
+
+## User channel link
+
+A symlink that ensures that [`nix-env`] can find your channels:
+
+- `~/.nix-defexpr/channels`
+- `$XDG_STATE_HOME/defexpr/channels` if [`use-xdg-base-directories`] is set to `true`.
+
+This symlink points to:
+
+- `$XDG_STATE_HOME/profiles/channels` for regular users
+- `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root`
+
+In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user.[`nix-env`]: ../nix-env.md
+
+[`nix-env`]: @docroot@/command-ref/nix-env.md
+[`nix-channel`]: @docroot@/command-ref/nix-channel.md
+[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
diff --git a/doc/manual/src/command-ref/files/manifest.json.md b/doc/manual/src/command-ref/files/manifest.json.md
new file mode 100644
index 000000000..bcfe7373d
--- /dev/null
+++ b/doc/manual/src/command-ref/files/manifest.json.md
@@ -0,0 +1,45 @@
+## `manifest.json`
+
+The manifest file records the provenance of the packages that are installed in a [profile](./profiles.md) managed by [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md) (experimental).
+
+Here is an example of what the file might look like after installing `zoom-us` from Nixpkgs:
+
+```json
+{
+  "version": 1,
+  "elements": [
+    {
+      "active": true,
+      "attrPath": "legacyPackages.x86_64-linux.zoom-us",
+      "originalUrl": "flake:nixpkgs",
+      "storePaths": [
+        "/nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927"
+      ],
+      "uri": "github:NixOS/nixpkgs/13d0c311e3ae923a00f734b43fd1d35b47d8943a"
+    },
+    …
+  ]
+}
+```
+
+Each object in the array `elements` denotes an installed package and
+has the following fields:
+
+* `originalUrl`: The [flake reference](@docroot@/command-ref/new-cli/nix3-flake.md) specified by
+  the user at the time of installation (e.g. `nixpkgs`). This is also
+  the flake reference that will be used by `nix profile upgrade`.
+
+* `uri`: The locked flake reference to which `originalUrl` resolved.
+
+* `attrPath`: The flake output attribute that provided this
+  package. Note that this is not necessarily the attribute that the
+  user specified, but the one resulting from applying the default
+  attribute paths and prefixes; for instance, `hello` might resolve to
+  `packages.x86_64-linux.hello` and the empty string to
+  `packages.x86_64-linux.default`.
+
+* `storePath`: The paths in the Nix store containing the package.
+
+* `active`: Whether the profile contains symlinks to the files of this
+  package. If set to false, the package is kept in the Nix store, but
+  is not "visible" in the profile's symlink tree.
diff --git a/doc/manual/src/command-ref/files/manifest.nix.md b/doc/manual/src/command-ref/files/manifest.nix.md
new file mode 100644
index 000000000..d7d1b605b
--- /dev/null
+++ b/doc/manual/src/command-ref/files/manifest.nix.md
@@ -0,0 +1,128 @@
+## `manifest.nix`
+
+The manifest file records the provenance of the packages that are installed in a [profile](./profiles.md) managed by [`nix-env`](@docroot@/command-ref/nix-env.md).
+
+Here is an example of how this file might look like after installing `hello` from Nixpkgs:
+
+```nix
+[{
+  meta = {
+    available = true;
+    broken = false;
+    changelog =
+      "https://git.savannah.gnu.org/cgit/hello.git/plain/NEWS?h=v2.12.1";
+    description = "A program that produces a familiar, friendly greeting";
+    homepage = "https://www.gnu.org/software/hello/manual/";
+    insecure = false;
+    license = {
+      deprecated = false;
+      free = true;
+      fullName = "GNU General Public License v3.0 or later";
+      redistributable = true;
+      shortName = "gpl3Plus";
+      spdxId = "GPL-3.0-or-later";
+      url = "https://spdx.org/licenses/GPL-3.0-or-later.html";
+    };
+    longDescription = ''
+      GNU Hello is a program that prints "Hello, world!" when you run it.
+      It is fully customizable.
+    '';
+    maintainers = [{
+      email = "edolstra+nixpkgs@gmail.com";
+      github = "edolstra";
+      githubId = 1148549;
+      name = "Eelco Dolstra";
+    }];
+    name = "hello-2.12.1";
+    outputsToInstall = [ "out" ];
+    platforms = [
+      "i686-cygwin"
+      "x86_64-cygwin"
+      "x86_64-darwin"
+      "i686-darwin"
+      "aarch64-darwin"
+      "armv7a-darwin"
+      "i686-freebsd13"
+      "x86_64-freebsd13"
+      "aarch64-genode"
+      "i686-genode"
+      "x86_64-genode"
+      "x86_64-solaris"
+      "js-ghcjs"
+      "aarch64-linux"
+      "armv5tel-linux"
+      "armv6l-linux"
+      "armv7a-linux"
+      "armv7l-linux"
+      "i686-linux"
+      "m68k-linux"
+      "microblaze-linux"
+      "microblazeel-linux"
+      "mipsel-linux"
+      "mips64el-linux"
+      "powerpc64-linux"
+      "powerpc64le-linux"
+      "riscv32-linux"
+      "riscv64-linux"
+      "s390-linux"
+      "s390x-linux"
+      "x86_64-linux"
+      "mmix-mmixware"
+      "aarch64-netbsd"
+      "armv6l-netbsd"
+      "armv7a-netbsd"
+      "armv7l-netbsd"
+      "i686-netbsd"
+      "m68k-netbsd"
+      "mipsel-netbsd"
+      "powerpc-netbsd"
+      "riscv32-netbsd"
+      "riscv64-netbsd"
+      "x86_64-netbsd"
+      "aarch64_be-none"
+      "aarch64-none"
+      "arm-none"
+      "armv6l-none"
+      "avr-none"
+      "i686-none"
+      "microblaze-none"
+      "microblazeel-none"
+      "msp430-none"
+      "or1k-none"
+      "m68k-none"
+      "powerpc-none"
+      "powerpcle-none"
+      "riscv32-none"
+      "riscv64-none"
+      "rx-none"
+      "s390-none"
+      "s390x-none"
+      "vc4-none"
+      "x86_64-none"
+      "i686-openbsd"
+      "x86_64-openbsd"
+      "x86_64-redox"
+      "wasm64-wasi"
+      "wasm32-wasi"
+      "x86_64-windows"
+      "i686-windows"
+    ];
+    position =
+      "/nix/store/7niq32w715567hbph0q13m5lqna64c1s-nixos-unstable.tar.gz/nixos-unstable.tar.gz/pkgs/applications/misc/hello/default.nix:34";
+    unfree = false;
+    unsupported = false;
+  };
+  name = "hello-2.12.1";
+  out = {
+    outPath = "/nix/store/260q5867crm1xjs4khgqpl6vr9kywql1-hello-2.12.1";
+  };
+  outPath = "/nix/store/260q5867crm1xjs4khgqpl6vr9kywql1-hello-2.12.1";
+  outputs = [ "out" ];
+  system = "x86_64-linux";
+  type = "derivation";
+}]
+```
+
+Each element in this list corresponds to an installed package.
+It incorporates some attributes of the original derivation, including `meta`, `name`, `out`, `outPath`, `outputs`, `system`.
+This information is used by Nix for querying and updating the package.
diff --git a/doc/manual/src/command-ref/files/profiles.md b/doc/manual/src/command-ref/files/profiles.md
new file mode 100644
index 000000000..b5c737880
--- /dev/null
+++ b/doc/manual/src/command-ref/files/profiles.md
@@ -0,0 +1,74 @@
+## Profiles
+
+A directory that contains links to profiles managed by [`nix-env`] and [`nix profile`]:
+
+- `$XDG_STATE_HOME/nix/profiles` for regular users
+- `$NIX_STATE_DIR/profiles/per-user/root` if the user is `root`
+
+A profile is a directory of symlinks to files in the Nix store.
+
+### Filesystem layout
+
+Profiles are versioned as follows. When using a profile named *path*, *path* is a symlink to *path*`-`*N*`-link`, where *N* is the version of the profile.
+In turn, *path*`-`*N*`-link` is a symlink to a path in the Nix store.
+For example:
+
+```console
+$ ls -l ~alice/.local/state/nix/profiles/profile*
+lrwxrwxrwx 1 alice users 14 Nov 25 14:35 /home/alice/.local/state/nix/profiles/profile -> profile-7-link
+lrwxrwxrwx 1 alice users 51 Oct 28 16:18 /home/alice/.local/state/nix/profiles/profile-5-link -> /nix/store/q69xad13ghpf7ir87h0b2gd28lafjj1j-profile
+lrwxrwxrwx 1 alice users 51 Oct 29 13:20 /home/alice/.local/state/nix/profiles/profile-6-link -> /nix/store/6bvhpysd7vwz7k3b0pndn7ifi5xr32dg-profile
+lrwxrwxrwx 1 alice users 51 Nov 25 14:35 /home/alice/.local/state/nix/profiles/profile-7-link -> /nix/store/mp0x6xnsg0b8qhswy6riqvimai4gm677-profile
+```
+
+Each of these symlinks is a root for the Nix garbage collector.
+
+The contents of the store path corresponding to each version of the
+profile is a tree of symlinks to the files of the installed packages,
+e.g.
+
+```console
+$ ll -R ~eelco/.local/state/nix/profiles/profile-7-link/
+/home/eelco/.local/state/nix/profiles/profile-7-link/:
+total 20
+dr-xr-xr-x 2 root root 4096 Jan  1  1970 bin
+-r--r--r-- 2 root root 1402 Jan  1  1970 manifest.nix
+dr-xr-xr-x 4 root root 4096 Jan  1  1970 share
+
+/home/eelco/.local/state/nix/profiles/profile-7-link/bin:
+total 20
+lrwxrwxrwx 5 root root 79 Jan  1  1970 chromium -> /nix/store/ijm5k0zqisvkdwjkc77mb9qzb35xfi4m-chromium-86.0.4240.111/bin/chromium
+lrwxrwxrwx 7 root root 87 Jan  1  1970 spotify -> /nix/store/w9182874m1bl56smps3m5zjj36jhp3rn-spotify-1.1.26.501.gbe11e53b-15/bin/spotify
+lrwxrwxrwx 3 root root 79 Jan  1  1970 zoom-us -> /nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927/bin/zoom-us
+
+/home/eelco/.local/state/nix/profiles/profile-7-link/share/applications:
+total 12
+lrwxrwxrwx 4 root root 120 Jan  1  1970 chromium-browser.desktop -> /nix/store/4cf803y4vzfm3gyk3vzhzb2327v0kl8a-chromium-unwrapped-86.0.4240.111/share/applications/chromium-browser.desktop
+lrwxrwxrwx 7 root root 110 Jan  1  1970 spotify.desktop -> /nix/store/w9182874m1bl56smps3m5zjj36jhp3rn-spotify-1.1.26.501.gbe11e53b-15/share/applications/spotify.desktop
+lrwxrwxrwx 3 root root 107 Jan  1  1970 us.zoom.Zoom.desktop -> /nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927/share/applications/us.zoom.Zoom.desktop
+
+…
+```
+
+Each profile version contains a manifest file:
+- [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) used by [`nix-env`](@docroot@/command-ref/nix-env.md).
+- [`manifest.json`](@docroot@/command-ref/files/manifest.json.md) used by [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md) (experimental).
+
+## User profile link
+
+A symbolic link to the user's current profile:
+
+- `~/.nix-profile`
+- `$XDG_STATE_HOME/nix/profile` if [`use-xdg-base-directories`] is set to `true`.
+
+By default, this symlink points to:
+
+- `$XDG_STATE_HOME/nix/profiles/profile` for regular users
+- `$NIX_STATE_DIR/profiles/per-user/root/profile` for `root`
+
+The `PATH` environment variable should include `/bin` subdirectory of the profile link (e.g. `~/.nix-profile/bin`) for the user environment to be visible to the user.
+The [installer](@docroot@/installation/installing-binary.md) sets this up by default, unless you enable [`use-xdg-base-directories`].
+
+[`nix-env`]: @docroot@/command-ref/nix-env.md
+[`nix profile`]: @docroot@/command-ref/new-cli/nix3-profile.md
+[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
diff --git a/doc/manual/src/command-ref/nix-channel.md b/doc/manual/src/command-ref/nix-channel.md
index 2027cc98d..72d3e422b 100644
--- a/doc/manual/src/command-ref/nix-channel.md
+++ b/doc/manual/src/command-ref/nix-channel.md
@@ -22,6 +22,9 @@ This command has the following operations:
     channels. If *name* is omitted, it defaults to the last component of
     *url*, with the suffixes `-stable` or `-unstable` removed.
 
+    A channel URL must point to a directory containing a file `nixexprs.tar.gz`.
+    At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point.
+
   - `--remove` *name*\
     Removes the channel named *name* from the list of subscribed
     channels.
@@ -71,30 +74,3 @@ switching from generation 483 to 482
 $ nix-instantiate --eval -E '(import <nixpkgs> {}).lib.version'
 "14.04.526.dbadfad"
 ```
-
-# Files
-
-  - `${XDG_STATE_HOME-$HOME/.local/state}/nix/profiles/channels`\
-    `nix-channel` uses a `nix-env` profile to keep track of previous
-    versions of the subscribed channels. Every time you run `nix-channel
-    --update`, a new channel generation (that is, a symlink to the
-    channel Nix expressions in the Nix store) is created. This enables
-    `nix-channel --rollback` to revert to previous versions.
-
-  - `~/.nix-defexpr/channels`\
-    This is a symlink to
-    `${XDG_STATE_HOME-$HOME/.local/state}/nix/profiles/channels`. It ensures that
-    `nix-env` can find your channels. In a multi-user installation, you
-    may also have `~/.nix-defexpr/channels_root`, which links to the
-    channels of the root user.
-
-# Channel format
-
-A channel URL should point to a directory containing the following
-files:
-
-  - `nixexprs.tar.xz`\
-    A tarball containing Nix expressions and files referenced by them
-    (such as build scripts and patches). At the top level, the tarball
-    should contain a single directory. That directory must contain a
-    file `default.nix` that serves as the channel’s “entry point”.
diff --git a/doc/manual/src/command-ref/nix-env.md b/doc/manual/src/command-ref/nix-env.md
index 42b5bca77..b4a3dce49 100644
--- a/doc/manual/src/command-ref/nix-env.md
+++ b/doc/manual/src/command-ref/nix-env.md
@@ -83,46 +83,6 @@ match. Here are some examples:
 
 # Files
 
-  - `~/.nix-defexpr`\
-    The source for the default Nix expressions used by the
-    `--install`, `--upgrade`, and `--query --available` operations to
-    obtain derivations. The `--file` option may be used to override
-    this default.
-
-    If `~/.nix-defexpr` is a file, it is loaded as a Nix expression. If
-    the expression is a set, it is used as the default Nix expression.
-    If the expression is a function, an empty set is passed as argument
-    and the return value is used as the default Nix expression.
-
-    If `~/.nix-defexpr` is a directory containing a `default.nix` file,
-    that file is loaded as in the above paragraph.
-
-    If `~/.nix-defexpr` is a directory without a `default.nix` file,
-    then its contents (both files and subdirectories) are loaded as Nix
-    expressions. The expressions are combined into a single set, each
-    expression under an attribute with the same name as the original
-    file or subdirectory.
-
-    For example, if `~/.nix-defexpr` contains two files, `foo.nix` and
-    `bar.nix`, then the default Nix expression will essentially be
-
-    ```nix
-    {
-      foo = import ~/.nix-defexpr/foo.nix;
-      bar = import ~/.nix-defexpr/bar.nix;
-    }
-    ```
-
-    The file `manifest.nix` is always ignored. Subdirectories without a
-    `default.nix` file are traversed recursively in search of more Nix
-    expressions, but the names of these intermediate directories are not
-    added to the attribute paths of the default Nix expression.
-
-    The command `nix-channel` places symlinks to the downloaded Nix
-    expressions from each subscribed channel in this directory.
-
-  - `~/.nix-profile`\
-    A symbolic link to the user's current profile. By default, this
-    symlink points to `prefix/var/nix/profiles/default`. The `PATH`
-    environment variable should include `~/.nix-profile/bin` for the
-    user environment to be visible to the user.
+{{#include ./files/default-nix-expression.md}}
+
+{{#include ./files/profiles.md}}