Merge remote-tracking branch 'upstream/master' into path-info

5 files changed, 129 insertions, 11 deletions

diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in
index 4b654567f..5bf274550 100644
--- a/doc/manual/src/SUMMARY.md.in
+++ b/doc/manual/src/SUMMARY.md.in
@@ -95,6 +95,7 @@
 - [Glossary](glossary.md)
 - [Contributing](contributing/contributing.md)
   - [Hacking](contributing/hacking.md)
+  - [Experimental Features](contributing/experimental-features.md)
   - [CLI guideline](contributing/cli-guideline.md)
 - [Release Notes](release-notes/release-notes.md)
   - [Release X.Y (202?-??-??)](release-notes/rl-next.md)
diff --git a/doc/manual/src/command-ref/nix-shell.md b/doc/manual/src/command-ref/nix-shell.md
index 9f36929f7..576e5ba0b 100644
--- a/doc/manual/src/command-ref/nix-shell.md
+++ b/doc/manual/src/command-ref/nix-shell.md
@@ -120,7 +120,8 @@ shell in which to build it:
 ```console
 $ nix-shell '<nixpkgs>' -A pan
 [nix-shell]$ eval ${unpackPhase:-unpackPhase}
-[nix-shell]$ cd pan-*
+[nix-shell]$ cd $sourceRoot
+[nix-shell]$ eval ${patchPhase:-patchPhase}
 [nix-shell]$ eval ${configurePhase:-configurePhase}
 [nix-shell]$ eval ${buildPhase:-buildPhase}
 [nix-shell]$ ./pan/gui/pan
diff --git a/doc/manual/src/contributing/experimental-features.md b/doc/manual/src/contributing/experimental-features.md
new file mode 100644
index 000000000..f1db22751
--- /dev/null
+++ b/doc/manual/src/contributing/experimental-features.md
@@ -0,0 +1,91 @@
+This section describes the notion of *experimental features*, and how it fits into the big picture of the development of Nix.
+
+# What are experimental features?
+
+Experimental features are considered unstable, which means that they can be changed or removed at any time.
+Users must explicitly enable them by toggling the associated [experimental feature flags](@docroot@/command-ref/conf-file.md#conf-experimental-features).
+This allows accessing unstable functionality without unwittingly relying on it.
+
+Experimental feature flags were first introduced in [Nix 2.4](@docroot@/release-notes/rl-2.4.md).
+Before that, Nix did have experimental features, but they were not guarded by flags and were merely documented as unstable.
+This was a source of confusion and controversy.
+
+# When should a new feature be marked experimental?
+
+A change in the Nix codebase should be guarded by an experimental feature flag if it is considered likely to be reverted or adapted in a backwards-incompatible manner after gathering more experience with it in practice.
+
+Examples:
+
+- Changes to the Nix language, such as new built-ins, syntactic or semantic changes, etc.
+- Changes to the command-line interface
+
+# Lifecycle of an experimental feature
+
+Experimental features have to be treated on a case-by-case basis.
+However, the standard workflow for an experimental feature is as follows:
+
+- A new feature is implemented in a *pull request*
+  - It is guarded by an experimental feature flag that is disabled by default
+- The pull request is merged, the *experimental* feature ends up in a release
+    - Using the feature requires explicitly enabling it, signifying awareness of the potential risks
+    - Being experimental, the feature can still be changed arbitrarily
+- The feature can be *removed*
+  - The associated experimental feature flag is also removed
+- The feature can be declared *stable*
+  - The associated experimental feature flag is removed
+  - There should be enough evidence of users having tried the feature, such as feedback, fixed bugs, demonstrations of how it is put to use
+  - Maintainers must feel confident that:
+    - The feature is designed and implemented sensibly, that it is fit for purpose
+    - Potential interactions are well-understood
+    - Stabilising the feature will not incur an outsized maintenance burden in the future
+
+The following diagram illustrates the process:
+
+```
+                  .------.
+                  | idea |
+                  '------'
+                      |
+       discussion, design, implementation
+                      |
+                      |     .-------.
+                      |     |       |
+                      v     v       |
+               .--------------.  review
+               | pull request |     |
+               '--------------'     |
+                   |     ^  |       |
+                   |     |  '-------'
+               .---'     '----.
+               |              |
+             merge       user feedback,
+               |       (breaking) changes
+               |              |
+               '---.     .----'
+                   |     |
+                   v     |
+               +--------------+
+           .---| experimental |----.
+           |   +--------------+    |
+           |                       |
+decision to stabilise      decision against
+           |              keeping the feature
+           |                       |
+           v                       v
+       +--------+             +---------+
+       | stable |             | removed |
+       +--------+             +---------+
+```
+
+# Relation to the RFC process
+
+Experimental features and [RFCs](https://github.com/NixOS/rfcs/) both allow approaching substantial changes while minimizing the risk.
+However they serve different purposes:
+
+- An experimental feature enables developers to iterate on and deliver a new idea without committing to it or requiring a costly long-running fork.
+  It is primarily an issue of *implementation*, targeting Nix developers and early testers.
+- The goal of an RFC is to make explicit all the implications of a change:
+  Explain why it is wanted, which new use-cases it enables, which interface changes it requires, etc.
+  It is primarily an issue of *design* and *communication*, targeting the broader community.
+
+This means that experimental features and RFCs are orthogonal mechanisms, and can be used independently or together as needed.
diff --git a/doc/manual/src/glossary.md b/doc/manual/src/glossary.md
index b56d857d1..4eedb2e93 100644
--- a/doc/manual/src/glossary.md
+++ b/doc/manual/src/glossary.md
@@ -15,7 +15,7 @@
 
     Example: `/nix/store/g946hcz4c8mdvq2g8vxx42z51qb71rvp-git-2.38.1.drv`
 
-    See [`nix show-derivation`](./command-ref/new-cli/nix3-show-derivation.md) (experimental) for displaying the contents of store derivations.
+    See [`nix derivation show`](./command-ref/new-cli/nix3-derivation-show.md) (experimental) for displaying the contents of store derivations.
 
     [store derivation]: #gloss-store-derivation
 
@@ -54,7 +54,7 @@
     invoked, the  Nix store can be  referred to
     as a "_local_" or a "_remote_" one:
 
-    + A *local  store* exists  on the filesystem of
+    + A [local store]{#gloss-local-store} exists on the filesystem of
       the machine where Nix is  invoked. You can use other
       local stores  by passing  the `--store` flag  to the
       `nix` command.  Local stores can be used for building derivations.
@@ -65,17 +65,17 @@
       served by the `nix-serve` Perl script.
 
     [store]: #gloss-store
+    [local store]: #gloss-local-store
 
   - [chroot store]{#gloss-chroot-store}\
-    A local store whose canonical path is anything other than `/nix/store`.
+    A [local store] whose canonical path is anything other than `/nix/store`.
 
   - [binary cache]{#gloss-binary-cache}\
     A *binary cache* is a Nix store which uses a different format: its
     metadata and signatures are kept in `.narinfo` files rather than in a
-    Nix database.  This different format simplifies serving store objects
-    over the network, but cannot host builds.  Examples of binary caches
-    include S3 buckets and the [NixOS binary
-    cache](https://cache.nixos.org).
+    [Nix database]. This different format simplifies serving store objects
+    over the network, but cannot host builds. Examples of binary caches
+    include S3 buckets and the [NixOS binary cache](https://cache.nixos.org).
 
   - [store path]{#gloss-store-path}\
     The location of a [store object] in the file system, i.e., an
@@ -108,7 +108,7 @@
     [fixed-output derivations](#gloss-fixed-output-derivation).
 
   - [substitute]{#gloss-substitute}\
-    A substitute is a command invocation stored in the Nix database that
+    A substitute is a command invocation stored in the [Nix database] that
     describes how to build a store object, bypassing the normal build
     mechanism (i.e., derivations). Typically, the substitute builds the
     store object by downloading a pre-built version of the store object
@@ -127,6 +127,14 @@
     builder can rely on external inputs such as the network or the
     system time) but the Nix model assumes it.
 
+  - Nix database{#gloss-nix-database}\
+    An SQlite database to track [reference]s between [store object]s.
+    This is an implementation detail of the [local store].
+
+    Default location: `/nix/var/nix/db`.
+
+    [Nix database]: #gloss-nix-database
+
   - [Nix expression]{#gloss-nix-expression}\
     A high-level description of software packages and compositions
     thereof. Deploying software using Nix entails writing Nix
@@ -175,9 +183,9 @@
   - [validity]{#gloss-validity}\
     A store path is valid if all [store object]s in its [closure] can be read from the [store].
 
-    For a local store, this means:
+    For a [local store], this means:
     - The store path leads to an existing [store object] in that [store].
-    - The store path is listed in the Nix database as being valid.
+    - The store path is listed in the [Nix database] as being valid.
     - All paths in the store path's [closure] are valid.
 
     [validity]: #gloss-validity
diff --git a/doc/manual/src/release-notes/rl-next.md b/doc/manual/src/release-notes/rl-next.md
index ae159de8f..5b62836bf 100644
--- a/doc/manual/src/release-notes/rl-next.md
+++ b/doc/manual/src/release-notes/rl-next.md
@@ -39,3 +39,20 @@
 
   * `man nix-store-<operation>` and `man nix-env-<operation>`
   * `nix-store --help --<operation>` and `nix-env --help --<operation>`.
+
+* Nix when used as a client now checks whether the store (the server) trusts the client.
+  (The store always had to check whether it trusts the client, but now the client is informed of the store's decision.)
+  This is useful for scripting interactions with (non-legacy-ssh) remote Nix stores.
+
+  `nix store ping` and `nix doctor` now display this information.
+
+* A new command `nix derivation add` is created, to allow adding derivations to the store without involving the Nix language.
+  It exists to round out our collection of basic utility/plumbing commands, and allow for a low barrier-to-entry way of experimenting with alternative front-ends to the Nix Store.
+  It uses the same JSON layout as `nix show-derivation`, and is its inverse.
+
+* `nix show-derivation` has been renamed to `nix derivation show`.
+  This matches `nix derivation add`, and avoids bloating the top-level namespace.
+  The old name is still kept as an alias for compatibility, however.
+
+* The `nix derivation {add,show}` JSON format now includes the derivation name as a top-level field.
+  This is useful in general, but especially necessary for the `add` direction, as otherwise we would need to pass in the name out of band for certain cases.