Make command infra less stateful and more regular

Already, we had classes like `BuiltPathsCommand` and `StorePathsCommand`
which provided alternative `run` virtual functions providing the
implementation with more arguments. This was a very nice and easy way to
make writing command; just fill in the virtual functions and it is
fairly clear what to do.

However, exception to this pattern were `Installable{,s}Command`. These
two classes instead just had a field where the installables would be
stored, and various side-effecting `prepare` and `load` machinery too
fill them in. Command would wish out those fields.

This isn't so clear to use.

What this commit does is make those command classes like the others,
with richer `run` functions.

Not only does this restore the pattern making commands easier to write,
it has a number of other benefits:

- `prepare` and `load` are gone entirely! One command just hands just
  hands off to the next.

- `useDefaultInstallables` because `defaultInstallables`. This takes
  over `prepare` for the one case that needs it, and provides enough
  flexiblity to handle `nix repl`'s idiosyncratic migration.

- We can use `ref` instead of `std::shared_ptr`. The former must be
  initialized (so it is like Rust's `Box` rather than `Option<Box>`,
  This expresses the invariant that the installable are in fact
  initialized much better.

  This is possible because since we just have local variables not
  fields, we can stop worrying about the not-yet-initialized case.

- Fewer lines of code! (Finally I have a large refactor that makes the
  number go down not up...)

- `nix repl` is now implemented in a clearer way.

The last item deserves further mention. `nix repl` is not like the other
installable commands because instead working from once-loaded
installables, it needs to be able to load them again and again.

To properly support this, we make a new superclass
`RawInstallablesCommand`. This class has the argument parsing and
completion logic, but does *not* hand off parsed installables but
instead just the raw string arguments.

This is exactly what `nix repl` needs, and allows us to instead of
having the logic awkwardly split between `prepare`,
`useDefaultInstallables,` and `load`, have everything right next to each
other. I think this will enable future simplifications of that argument
defaulting logic, but I am saving those for a future PR --- best to keep
code motion and more complicated boolean expression rewriting separate
steps.

The "diagnostic ignored `-Woverloaded-virtual`" pragma helps because C++
doesn't like our many `run` methods. In our case, we don't mind the
shadowing it all --- it is *intentional* that the derived class only
provides a `run` method, and doesn't call any of the overridden `run`
methods.

Helps with https://github.com/NixOS/rfcs/pull/134

4 files changed, 87 insertions, 71 deletions

diff --git a/src/libcmd/command.cc b/src/libcmd/command.cc
index ab51c229d..bedf11e2c 100644
--- a/src/libcmd/command.cc
+++ b/src/libcmd/command.cc
@@ -165,7 +165,7 @@ BuiltPathsCommand::BuiltPathsCommand(bool recursive)
     });
 }
 
-void BuiltPathsCommand::run(ref<Store> store)
+void BuiltPathsCommand::run(ref<Store> store, Installables && installables)
 {
     BuiltPaths paths;
     if (all) {
@@ -211,7 +211,7 @@ void StorePathsCommand::run(ref<Store> store, BuiltPaths && paths)
     run(store, std::move(sorted));
 }
 
-void StorePathCommand::run(ref<Store> store, std::vector<StorePath> && storePaths)
+void StorePathCommand::run(ref<Store> store, StorePaths && storePaths)
 {
     if (storePaths.size() != 1)
         throw UsageError("this command requires exactly one store path");
@@ -246,7 +246,7 @@ void MixProfile::updateProfile(const BuiltPaths & buildables)
 {
     if (!profile) return;
 
-    std::vector<StorePath> result;
+    StorePaths result;
 
     for (auto & buildable : buildables) {
         std::visit(overloaded {
diff --git a/src/libcmd/command.hh b/src/libcmd/command.hh
index 0d84c8395..874ca3249 100644
--- a/src/libcmd/command.hh
+++ b/src/libcmd/command.hh
@@ -29,6 +29,9 @@ struct NixMultiCommand : virtual MultiCommand, virtual Command
     nlohmann::json toJSON() override;
 };
 
+// For the overloaded run methods
+#pragma GCC diagnostic ignored "-Woverloaded-virtual"
+
 /* A command that requires a Nix store. */
 struct StoreCommand : virtual Command
 {
@@ -97,10 +100,10 @@ struct SourceExprCommand : virtual Args, MixFlakeOptions
 
     SourceExprCommand();
 
-    std::vector<std::shared_ptr<Installable>> parseInstallables(
+    Installables parseInstallables(
         ref<Store> store, std::vector<std::string> ss);
 
-    std::shared_ptr<Installable> parseInstallable(
+    ref<Installable> parseInstallable(
         ref<Store> store, const std::string & installable);
 
     virtual Strings getDefaultFlakeAttrPaths();
@@ -115,36 +118,43 @@ struct MixReadOnlyOption : virtual Args
     MixReadOnlyOption();
 };
 
-/* A command that operates on a list of "installables", which can be
-   store paths, attribute paths, Nix expressions, etc. */
-struct InstallablesCommand : virtual Args, SourceExprCommand
+/* Like InstallablesCommand but the installables are not loaded */
+struct RawInstallablesCommand : virtual Args, SourceExprCommand
 {
-    std::vector<std::shared_ptr<Installable>> installables;
+    RawInstallablesCommand();
 
-    InstallablesCommand();
+    virtual void run(ref<Store> store, std::vector<std::string> && rawInstallables) = 0;
 
-    void prepare() override;
-    Installables load();
+    void run(ref<Store> store) override;
 
-    virtual bool useDefaultInstallables() { return true; }
+    // FIXME make const after CmdRepl's override is fixed up
+    virtual void applyDefaultInstallables(std::vector<std::string> & rawInstallables);
 
     bool readFromStdIn = false;
 
     std::vector<std::string> getFlakesForCompletion() override;
 
-protected:
+private:
+
+    std::vector<std::string> rawInstallables;
+};
+/* A command that operates on a list of "installables", which can be
+   store paths, attribute paths, Nix expressions, etc. */
+struct InstallablesCommand : RawInstallablesCommand
+{
+    virtual void run(ref<Store> store, Installables && installables) = 0;
 
-    std::vector<std::string> _installables;
+    void run(ref<Store> store, std::vector<std::string> && rawInstallables) override;
 };
 
 /* A command that operates on exactly one "installable" */
 struct InstallableCommand : virtual Args, SourceExprCommand
 {
-    std::shared_ptr<Installable> installable;
-
     InstallableCommand();
 
-    void prepare() override;
+    virtual void run(ref<Store> store, ref<Installable> installable) = 0;
+
+    void run(ref<Store> store) override;
 
     std::vector<std::string> getFlakesForCompletion() override
     {
@@ -179,22 +189,18 @@ public:
 
     BuiltPathsCommand(bool recursive = false);
 
-    using StoreCommand::run;
-
     virtual void run(ref<Store> store, BuiltPaths && paths) = 0;
 
-    void run(ref<Store> store) override;
+    void run(ref<Store> store, Installables && installables) override;
 
-    bool useDefaultInstallables() override { return !all; }
+    void applyDefaultInstallables(std::vector<std::string> & rawInstallables) override;
 };
 
 struct StorePathsCommand : public BuiltPathsCommand
 {
     StorePathsCommand(bool recursive = false);
 
-    using BuiltPathsCommand::run;
-
-    virtual void run(ref<Store> store, std::vector<StorePath> && storePaths) = 0;
+    virtual void run(ref<Store> store, StorePaths && storePaths) = 0;
 
     void run(ref<Store> store, BuiltPaths && paths) override;
 };
@@ -202,11 +208,9 @@ struct StorePathsCommand : public BuiltPathsCommand
 /* A command that operates on exactly one store path. */
 struct StorePathCommand : public StorePathsCommand
 {
-    using StorePathsCommand::run;
-
     virtual void run(ref<Store> store, const StorePath & storePath) = 0;
 
-    void run(ref<Store> store, std::vector<StorePath> && storePaths) override;
+    void run(ref<Store> store, StorePaths && storePaths) override;
 };
 
 /* A helper class for registering commands globally. */
diff --git a/src/libcmd/installables.cc b/src/libcmd/installables.cc
index 5ecf6293f..5cbf26b88 100644
--- a/src/libcmd/installables.cc
+++ b/src/libcmd/installables.cc
@@ -422,10 +422,10 @@ ref<eval_cache::EvalCache> openEvalCache(
         });
 }
 
-std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
+Installables SourceExprCommand::parseInstallables(
     ref<Store> store, std::vector<std::string> ss)
 {
-    std::vector<std::shared_ptr<Installable>> result;
+    Installables result;
 
     if (file || expr) {
         if (file && expr)
@@ -451,7 +451,7 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
         for (auto & s : ss) {
             auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s);
             result.push_back(
-                std::make_shared<InstallableAttrPath>(
+                make_ref<InstallableAttrPath>(
                     InstallableAttrPath::parse(
                         state, *this, vFile, prefix, extendedOutputsSpec)));
         }
@@ -468,7 +468,7 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
 
             if (prefix.find('/') != std::string::npos) {
                 try {
-                    result.push_back(std::make_shared<InstallableDerivedPath>(
+                    result.push_back(make_ref<InstallableDerivedPath>(
                         InstallableDerivedPath::parse(store, prefix, extendedOutputsSpec)));
                     continue;
                 } catch (BadStorePath &) {
@@ -480,7 +480,7 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
 
             try {
                 auto [flakeRef, fragment] = parseFlakeRefWithFragment(std::string { prefix }, absPath("."));
-                result.push_back(std::make_shared<InstallableFlake>(
+                result.push_back(make_ref<InstallableFlake>(
                         this,
                         getEvalState(),
                         std::move(flakeRef),
@@ -501,7 +501,7 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
     return result;
 }
 
-std::shared_ptr<Installable> SourceExprCommand::parseInstallable(
+ref<Installable> SourceExprCommand::parseInstallable(
     ref<Store> store, const std::string & installable)
 {
     auto installables = parseInstallables(store, {installable});
@@ -513,7 +513,7 @@ std::vector<BuiltPathWithResult> Installable::build(
     ref<Store> evalStore,
     ref<Store> store,
     Realise mode,
-    const std::vector<std::shared_ptr<Installable>> & installables,
+    const Installables & installables,
     BuildMode bMode)
 {
     std::vector<BuiltPathWithResult> res;
@@ -522,11 +522,11 @@ std::vector<BuiltPathWithResult> Installable::build(
     return res;
 }
 
-std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> Installable::build2(
+std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> Installable::build2(
     ref<Store> evalStore,
     ref<Store> store,
     Realise mode,
-    const std::vector<std::shared_ptr<Installable>> & installables,
+    const Installables & installables,
     BuildMode bMode)
 {
     if (mode == Realise::Nothing)
@@ -535,7 +535,7 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> Instal
     struct Aux
     {
         ExtraPathInfo info;
-        std::shared_ptr<Installable> installable;
+        ref<Installable> installable;
     };
 
     std::vector<DerivedPath> pathsToBuild;
@@ -548,7 +548,7 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> Instal
         }
     }
 
-    std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> res;
+    std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> res;
 
     switch (mode) {
 
@@ -620,7 +620,7 @@ BuiltPaths Installable::toBuiltPaths(
     ref<Store> store,
     Realise mode,
     OperateOn operateOn,
-    const std::vector<std::shared_ptr<Installable>> & installables)
+    const Installables & installables)
 {
     if (operateOn == OperateOn::Output) {
         BuiltPaths res;
@@ -642,7 +642,7 @@ StorePathSet Installable::toStorePaths(
     ref<Store> evalStore,
     ref<Store> store,
     Realise mode, OperateOn operateOn,
-    const std::vector<std::shared_ptr<Installable>> & installables)
+    const Installables & installables)
 {
     StorePathSet outPaths;
     for (auto & path : toBuiltPaths(evalStore, store, mode, operateOn, installables)) {
@@ -656,7 +656,7 @@ StorePath Installable::toStorePath(
     ref<Store> evalStore,
     ref<Store> store,
     Realise mode, OperateOn operateOn,
-    std::shared_ptr<Installable> installable)
+    ref<Installable> installable)
 {
     auto paths = toStorePaths(evalStore, store, mode, operateOn, {installable});
 
@@ -668,7 +668,7 @@ StorePath Installable::toStorePath(
 
 StorePathSet Installable::toDerivations(
     ref<Store> store,
-    const std::vector<std::shared_ptr<Installable>> & installables,
+    const Installables & installables,
     bool useDeriver)
 {
     StorePathSet drvPaths;
@@ -692,9 +692,8 @@ StorePathSet Installable::toDerivations(
     return drvPaths;
 }
 
-InstallablesCommand::InstallablesCommand()
+RawInstallablesCommand::RawInstallablesCommand()
 {
-
     addFlag({
         .longName = "stdin",
         .description = "Read installables from the standard input.",
@@ -703,40 +702,45 @@ InstallablesCommand::InstallablesCommand()
 
     expectArgs({
         .label = "installables",
-        .handler = {&_installables},
+        .handler = {&rawInstallables},
         .completer = {[&](size_t, std::string_view prefix) {
             completeInstallable(prefix);
         }}
     });
 }
 
-void InstallablesCommand::prepare()
-{
-    installables = load();
-}
-
-Installables InstallablesCommand::load()
+void RawInstallablesCommand::applyDefaultInstallables(std::vector<std::string> & rawInstallables)
 {
-    if (_installables.empty() && useDefaultInstallables() && !readFromStdIn)
+    if (rawInstallables.empty()) {
         // FIXME: commands like "nix profile install" should not have a
         // default, probably.
-        _installables.push_back(".");
+        rawInstallables.push_back(".");
+    }
+}
 
+void RawInstallablesCommand::run(ref<Store> store)
+{
     if (readFromStdIn && !isatty(STDIN_FILENO)) {
         std::string word;
         while (std::cin >> word) {
-            _installables.emplace_back(std::move(word));
+            rawInstallables.emplace_back(std::move(word));
         }
     }
 
-    return parseInstallables(getStore(), _installables);
+    applyDefaultInstallables(rawInstallables);
+    run(store, std::move(rawInstallables));
 }
 
-std::vector<std::string> InstallablesCommand::getFlakesForCompletion()
+std::vector<std::string> RawInstallablesCommand::getFlakesForCompletion()
 {
-    if (_installables.empty() && useDefaultInstallables())
-        return {"."};
-    return _installables;
+    applyDefaultInstallables(rawInstallables);
+    return rawInstallables;
+}
+
+void InstallablesCommand::run(ref<Store> store, std::vector<std::string> && rawInstallables)
+{
+    auto installables = parseInstallables(store, rawInstallables);
+    run(store, std::move(installables));
 }
 
 InstallableCommand::InstallableCommand()
@@ -752,9 +756,16 @@ InstallableCommand::InstallableCommand()
     });
 }
 
-void InstallableCommand::prepare()
+void InstallableCommand::run(ref<Store> store)
+{
+    auto installable = parseInstallable(store, _installable);
+    run(store, std::move(installable));
+}
+
+void BuiltPathsCommand::applyDefaultInstallables(std::vector<std::string> & rawInstallables)
 {
-    installable = parseInstallable(getStore(), _installable);
+    if (rawInstallables.empty() && !all)
+        rawInstallables.push_back(".");
 }
 
 }
diff --git a/src/libcmd/installables.hh b/src/libcmd/installables.hh
index be77fdc81..6c2922d89 100644
--- a/src/libcmd/installables.hh
+++ b/src/libcmd/installables.hh
@@ -79,6 +79,9 @@ struct BuiltPathWithResult
 
 typedef std::vector<DerivedPathWithInfo> DerivedPathsWithInfo;
 
+struct Installable;
+typedef std::vector<ref<Installable>> Installables;
+
 struct Installable
 {
     virtual ~Installable() { }
@@ -122,14 +125,14 @@ struct Installable
         ref<Store> evalStore,
         ref<Store> store,
         Realise mode,
-        const std::vector<std::shared_ptr<Installable>> & installables,
+        const Installables & installables,
         BuildMode bMode = bmNormal);
 
-    static std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> build2(
+    static std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> build2(
         ref<Store> evalStore,
         ref<Store> store,
         Realise mode,
-        const std::vector<std::shared_ptr<Installable>> & installables,
+        const Installables & installables,
         BuildMode bMode = bmNormal);
 
     static std::set<StorePath> toStorePaths(
@@ -137,18 +140,18 @@ struct Installable
         ref<Store> store,
         Realise mode,
         OperateOn operateOn,
-        const std::vector<std::shared_ptr<Installable>> & installables);
+        const Installables & installables);
 
     static StorePath toStorePath(
         ref<Store> evalStore,
         ref<Store> store,
         Realise mode,
         OperateOn operateOn,
-        std::shared_ptr<Installable> installable);
+        ref<Installable> installable);
 
     static std::set<StorePath> toDerivations(
         ref<Store> store,
-        const std::vector<std::shared_ptr<Installable>> & installables,
+        const Installables & installables,
         bool useDeriver = false);
 
     static BuiltPaths toBuiltPaths(
@@ -156,9 +159,7 @@ struct Installable
         ref<Store> store,
         Realise mode,
         OperateOn operateOn,
-        const std::vector<std::shared_ptr<Installable>> & installables);
+        const Installables & installables);
 };
 
-typedef std::vector<std::shared_ptr<Installable>> Installables;
-
 }