diff options
63 files changed, 2080 insertions, 402 deletions
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 000000000..a268d7caf --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,28 @@ +# Motivation +<!-- Briefly explain what the change is about and why it is desirable. --> + +# Context +<!-- Provide context. Reference open issues if available. --> + +<!-- Non-trivial change: Briefly outline the implementation strategy. --> + +<!-- Invasive change: Discuss alternative designs or approaches you considered. --> + +<!-- Large change: Provide instructions to reviewers how to read the diff. --> + +# Checklist for maintainers + +<!-- Contributors: please leave this as is --> + +Maintainers: tick if completed or explain if not relevant + + - [ ] agreed on idea + - [ ] agreed on implementation strategy + - [ ] tests, as appropriate + - functional tests - `tests/**.sh` + - unit tests - `src/*/tests` + - integration tests + - [ ] documentation in the manual + - [ ] code and comments are self-explanatory + - [ ] commit message explains why the change was made + - [ ] new feature or bug fix: updated release notes @@ -1 +1 @@ -2.13.0 +2.14.0
\ No newline at end of file diff --git a/configure.ac b/configure.ac index 1b0d6fd27..0066bc389 100644 --- a/configure.ac +++ b/configure.ac @@ -274,6 +274,12 @@ fi PKG_CHECK_MODULES([GTEST], [gtest_main]) +# Look for rapidcheck. +# No pkg-config yet, https://github.com/emil-e/rapidcheck/issues/302 +AC_CHECK_HEADERS([rapidcheck/gtest.h], [], [], [#include <gtest/gtest.h>]) +AC_CHECK_LIB([rapidcheck], []) + + # Look for nlohmann/json. PKG_CHECK_MODULES([NLOHMANN_JSON], [nlohmann_json >= 3.9]) diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in index 4f1fc34ce..b1c551969 100644 --- a/doc/manual/src/SUMMARY.md.in +++ b/doc/manual/src/SUMMARY.md.in @@ -67,6 +67,7 @@ - [CLI guideline](contributing/cli-guideline.md) - [Release Notes](release-notes/release-notes.md) - [Release X.Y (202?-??-??)](release-notes/rl-next.md) + - [Release 2.13 (2023-01-17)](release-notes/rl-2.13.md) - [Release 2.12 (2022-12-06)](release-notes/rl-2.12.md) - [Release 2.11 (2022-08-25)](release-notes/rl-2.11.md) - [Release 2.10 (2022-07-11)](release-notes/rl-2.10.md) diff --git a/doc/manual/src/command-ref/nix-store.md b/doc/manual/src/command-ref/nix-store.md index 6d0e02ca5..f6017481c 100644 --- a/doc/manual/src/command-ref/nix-store.md +++ b/doc/manual/src/command-ref/nix-store.md @@ -66,11 +66,11 @@ The operation `--realise` essentially “builds” the specified store paths. Realisation is a somewhat overloaded term: - If the store path is a *derivation*, realisation ensures that the - output paths of the derivation are [valid](../glossary.md) (i.e., + output paths of the derivation are [valid] (i.e., the output path and its closure exist in the file system). This can be done in several ways. First, it is possible that the outputs are already valid, in which case we are done - immediately. Otherwise, there may be [substitutes](../glossary.md) + immediately. Otherwise, there may be [substitutes] that produce the outputs (e.g., by downloading them). Finally, the outputs can be produced by running the build task described by the derivation. @@ -82,6 +82,9 @@ paths. Realisation is a somewhat overloaded term: produced through substitutes. If there are no (successful) substitutes, realisation fails. +[valid]: ../glossary.md#validity +[substitutes]: ../glossary.md#substitute + The output path of each derivation is printed on standard output. (For non-derivations argument, the argument itself is printed.) @@ -295,8 +298,8 @@ error: cannot delete path `/nix/store/zq0h41l75vlb4z45kzgjjmsjxvcv1qk7-mesa-6.4' ## Description -The operation `--query` displays various bits of information about the -store paths . The queries are described below. At most one query can be +The operation `--query` displays information about [store path]s. +The queries are described below. At most one query can be specified. The default query is `--outputs`. The paths *paths* may also be symlinks from outside of the Nix store, to @@ -316,12 +319,12 @@ symlink. ## Queries - `--outputs`\ - Prints out the [output paths](../glossary.md) of the store + Prints out the [output path]s of the store derivations *paths*. These are the paths that will be produced when the derivation is built. - `--requisites`; `-R`\ - Prints out the [closure](../glossary.md) of the store path *paths*. + Prints out the [closure] of the given *paths*. This query has one option: @@ -338,10 +341,12 @@ symlink. derivation and specifying the option `--include-outputs`. - `--references`\ - Prints the set of [references](../glossary.md) of the store paths + Prints the set of [references]s of the store paths *paths*, that is, their immediate dependencies. (For *all* dependencies, use `--requisites`.) + [reference]: ../glossary.md#gloss-reference + - `--referrers`\ Prints the set of *referrers* of the store paths *paths*, that is, the store paths currently existing in the Nix store that refer to @@ -356,11 +361,13 @@ symlink. in the Nix store that are dependent on *paths*. - `--deriver`; `-d`\ - Prints the [deriver](../glossary.md) of the store paths *paths*. If + Prints the [deriver] of the store paths *paths*. If the path has no deriver (e.g., if it is a source file), or if the deriver is not known (e.g., in the case of a binary-only deployment), the string `unknown-deriver` is printed. + [deriver]: ../glossary.md#gloss-deriver + - `--graph`\ Prints the references graph of the store paths *paths* in the format of the `dot` tool of AT\&T's [Graphviz diff --git a/doc/manual/src/contributing/hacking.md b/doc/manual/src/contributing/hacking.md index aeb0d41b3..9dbafcc0a 100644 --- a/doc/manual/src/contributing/hacking.md +++ b/doc/manual/src/contributing/hacking.md @@ -92,7 +92,8 @@ $ nix develop The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined under `src/{library_name}/tests` using the -[googletest](https://google.github.io/googletest/) framework. +[googletest](https://google.github.io/googletest/) and +[rapidcheck](https://github.com/emil-e/rapidcheck) frameworks. You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`. Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option. diff --git a/doc/manual/src/glossary.md b/doc/manual/src/glossary.md index e63f6becc..5a49af8dc 100644 --- a/doc/manual/src/glossary.md +++ b/doc/manual/src/glossary.md @@ -19,6 +19,17 @@ [store derivation]: #gloss-store-derivation + - [realise]{#gloss-realise}, realisation\ + Ensure a [store path] is [valid][validity]. + + This means either running the `builder` executable as specified in the corresponding [derivation] or fetching a pre-built [store object] from a [substituter]. + + See [`nix-build`](./command-ref/nix-build.md) and [`nix-store --realise`](./command-ref/nix-store.md#operation---realise). + + See [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental). + + [realise]: #gloss-realise + - [content-addressed derivation]{#gloss-content-addressed-derivation}\ A derivation which has the [`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed) @@ -101,6 +112,8 @@ copy store objects it doesn't have. For details, see the [`substituters` option](./command-ref/conf-file.md#conf-substituters). + [substituter]: #gloss-substituter + - [purity]{#gloss-purity}\ The assumption that equal Nix derivations when run always produce the same output. This cannot be guaranteed in general (e.g., a @@ -149,13 +162,15 @@ [output path]: #gloss-output-path - [deriver]{#gloss-deriver}\ - The deriver of an *output path* is the store - derivation that built it. + The [store derivation] that produced an [output path]. - [validity]{#gloss-validity}\ - A store path is considered *valid* if it exists in the file system, - is listed in the Nix database as being valid, and if all paths in - its closure are also valid. + A store path is valid if all [store object]s in its [closure] can be read from the [store]. + + For a local store, this means: + - The store path leads to an existing [store object] in that [store]. + - The store path is listed in the Nix database as being valid. + - All paths in the store path's [closure] are valid. - [user environment]{#gloss-user-env}\ An automatically generated store object that consists of a set of diff --git a/doc/manual/src/language/index.md b/doc/manual/src/language/index.md index db34fde75..31300631c 100644 --- a/doc/manual/src/language/index.md +++ b/doc/manual/src/language/index.md @@ -191,12 +191,12 @@ This is an incomplete overview of language features, by example. <tr> <td> - <nixpkgs> + `<nixpkgs>` </td> <td> - Search path. Value determined by [`$NIX_PATH` environment variable](../command-ref/env-common.md#env-NIX_PATH). + Search path for Nix files. Value determined by [`$NIX_PATH` environment variable](../command-ref/env-common.md#env-NIX_PATH). </td> </tr> diff --git a/doc/manual/src/language/operators.md b/doc/manual/src/language/operators.md index 797f13bd3..1f918bd4d 100644 --- a/doc/manual/src/language/operators.md +++ b/doc/manual/src/language/operators.md @@ -24,7 +24,7 @@ | [Equality] | *expr* `==` *expr* | none | 11 | | Inequality | *expr* `!=` *expr* | none | 11 | | Logical conjunction (`AND`) | *bool* `&&` *bool* | left | 12 | -| Logical disjunction (`OR`) | *bool* `||` *bool* | left | 13 | +| Logical disjunction (`OR`) | *bool* `\|\|` *bool* | left | 13 | | [Logical implication] | *bool* `->` *bool* | none | 14 | [string]: ./values.md#type-string @@ -120,12 +120,12 @@ The result is a string. ## Update -> *attrset1* + *attrset2* +> *attrset1* // *attrset2* Update [attribute set] *attrset1* with names and values from *attrset2*. -The returned attribute set will have of all the attributes in *e1* and *e2*. -If an attribute name is present in both, the attribute value from the former is taken. +The returned attribute set will have of all the attributes in *attrset1* and *attrset2*. +If an attribute name is present in both, the attribute value from the latter is taken. [Update]: #update diff --git a/doc/manual/src/package-management/binary-cache-substituter.md b/doc/manual/src/package-management/binary-cache-substituter.md index ef738794b..5befad9f8 100644 --- a/doc/manual/src/package-management/binary-cache-substituter.md +++ b/doc/manual/src/package-management/binary-cache-substituter.md @@ -32,13 +32,13 @@ which should print something like: Priority: 30 On the client side, you can tell Nix to use your binary cache using -`--option extra-binary-caches`, e.g.: +`--substituters`, e.g.: ```console -$ nix-env -iA nixpkgs.firefox --option extra-binary-caches http://avalon:8080/ +$ nix-env -iA nixpkgs.firefox --substituters http://avalon:8080/ ``` -The option `extra-binary-caches` tells Nix to use this binary cache in +The option `substituters` tells Nix to use this binary cache in addition to your default caches, such as <https://cache.nixos.org>. Thus, for any path in the closure of Firefox, Nix will first check if the path is available on the server `avalon` or another binary caches. @@ -47,4 +47,4 @@ If not, it will fall back to building from source. You can also tell Nix to always use your binary cache by adding a line to the `nix.conf` configuration file like this: - binary-caches = http://avalon:8080/ https://cache.nixos.org/ + substituters = http://avalon:8080/ https://cache.nixos.org/ diff --git a/doc/manual/src/release-notes/rl-2.13.md b/doc/manual/src/release-notes/rl-2.13.md new file mode 100644 index 000000000..2ebf19f60 --- /dev/null +++ b/doc/manual/src/release-notes/rl-2.13.md @@ -0,0 +1,44 @@ +# Release 2.13 (2023-01-17) + +* The `repeat` and `enforce-determinism` options have been removed + since they had been broken under many circumstances for a long time. + +* You can now use [flake references] in the [old command line interface], e.g. + + [flake references]: ../command-ref/new-cli/nix3-flake.md#flake-references + [old command line interface]: ../command-ref/main-commands.md + + ```shell-session + # nix-build flake:nixpkgs -A hello + # nix-build -I nixpkgs=flake:github:NixOS/nixpkgs/nixos-22.05 \ + '<nixpkgs>' -A hello + # NIX_PATH=nixpkgs=flake:nixpkgs nix-build '<nixpkgs>' -A hello + ``` + +* Instead of "antiquotation", the more common term [string interpolation](../language/string-interpolation.md) is now used consistently. + Historical release notes were not changed. + +* Error traces have been reworked to provide detailed explanations and more + accurate error locations. A short excerpt of the trace is now shown by + default when an error occurs. + +* Allow explicitly selecting outputs in a store derivation installable, just like we can do with other sorts of installables. + For example, + ```shell-session + # nix-build /nix/store/gzaflydcr6sb3567hap9q6srzx8ggdgg-glibc-2.33-78.drv^dev + ``` + now works just as + ```shell-session + # nix-build glibc^dev + ``` + does already. + +* On Linux, `nix develop` now sets the + [*personality*](https://man7.org/linux/man-pages/man2/personality.2.html) + for the development shell in the same way as the actual build of the + derivation. This makes shells for `i686-linux` derivations work + correctly on `x86_64-linux`. + +* You can now disable the global flake registry by setting the `flake-registry` + configuration option to an empty string. The same can be achieved at runtime with + `--flake-registry ""`. diff --git a/doc/manual/src/release-notes/rl-next.md b/doc/manual/src/release-notes/rl-next.md index 9f491efc8..8a79703ab 100644 --- a/doc/manual/src/release-notes/rl-next.md +++ b/doc/manual/src/release-notes/rl-next.md @@ -1,23 +1,10 @@ # Release X.Y (202?-??-??) -* The `repeat` and `enforce-determinism` options have been removed - since they had been broken under many circumstances for a long time. - -* You can now use [flake references] in the [old command line interface], e.g. - - [flake references]: ../command-ref/new-cli/nix3-flake.md#flake-references - [old command line interface]: ../command-ref/main-commands.md - - ``` - # nix-build flake:nixpkgs -A hello - # nix-build -I nixpkgs=flake:github:NixOS/nixpkgs/nixos-22.05 \ - '<nixpkgs>' -A hello - # NIX_PATH=nixpkgs=flake:nixpkgs nix-build '<nixpkgs>' -A hello - ``` - -* Instead of "antiquotation", the more common term [string interpolation](../language/string-interpolation.md) is now used consistently. - Historical release notes were not changed. - -* Error traces have been reworked to provide detailed explanations and more - accurate error locations. A short excerpt of the trace is now shown by - default when an error occurs. +* A new function `builtins.readFileType` is available. It is similar to + `builtins.readDir` but acts on a single file or directory. + +* The `builtins.readDir` function has been optimized when encountering not-yet-known + file types from POSIX's `readdir`. In such cases the type of each file is/was + discovered by making multiple syscalls. This change makes these operations + lazy such that these lookups will only be performed if the attribute is used. + This optimization affects a minority of filesystems and operating systems. @@ -82,7 +82,9 @@ }); configureFlags = - lib.optionals stdenv.isLinux [ + [ + "CXXFLAGS=-I${lib.getDev rapidcheck}/extras/gtest/include" + ] ++ lib.optionals stdenv.isLinux [ "--with-boost=${boost}/lib" "--with-sandbox-shell=${sh}/bin/busybox" ] @@ -116,6 +118,7 @@ boost lowdown-nix gtest + rapidcheck ] ++ lib.optionals stdenv.isLinux [libseccomp] ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium @@ -532,6 +535,12 @@ mkdir $out ''; + tests.nixpkgsLibTests = + nixpkgs.lib.genAttrs systems (system: + import (nixpkgs + "/lib/tests/release.nix") + { pkgs = nixpkgsFor.${system}; } + ); + metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" { pkgs = nixpkgsFor.x86_64-linux; nixpkgs = nixpkgs-regression; @@ -562,6 +571,7 @@ binaryTarball = self.hydraJobs.binaryTarball.${system}; perlBindings = self.hydraJobs.perlBindings.${system}; installTests = self.hydraJobs.installTests.${system}; + nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system}; } // (nixpkgs.lib.optionalAttrs (builtins.elem system linux64BitSystems)) { dockerImage = self.hydraJobs.dockerImage.${system}; }); diff --git a/scripts/install-systemd-multi-user.sh b/scripts/install-systemd-multi-user.sh index 62397127a..7dd567747 100755 --- a/scripts/install-systemd-multi-user.sh +++ b/scripts/install-systemd-multi-user.sh @@ -24,12 +24,17 @@ $1 EOF } +escape_systemd_env() { + temp_var="${1//\'/\\\'}" + echo "${temp_var//\%/%%}" +} + # Gather all non-empty proxy environment variables into a string create_systemd_proxy_env() { vars="http_proxy https_proxy ftp_proxy no_proxy HTTP_PROXY HTTPS_PROXY FTP_PROXY NO_PROXY" for v in $vars; do if [ "x${!v:-}" != "x" ]; then - echo "Environment=${v}=${!v}" + echo "Environment=${v}=$(escape_systemd_env ${!v})" fi done } diff --git a/src/libcmd/repl.cc b/src/libcmd/repl.cc index 9b12f8fa2..4158439b6 100644 --- a/src/libcmd/repl.cc +++ b/src/libcmd/repl.cc @@ -397,7 +397,7 @@ StringSet NixRepl::completePrefix(const std::string & prefix) Expr * e = parseString(expr); Value v; e->eval(*state, *env, v); - state->forceAttrs(v, noPos, "nevermind, it is ignored anyway"); + state->forceAttrs(v, noPos, "while evaluating an attrset for the purpose of completion (this error should not be displayed; file an issue?)"); for (auto & i : *v.attrs) { std::string_view name = state->symbols[i.name]; diff --git a/src/libexpr/eval.cc b/src/libexpr/eval.cc index 277cbb5f9..1828b8c2e 100644 --- a/src/libexpr/eval.cc +++ b/src/libexpr/eval.cc @@ -11,7 +11,9 @@ #include <algorithm> #include <chrono> +#include <iostream> #include <cstring> +#include <optional> #include <unistd.h> #include <sys/time.h> #include <sys/resource.h> @@ -1927,7 +1929,9 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v) /* skip canonization of first path, which would only be not canonized in the first place if it's coming from a ./${foo} type path */ - auto part = state.coerceToString(i_pos, vTmp, context, false, firstType == nString, !first, "while evaluating a path segment"); + auto part = state.coerceToString(i_pos, vTmp, context, + "while evaluating a path segment", + false, firstType == nString, !first); sSize += part->size(); s.emplace_back(std::move(part)); } @@ -2123,15 +2127,16 @@ std::optional<std::string> EvalState::tryAttrsToString(const PosIdx pos, Value & if (i != v.attrs->end()) { Value v1; callFunction(*i->value, v, v1, pos); - return coerceToString(pos, v1, context, coerceMore, copyToStore, - "while evaluating the result of the `toString` attribute").toOwned(); + return coerceToString(pos, v1, context, + "while evaluating the result of the `__toString` attribute", + coerceMore, copyToStore).toOwned(); } return {}; } -BackedStringView EvalState::coerceToString(const PosIdx pos, Value & v, PathSet & context, - bool coerceMore, bool copyToStore, bool canonicalizePath, std::string_view errorCtx) +BackedStringView EvalState::coerceToString(const PosIdx pos, Value &v, PathSet &context, + std::string_view errorCtx, bool coerceMore, bool copyToStore, bool canonicalizePath) { forceValue(v, pos); @@ -2154,13 +2159,23 @@ BackedStringView EvalState::coerceToString(const PosIdx pos, Value & v, PathSet if (maybeString) return std::move(*maybeString); auto i = v.attrs->find(sOutPath); - if (i == v.attrs->end()) - error("cannot coerce a set to a string", showType(v)).withTrace(pos, errorCtx).debugThrow<TypeError>(); - return coerceToString(pos, *i->value, context, coerceMore, copyToStore, canonicalizePath, errorCtx); + if (i == v.attrs->end()) { + error("cannot coerce %1% to a string", showType(v)) + .withTrace(pos, errorCtx) + .debugThrow<TypeError>(); + } + return coerceToString(pos, *i->value, context, errorCtx, + coerceMore, copyToStore, canonicalizePath); } - if (v.type() == nExternal) - return v.external->coerceToString(positions[pos], context, coerceMore, copyToStore, errorCtx); + if (v.type() == nExternal) { + try { + return v.external->coerceToString(positions[pos], context, coerceMore, copyToStore); + } catch (Error & e) { + e.addTrace(nullptr, errorCtx); + throw; + } + } if (coerceMore) { /* Note that `false' is represented as an empty string for @@ -2175,8 +2190,9 @@ BackedStringView EvalState::coerceToString(const PosIdx pos, Value & v, PathSet std::string result; for (auto [n, v2] : enumerate(v.listItems())) { try { - result += *coerceToString(noPos, *v2, context, coerceMore, copyToStore, canonicalizePath, - "while evaluating one element of the list"); + result += *coerceToString(noPos, *v2, context, + "while evaluating one element of the list", + coerceMore, copyToStore, canonicalizePath); } catch (Error & e) { e.addTrace(positions[pos], errorCtx); throw; @@ -2190,7 +2206,9 @@ BackedStringView EvalState::coerceToString(const PosIdx pos, Value & v, PathSet } } - error("cannot coerce %1% to a string", showType(v)).withTrace(pos, errorCtx).debugThrow<TypeError>(); + error("cannot coerce %1% to a string", showType(v)) + .withTrace(pos, errorCtx) + .debugThrow<TypeError>(); } @@ -2220,7 +2238,7 @@ StorePath EvalState::copyPathToStore(PathSet & context, const Path & path) Path EvalState::coerceToPath(const PosIdx pos, Value & v, PathSet & context, std::string_view errorCtx) { - auto path = coerceToString(pos, v, context, false, false, true, errorCtx).toOwned(); + auto path = coerceToString(pos, v, context, errorCtx, false, false, true).toOwned(); if (path == "" || path[0] != '/') error("string '%1%' doesn't represent an absolute path", path).withTrace(pos, errorCtx).debugThrow<EvalError>(); return path; @@ -2229,7 +2247,7 @@ Path EvalState::coerceToPath(const PosIdx pos, Value & v, PathSet & context, std StorePath EvalState::coerceToStorePath(const PosIdx pos, Value & v, PathSet & context, std::string_view errorCtx) { - auto path = coerceToString(pos, v, context, false, false, true, errorCtx).toOwned(); + auto path = coerceToString(pos, v, context, errorCtx, false, false, true).toOwned(); if (auto storePath = store->maybeParseStorePath(path)) return *storePath; error("path '%1%' is not in the Nix store", path).withTrace(pos, errorCtx).debugThrow<EvalError>(); @@ -2433,13 +2451,11 @@ void EvalState::printStats() } -std::string ExternalValueBase::coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore, std::string_view errorCtx) const +std::string ExternalValueBase::coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore) const { - auto e = TypeError({ + throw TypeError({ .msg = hintfmt("cannot coerce %1% to a string", showType()) }); - e.addTrace(pos, errorCtx); - throw e; } diff --git a/src/libexpr/eval.hh b/src/libexpr/eval.hh index 46b8cbaa5..e4d5906bd 100644 --- a/src/libexpr/eval.hh +++ b/src/libexpr/eval.hh @@ -203,6 +203,9 @@ public: throw std::move(error); } + // This is dangerous, but gets in line with the idea that error creation and + // throwing should not allocate on the stack of hot functions. + // as long as errors are immediately thrown, it works. ErrorBuilder * errorBuilder; template<typename... Args> @@ -375,9 +378,9 @@ public: booleans and lists to a string. If `copyToStore' is set, referenced paths are copied to the Nix store as a side effect. */ BackedStringView coerceToString(const PosIdx pos, Value & v, PathSet & context, + std::string_view errorCtx, bool coerceMore = false, bool copyToStore = true, - bool canonicalizePath = true, - std::string_view errorCtx = ""); + bool canonicalizePath = true); StorePath copyPathToStore(PathSet & context, const Path & path); diff --git a/src/libexpr/flake/flake.cc b/src/libexpr/flake/flake.cc index fc4be5678..336eb274d 100644 --- a/src/libexpr/flake/flake.cc +++ b/src/libexpr/flake/flake.cc @@ -264,7 +264,7 @@ static Flake getFlake( PathSet emptyContext = {}; flake.config.settings.emplace( state.symbols[setting.name], - state.coerceToString(setting.pos, *setting.value, emptyContext, false, true, true, "") .toOwned()); + state.coerceToString(setting.pos, *setting.value, emptyContext, "", false, true, true) .toOwned()); } else if (setting.value->type() == nInt) flake.config.settings.emplace( diff --git a/src/libexpr/primops.cc b/src/libexpr/primops.cc index f3bd0557b..db33749a1 100644 --- a/src/libexpr/primops.cc +++ b/src/libexpr/primops.cc @@ -350,26 +350,22 @@ void prim_exec(EvalState & state, const PosIdx pos, Value * * args, Value & v) auto elems = args[0]->listElems(); auto count = args[0]->listSize(); if (count == 0) - state.debugThrowLastTrace(EvalError({ - .msg = hintfmt("at least one argument to 'exec' required"), - .errPos = state.positions[pos] - })); + state.error("at least one argument to 'exec' required").atPos(pos).debugThrow<EvalError>(); PathSet context; - auto program = state.coerceToString(pos, *elems[0], context, false, false, - "while evaluating the first element of the argument passed to builtins.exec").toOwned(); + auto program = state.coerceToString(pos, *elems[0], context, + "while evaluating the first element of the argument passed to builtins.exec", + false, false).toOwned(); Strings commandArgs; for (unsigned int i = 1; i < args[0]->listSize(); ++i) { - commandArgs.push_back(state.coerceToString(pos, *elems[i], context, false, false, - "while evaluating an element of the argument passed to builtins.exec").toOwned()); + commandArgs.push_back( + state.coerceToString(pos, *elems[i], context, + "while evaluating an element of the argument passed to builtins.exec", + false, false).toOwned()); } try { auto _ = state.realiseContext(context); // FIXME: Handle CA derivations } catch (InvalidPathError & e) { - state.debugThrowLastTrace(EvalError({ - .msg = hintfmt("cannot execute '%1%', since path '%2%' is not valid", - program, e.path), - .errPos = state.positions[pos] - })); + state.error("cannot execute '%1%', since path '%2%' is not valid", program, e.path).atPos(pos).debugThrow<EvalError>(); } auto output = runProgram(program, true, commandArgs); @@ -598,7 +594,8 @@ struct CompareValues state.error("cannot compare %s with %s; values of that type are incomparable", showType(*v1), showType(*v2)).debugThrow<EvalError>(); } } catch (Error & e) { - e.addTrace(nullptr, errorCtx); + if (!errorCtx.empty()) + e.addTrace(nullptr, errorCtx); throw; } } @@ -620,15 +617,7 @@ static Bindings::iterator getAttr( { Bindings::iterator value = attrSet->find(attrSym); if (value == attrSet->end()) { - throw TypeError({ - .msg = hintfmt("attribute '%s' missing %s", state.symbols[attrSym], normaltxt(errorCtx)), - .errPos = state.positions[attrSet->pos], - }); - // TODO XXX - // Adding another trace for the function name to make it clear - // which call received wrong arguments. - //e.addTrace(state.positions[pos], hintfmt("while invoking '%s'", funcName)); - //state.debugThrowLastTrace(e); + state.error("attribute '%s' missing", state.symbols[attrSym]).withTrace(noPos, errorCtx).debugThrow<TypeError>(); } return value; } @@ -801,8 +790,10 @@ static void prim_addErrorContext(EvalState & state, const PosIdx pos, Value * * v = *args[1]; } catch (Error & e) { PathSet context; - e.addTrace(nullptr, state.coerceToString(pos, *args[0], context, - "while evaluating the error message passed to builtins.addErrorContext").toOwned()); + auto message = state.coerceToString(pos, *args[0], context, + "while evaluating the error message passed to builtins.addErrorContext", + false, false).toOwned(); + e.addTrace(nullptr, message, true); throw; } } @@ -1006,6 +997,7 @@ static void prim_second(EvalState & state, const PosIdx pos, Value * * args, Val * Derivations *************************************************************/ +static void derivationStrictInternal(EvalState & state, const std::string & name, Bindings * attrs, Value & v); /* Construct (as a unobservable side effect) a Nix derivation expression that performs the derivation described by the argument @@ -1016,32 +1008,68 @@ static void prim_second(EvalState & state, const PosIdx pos, Value * * args, Val derivation. */ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * args, Value & v) { - using nlohmann::json; state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.derivationStrict"); + Bindings * attrs = args[0]->attrs; + /* Figure out the name first (for stack backtraces). */ - Bindings::iterator attr = getAttr(state, state.sName, args[0]->attrs, "in the attrset passed as argument to builtins.derivationStrict"); + Bindings::iterator nameAttr = getAttr(state, state.sName, attrs, "in the attrset passed as argument to builtins.derivationStrict"); std::string drvName; - const auto posDrvName = attr->pos; try { - drvName = state.forceStringNoCtx(*attr->value, pos, "while evaluating the `name` attribute passed to builtins.derivationStrict"); + drvName = state.forceStringNoCtx(*nameAttr->value, pos, "while evaluating the `name` attribute passed to builtins.derivationStrict"); + } catch (Error & e) { + e.addTrace(state.positions[nameAttr->pos], "while evaluating the derivation attribute 'name'"); + throw; + } + + try { + derivationStrictInternal(state, drvName, attrs, v); } catch (Error & e) { - e.addTrace(state.positions[posDrvName], "while evaluating the derivation attribute 'name'"); + Pos pos = state.positions[nameAttr->pos]; + /* + * Here we make two abuses of the error system + * + * 1. We print the location as a string to avoid a code snippet being + * printed. While the location of the name attribute is a good hint, the + * exact code there is irrelevant. + * + * 2. We mark this trace as a frame trace, meaning that we stop printing + * less important traces from now on. In particular, this prevents the + * display of the automatic "while calling builtins.derivationStrict" + * trace, which is of little use for the public we target here. + * + * Please keep in mind that error reporting is done on a best-effort + * basis in nix. There is no accurate location for a derivation, as it + * often results from the composition of several functions + * (derivationStrict, derivation, mkDerivation, mkPythonModule, etc.) + */ + e.addTrace(nullptr, hintfmt( + "while evaluating derivation '%s'\n" + " whose name attribute is located at %s", + drvName, pos), true); throw; } +} +static void derivationStrictInternal(EvalState & state, const std::string & +drvName, Bindings * attrs, Value & v) +{ /* Check whether attributes should be passed as a JSON file. */ + using nlohmann::json; std::optional<json> jsonObject; - attr = args[0]->attrs->find(state.sStructuredAttrs); - if (attr != args[0]->attrs->end() && state.forceBool(*attr->value, pos, "while evaluating the `__structuredAttrs` attribute passed to builtins.derivationStrict")) + auto attr = attrs->find(state.sStructuredAttrs); + if (attr != attrs->end() && + state.forceBool(*attr->value, noPos, + "while evaluating the `__structuredAttrs` " + "attribute passed to builtins.derivationStrict")) jsonObject = json::object(); /* Check whether null attributes should be ignored. */ bool ignoreNulls = false; - attr = args[0]->attrs->find(state.sIgnoreNulls); - if (attr != args[0]->attrs->end()) - ignoreNulls = state.forceBool(*attr->value, pos, "while evaluating the `__ignoreNulls` attribute passed to builtins.derivationStrict"); + attr = attrs->find(state.sIgnoreNulls); + if (attr != attrs->end()) + ignoreNulls = state.forceBool(*attr->value, noPos, "while evaluating the `__ignoreNulls` attribute " "passed to builtins.derivationStrict"); /* Build the derivation expression by processing the attributes. */ Derivation drv; @@ -1058,7 +1086,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * StringSet outputs; outputs.insert("out"); - for (auto & i : args[0]->attrs->lexicographicOrder(state.symbols)) { + for (auto & i : attrs->lexicographicOrder(state.symbols)) { if (i->name == state.sIgnoreNulls) continue; const std::string & key = state.symbols[i->name]; vomit("processing attribute '%1%'", key); @@ -1070,7 +1098,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * else state.debugThrowLastTrace(EvalError({ .msg = hintfmt("invalid value '%s' for 'outputHashMode' attribute", s), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); }; @@ -1080,7 +1108,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (outputs.find(j) != outputs.end()) state.debugThrowLastTrace(EvalError({ .msg = hintfmt("duplicate derivation output '%1%'", j), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); /* !!! Check whether j is a valid attribute name. */ @@ -1090,34 +1118,35 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (j == "drv") state.debugThrowLastTrace(EvalError({ .msg = hintfmt("invalid derivation output name 'drv'" ), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); outputs.insert(j); } if (outputs.empty()) state.debugThrowLastTrace(EvalError({ .msg = hintfmt("derivation cannot have an empty set of outputs"), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); }; try { + // This try-catch block adds context for most errors. + // Use this empty error context to signify that we defer to it. + const std::string_view context_below(""); if (ignoreNulls) { - state.forceValue(*i->value, pos); + state.forceValue(*i->value, noPos); if (i->value->type() == nNull) continue; } if (i->name == state.sContentAddressed) { - contentAddressed = state.forceBool(*i->value, pos, - "while evaluating the `__contentAddressed` attribute passed to builtins.derivationStrict"); + contentAddressed = state.forceBool(*i->value, noPos, context_below); if (contentAddressed) settings.requireExperimentalFeature(Xp::CaDerivations); } else if (i->name == state.sImpure) { - isImpure = state.forceBool(*i->value, pos, - "while evaluating the 'impure' attribute passed to builtins.derivationStrict"); + isImpure = state.forceBool(*i->value, noPos, context_below); if (isImpure) settings.requireExperimentalFeature(Xp::ImpureDerivations); } @@ -1125,11 +1154,11 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * /* The `args' attribute is special: it supplies the command-line arguments to the builder. */ else if (i->name == state.sArgs) { - state.forceList(*i->value, pos, - "while evaluating the `args` attribute passed to builtins.derivationStrict"); + state.forceList(*i->value, noPos, context_below); for (auto elem : i->value->listItems()) { - auto s = state.coerceToString(posDrvName, *elem, context, true, - "while evaluating an element of the `args` argument passed to builtins.derivationStrict").toOwned(); + auto s = state.coerceToString(noPos, *elem, context, + "while evaluating an element of the argument list", + true).toOwned(); drv.args.push_back(s); } } @@ -1142,29 +1171,29 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (i->name == state.sStructuredAttrs) continue; - (*jsonObject)[key] = printValueAsJSON(state, true, *i->value, pos, context); + (*jsonObject)[key] = printValueAsJSON(state, true, *i->value, noPos, context); if (i->name == state.sBuilder) - drv.builder = state.forceString(*i->value, context, posDrvName, "while evaluating the `builder` attribute passed to builtins.derivationStrict"); + drv.builder = state.forceString(*i->value, context, noPos, context_below); else if (i->name == state.sSystem) - drv.platform = state.forceStringNoCtx(*i->value, posDrvName, "while evaluating the `system` attribute passed to builtins.derivationStrict"); + drv.platform = state.forceStringNoCtx(*i->value, noPos, context_below); else if (i->name == state.sOutputHash) - outputHash = state.forceStringNoCtx(*i->value, posDrvName, "while evaluating the `outputHash` attribute passed to builtins.derivationStrict"); + outputHash = state.forceStringNoCtx(*i->value, noPos, context_below); else if (i->name == state.sOutputHashAlgo) - outputHashAlgo = state.forceStringNoCtx(*i->value, posDrvName, "while evaluating the `outputHashAlgo` attribute passed to builtins.derivationStrict"); + outputHashAlgo = state.forceStringNoCtx(*i->value, noPos, context_below); else if (i->name == state.sOutputHashMode) - handleHashMode(state.forceStringNoCtx(*i->value, posDrvName, "while evaluating the `outputHashMode` attribute passed to builtins.derivationStrict")); + handleHashMode(state.forceStringNoCtx(*i->value, noPos, context_below)); else if (i->name == state.sOutputs) { /* Require ‘outputs’ to be a list of strings. */ - state.forceList(*i->value, posDrvName, "while evaluating the `outputs` attribute passed to builtins.derivationStrict"); + state.forceList(*i->value, noPos, context_below); Strings ss; for (auto elem : i->value->listItems()) - ss.emplace_back(state.forceStringNoCtx(*elem, posDrvName, "while evaluating an element of the `outputs` attribute passed to builtins.derivationStrict")); + ss.emplace_back(state.forceStringNoCtx(*elem, noPos, context_below)); handleOutputs(ss); } } else { - auto s = state.coerceToString(i->pos, *i->value, context, true, "while evaluating an attribute passed to builtins.derivationStrict").toOwned(); + auto s = state.coerceToString(noPos, *i->value, context, context_below, true).toOwned(); drv.env.emplace(key, s); if (i->name == state.sBuilder) drv.builder = std::move(s); else if (i->name == state.sSystem) drv.platform = std::move(s); @@ -1178,8 +1207,8 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * } } catch (Error & e) { - e.addTrace(nullptr, - hintfmt("while evaluating the attribute '%1%' of the derivation '%2%'", key, drvName), + e.addTrace(state.positions[i->pos], + hintfmt("while evaluating attribute '%1%' of derivation '%2%'", key, drvName), true); throw; } @@ -1224,20 +1253,20 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (drv.builder == "") state.debugThrowLastTrace(EvalError({ .msg = hintfmt("required attribute 'builder' missing"), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); if (drv.platform == "") state.debugThrowLastTrace(EvalError({ .msg = hintfmt("required attribute 'system' missing"), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); /* Check whether the derivation name is valid. */ if (isDerivation(drvName) && ingestionMethod != ContentAddressMethod { TextHashMethod { } }) state.debugThrowLastTrace(EvalError({ .msg = hintfmt("derivation names are allowed to end in '%s' only if they produce a single derivation file", drvExtension), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); if (outputHash) { @@ -1248,7 +1277,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (outputs.size() != 1 || *(outputs.begin()) != "out") state.debugThrowLastTrace(Error({ .msg = hintfmt("multiple outputs are not supported in fixed-output derivations"), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] })); auto h = newHashAllowEmpty(*outputHash, parseHashTypeOpt(outputHashAlgo)); @@ -1268,7 +1297,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (contentAddressed && isImpure) throw EvalError({ .msg = hintfmt("derivation cannot be both content-addressed and impure"), - .errPos = state.positions[posDrvName] + .errPos = state.positions[noPos] }); auto ht = parseHashTypeOpt(outputHashAlgo).value_or(htSHA256); @@ -1312,7 +1341,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * if (!h) throw AssertionError({ .msg = hintfmt("derivation produced no hash for output '%s'", i), - .errPos = state.positions[posDrvName], + .errPos = state.positions[noPos], }); auto outPath = state.store->makeOutputPath(i, *h, drvName); drv.env[i] = state.store->printStorePath(outPath); @@ -1345,11 +1374,12 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * drvHashes.lock()->insert_or_assign(drvPath, h); } - auto attrs = state.buildBindings(1 + drv.outputs.size()); - attrs.alloc(state.sDrvPath).mkString(drvPathS, {"=" + drvPathS}); + auto result = state.buildBindings(1 + drv.outputs.size()); + result.alloc(state.sDrvPath).mkString(drvPathS, {"=" + drvPathS}); for (auto & i : drv.outputs) - mkOutputString(state, attrs, drvPath, drv, i); - v.mkAttrs(attrs); + mkOutputString(state, result, drvPath, drv, i); + + v.mkAttrs(result); } static RegisterPrimOp primop_derivationStrict(RegisterPrimOp::Info { @@ -1492,7 +1522,9 @@ static RegisterPrimOp primop_pathExists({ static void prim_baseNameOf(EvalState & state, const PosIdx pos, Value * * args, Value & v) { PathSet context; - v.mkString(baseNameOf(*state.coerceToString(pos, *args[0], context, false, false, "while evaluating the first argument passed to builtins.baseNameOf")), context); + v.mkString(baseNameOf(*state.coerceToString(pos, *args[0], context, + "while evaluating the first argument passed to builtins.baseNameOf", + false, false)), context); } static RegisterPrimOp primop_baseNameOf({ @@ -1512,7 +1544,9 @@ static RegisterPrimOp primop_baseNameOf({ static void prim_dirOf(EvalState & state, const PosIdx pos, Value * * args, Value & v) { PathSet context; - auto path = state.coerceToString(pos, *args[0], context, false, false, "while evaluating the first argument passed to builtins.dirOf"); + auto path = state.coerceToString(pos, *args[0], context, + "while evaluating the first argument passed to builtins.dirOf", + false, false); auto dir = dirOf(*path); if (args[0]->type() == nPath) v.mkPath(dir); else v.mkString(dir, context); } @@ -1578,8 +1612,9 @@ static void prim_findFile(EvalState & state, const PosIdx pos, Value * * args, V i = getAttr(state, state.sPath, v2->attrs, "in an element of the __nixPath"); PathSet context; - auto path = state.coerceToString(pos, *i->value, context, false, false, - "while evaluating the `path` attribute of an element of the list passed to builtins.findFile").toOwned(); + auto path = state.coerceToString(pos, *i->value, context, + "while evaluating the `path` attribute of an element of the list passed to builtins.findFile", + false, false).toOwned(); try { auto rewrites = state.realiseContext(context); @@ -1632,23 +1667,73 @@ static RegisterPrimOp primop_hashFile({ .fun = prim_hashFile, }); + +/* Stringize a directory entry enum. Used by `readFileType' and `readDir'. */ +static const char * dirEntTypeToString(unsigned char dtType) +{ + /* Enum DT_(DIR|LNK|REG|UNKNOWN) */ + switch(dtType) { + case DT_REG: return "regular"; break; + case DT_DIR: return "directory"; break; + case DT_LNK: return "symlink"; break; + default: return "unknown"; break; + } + return "unknown"; /* Unreachable */ +} + + +static void prim_readFileType(EvalState & state, const PosIdx pos, Value * * args, Value & v) +{ + auto path = realisePath(state, pos, *args[0]); + /* Retrieve the directory entry type and stringize it. */ + v.mkString(dirEntTypeToString(getFileType(path))); +} + +static RegisterPrimOp primop_readFileType({ + .name = "__readFileType", + .args = {"p"}, + .doc = R"( + Determine the directory entry type of a filesystem node, being + one of "directory", "regular", "symlink", or "unknown". + )", + .fun = prim_readFileType, +}); + /* Read a directory (without . or ..) */ static void prim_readDir(EvalState & state, const PosIdx pos, Value * * args, Value & v) { auto path = realisePath(state, pos, *args[0]); + // Retrieve directory entries for all nodes in a directory. + // This is similar to `getFileType` but is optimized to reduce system calls + // on many systems. DirEntries entries = readDirectory(path); auto attrs = state.buildBindings(entries.size()); + // If we hit unknown directory entry types we may need to fallback to + // using `getFileType` on some systems. + // In order to reduce system calls we make each lookup lazy by using + // `builtins.readFileType` application. + Value * readFileType = nullptr; + for (auto & ent : entries) { - if (ent.type == DT_UNKNOWN) - ent.type = getFileType(path + "/" + ent.name); - attrs.alloc(ent.name).mkString( - ent.type == DT_REG ? "regular" : - ent.type == DT_DIR ? "directory" : - ent.type == DT_LNK ? "symlink" : - "unknown"); + auto & attr = attrs.alloc(ent.name); + if (ent.type == DT_UNKNOWN) { + // Some filesystems or operating systems may not be able to return + // detailed node info quickly in this case we produce a thunk to + // query the file type lazily. + auto epath = state.allocValue(); + Path path2 = path + "/" + ent.name; + epath->mkString(path2); + if (!readFileType) + readFileType = &state.getBuiltin("readFileType"); + attr.mkApp(readFileType, epath); + } else { + // This branch of the conditional is much more likely. + // Here we just stringize the directory entry type. + attr.mkString(dirEntTypeToString(ent.type)); + } } v.mkAttrs(attrs); @@ -2628,14 +2713,9 @@ static void prim_zipAttrsWith(EvalState & state, const PosIdx pos, Value * * arg for (unsigned int n = 0; n < listSize; ++n) { Value * vElem = listElems[n]; - try { - state.forceAttrs(*vElem, noPos, "while evaluating a value of the list passed as second argument to builtins.zipAttrsWith"); - for (auto & attr : *vElem->attrs) - attrsSeen[attr.name].first++; - } catch (TypeError & e) { - e.addTrace(state.positions[pos], hintfmt("while invoking '%s'", "zipAttrsWith")); - state.debugThrowLastTrace(e); - } + state.forceAttrs(*vElem, noPos, "while evaluating a value of the list passed as second argument to builtins.zipAttrsWith"); + for (auto & attr : *vElem->attrs) + attrsSeen[attr.name].first++; } auto attrs = state.buildBindings(attrsSeen.size()); @@ -3019,13 +3099,13 @@ static void prim_genList(EvalState & state, const PosIdx pos, Value * * args, Va auto len = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.genList"); if (len < 0) - state.debugThrowLastTrace(EvalError({ - .msg = hintfmt("cannot create list of size %1%", len), - .errPos = state.positions[pos] - })); + state.error("cannot create list of size %1%", len).debugThrow<EvalError>(); - state.mkList(v, len); + // More strict than striclty (!) necessary, but acceptable + // as evaluating map without accessing any values makes little sense. + state.forceFunction(*args[0], noPos, "while evaluating the first argument passed to builtins.genList"); + state.mkList(v, len); for (unsigned int n = 0; n < (unsigned int) len; ++n) { auto arg = state.allocValue(); arg->mkInt(n); @@ -3073,6 +3153,8 @@ static void prim_sort(EvalState & state, const PosIdx pos, Value * * args, Value auto comparator = [&](Value * a, Value * b) { /* Optimization: if the comparator is lessThan, bypass callFunction. */ + /* TODO: (layus) this is absurd. An optimisation like this + should be outside the lambda creation */ if (args[0]->isPrimOp() && args[0]->primOp->fun == prim_lessThan) return CompareValues(state, noPos, "while evaluating the ordering function passed to builtins.sort")(a, b); @@ -3233,12 +3315,7 @@ static void prim_concatMap(EvalState & state, const PosIdx pos, Value * * args, for (unsigned int n = 0; n < nrLists; ++n) { Value * vElem = args[1]->listElems()[n]; state.callFunction(*args[0], *vElem, lists[n], pos); - try { - state.forceList(lists[n], lists[n].determinePos(args[0]->determinePos(pos)), "while evaluating the return value of the function passed to buitlins.concatMap"); - } catch (TypeError &e) { - e.addTrace(state.positions[pos], hintfmt("while invoking '%s'", "concatMap")); - state.debugThrowLastTrace(e); - } + state.forceList(lists[n], lists[n].determinePos(args[0]->determinePos(pos)), "while evaluating the return value of the function passed to buitlins.concatMap"); len += lists[n].listSize(); } @@ -3418,7 +3495,7 @@ static void prim_lessThan(EvalState & state, const PosIdx pos, Value * * args, V state.forceValue(*args[0], pos); state.forceValue(*args[1], pos); // pos is exact here, no need for a message. - CompareValues comp(state, pos, ""); + CompareValues comp(state, noPos, ""); v.mkBool(comp(args[0], args[1])); } @@ -3445,7 +3522,9 @@ static RegisterPrimOp primop_lessThan({ static void prim_toString(EvalState & state, const PosIdx pos, Value * * args, Value & v) { PathSet context; - auto s = state.coerceToString(pos, *args[0], context, true, false, "while evaluating the first argument passed to builtins.toString"); + auto s = state.coerceToString(pos, *args[0], context, + "while evaluating the first argument passed to builtins.toString", + true, false); v.mkString(*s, context); } @@ -3797,21 +3876,18 @@ static void prim_replaceStrings(EvalState & state, const PosIdx pos, Value * * a state.forceList(*args[0], pos, "while evaluating the first argument passed to builtins.replaceStrings"); state.forceList(*args[1], pos, "while evaluating the second argument passed to builtins.replaceStrings"); if (args[0]->listSize() != args[1]->listSize()) - state.debugThrowLastTrace(EvalError({ - .msg = hintfmt("'from' and 'to' arguments to 'replaceStrings' have different lengths"), - .errPos = state.positions[pos] - })); + state.error("'from' and 'to' arguments passed to builtins.replaceStrings have different lengths").atPos(pos).debugThrow<EvalError>(); std::vector<std::string> from; from.reserve(args[0]->listSize()); for (auto elem : args[0]->listItems()) - from.emplace_back(state.forceString(*elem, pos, "while evaluating one of the strings to replace in builtins.replaceStrings")); + from.emplace_back(state.forceString(*elem, pos, "while evaluating one of the strings to replace passed to builtins.replaceStrings")); std::vector<std::pair<std::string, PathSet>> to; to.reserve(args[1]->listSize()); for (auto elem : args[1]->listItems()) { PathSet ctx; - auto s = state.forceString(*elem, ctx, pos, "while evaluating one of the replacement strings of builtins.replaceStrings"); + auto s = state.forceString(*elem, ctx, pos, "while evaluating one of the replacement strings passed to builtins.replaceStrings"); to.emplace_back(s, std::move(ctx)); } diff --git a/src/libexpr/primops/context.cc b/src/libexpr/primops/context.cc index 0c65a6b98..db43e5771 100644 --- a/src/libexpr/primops/context.cc +++ b/src/libexpr/primops/context.cc @@ -83,15 +83,13 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value * * args, state.forceString(*args[0], context, pos, "while evaluating the argument passed to builtins.getContext"); auto contextInfos = std::map<StorePath, ContextInfo>(); for (const auto & p : context) { - Path drv; - std::string output; NixStringContextElem ctx = NixStringContextElem::parse(*state.store, p); std::visit(overloaded { [&](NixStringContextElem::DrvDeep & d) { contextInfos[d.drvPath].allOutputs = true; }, [&](NixStringContextElem::Built & b) { - contextInfos[b.drvPath].outputs.emplace_back(std::move(output)); + contextInfos[b.drvPath].outputs.emplace_back(std::move(b.output)); }, [&](NixStringContextElem::Opaque & o) { contextInfos[o.path].path = true; diff --git a/src/libexpr/primops/fetchMercurial.cc b/src/libexpr/primops/fetchMercurial.cc index c9c93bdba..c41bd60b6 100644 --- a/src/libexpr/primops/fetchMercurial.cc +++ b/src/libexpr/primops/fetchMercurial.cc @@ -22,7 +22,9 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value * * a for (auto & attr : *args[0]->attrs) { std::string_view n(state.symbols[attr.name]); if (n == "url") - url = state.coerceToString(attr.pos, *attr.value, context, false, false, "while evaluating the `url` attribute passed to builtins.fetchMercurial").toOwned(); + url = state.coerceToString(attr.pos, *attr.value, context, + "while evaluating the `url` attribute passed to builtins.fetchMercurial", + false, false).toOwned(); else if (n == "rev") { // Ugly: unlike fetchGit, here the "rev" attribute can // be both a revision or a branch/tag name. @@ -48,7 +50,9 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value * * a }); } else - url = state.coerceToString(pos, *args[0], context, false, false, "while evaluating the first argument passed to builtins.fetchMercurial").toOwned(); + url = state.coerceToString(pos, *args[0], context, + "while evaluating the first argument passed to builtins.fetchMercurial", + false, false).toOwned(); // FIXME: git externals probably can be used to bypass the URI // whitelist. Ah well. diff --git a/src/libexpr/primops/fetchTree.cc b/src/libexpr/primops/fetchTree.cc index 560a086f0..e194462e4 100644 --- a/src/libexpr/primops/fetchTree.cc +++ b/src/libexpr/primops/fetchTree.cc @@ -125,7 +125,7 @@ static void fetchTree( if (attr.name == state.sType) continue; state.forceValue(*attr.value, attr.pos); if (attr.value->type() == nPath || attr.value->type() == nString) { - auto s = state.coerceToString(attr.pos, *attr.value, context, false, false, "").toOwned(); + auto s = state.coerceToString(attr.pos, *attr.value, context, "", false, false).toOwned(); attrs.emplace(state.symbols[attr.name], state.symbols[attr.name] == "url" ? type == "git" @@ -151,7 +151,9 @@ static void fetchTree( input = fetchers::Input::fromAttrs(std::move(attrs)); } else { - auto url = state.coerceToString(pos, *args[0], context, false, false, "while evaluating the first argument passed to the fetcher").toOwned(); + auto url = state.coerceToString(pos, *args[0], context, + "while evaluating the first argument passed to the fetcher", + false, false).toOwned(); if (type == "git") { fetchers::Attrs attrs; @@ -218,6 +220,9 @@ static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v } else url = state.forceStringNoCtx(*args[0], pos, "while evaluating the url we should fetch"); + if (who == "fetchTarball") + url = evalSettings.resolvePseudoUrl(*url); + state.checkURI(*url); if (name == "") diff --git a/src/libexpr/tests/error_traces.cc b/src/libexpr/tests/error_traces.cc index 8741ecdd2..5e2213f69 100644 --- a/src/libexpr/tests/error_traces.cc +++ b/src/libexpr/tests/error_traces.cc @@ -10,16 +10,71 @@ namespace nix { // Testing eval of PrimOp's class ErrorTraceTest : public LibExprTest { }; + TEST_F(ErrorTraceTest, TraceBuilder) { + ASSERT_THROW( + state.error("Not much").debugThrow<EvalError>(), + EvalError + ); + + ASSERT_THROW( + state.error("Not much").withTrace(noPos, "No more").debugThrow<EvalError>(), + EvalError + ); + + ASSERT_THROW( + try { + try { + state.error("Not much").withTrace(noPos, "No more").debugThrow<EvalError>(); + } catch (Error & e) { + e.addTrace(state.positions[noPos], "Something", ""); + throw; + } + } catch (BaseError & e) { + ASSERT_EQ(PrintToString(e.info().msg), + PrintToString(hintfmt("Not much"))); + auto trace = e.info().traces.rbegin(); + ASSERT_EQ(e.info().traces.size(), 2); + ASSERT_EQ(PrintToString(trace->hint), + PrintToString(hintfmt("No more"))); + trace++; + ASSERT_EQ(PrintToString(trace->hint), + PrintToString(hintfmt("Something"))); + throw; + } + , EvalError + ); + } + + TEST_F(ErrorTraceTest, NestedThrows) { + try { + state.error("Not much").withTrace(noPos, "No more").debugThrow<EvalError>(); + } catch (BaseError & e) { + try { + state.error("Not much more").debugThrow<EvalError>(); + } catch (Error & e2) { + e.addTrace(state.positions[noPos], "Something", ""); + //e2.addTrace(state.positions[noPos], "Something", ""); + ASSERT_TRUE(e.info().traces.size() == 2); + ASSERT_TRUE(e2.info().traces.size() == 0); + ASSERT_FALSE(&e.info() == &e2.info()); + } + } + } + #define ASSERT_TRACE1(args, type, message) \ ASSERT_THROW( \ + std::string expr(args); \ + std::string name = expr.substr(0, expr.find(" ")); \ try { \ - eval("builtins." args); \ + Value v = eval("builtins." args); \ + state.forceValueDeep(v); \ } catch (BaseError & e) { \ ASSERT_EQ(PrintToString(e.info().msg), \ PrintToString(message)); \ + ASSERT_EQ(e.info().traces.size(), 1) << "while testing " args << std::endl << e.what(); \ auto trace = e.info().traces.rbegin(); \ ASSERT_EQ(PrintToString(trace->hint), \ - PrintToString(hintfmt("while calling the '%s' builtin", "genericClosure"))); \ + PrintToString(hintfmt("while calling the '%s' builtin", name))); \ throw; \ } \ , type \ @@ -27,39 +82,42 @@ namespace nix { #define ASSERT_TRACE2(args, type, message, context) \ ASSERT_THROW( \ + std::string expr(args); \ + std::string name = expr.substr(0, expr.find(" ")); \ try { \ - eval("builtins." args); \ + Value v = eval("builtins." args); \ + state.forceValueDeep(v); \ } catch (BaseError & e) { \ ASSERT_EQ(PrintToString(e.info().msg), \ PrintToString(message)); \ + ASSERT_EQ(e.info().traces.size(), 2) << "while testing " args << std::endl << e.what(); \ auto trace = e.info().traces.rbegin(); \ ASSERT_EQ(PrintToString(trace->hint), \ PrintToString(context)); \ ++trace; \ ASSERT_EQ(PrintToString(trace->hint), \ - PrintToString(hintfmt("while calling the '%s' builtin", "genericClosure"))); \ + PrintToString(hintfmt("while calling the '%s' builtin", name))); \ throw; \ } \ , type \ ) - TEST_F(ErrorTraceTest, genericClosure) { \ + TEST_F(ErrorTraceTest, genericClosure) { ASSERT_TRACE2("genericClosure 1", TypeError, hintfmt("value is %s while a set was expected", "an integer"), hintfmt("while evaluating the first argument passed to builtins.genericClosure")); - ASSERT_TRACE1("genericClosure {}", + ASSERT_TRACE2("genericClosure {}", TypeError, - hintfmt("attribute '%s' missing %s", "startSet", normaltxt("in the attrset passed as argument to builtins.genericClosure"))); + hintfmt("attribute '%s' missing", "startSet"), + hintfmt("in the attrset passed as argument to builtins.genericClosure")); ASSERT_TRACE2("genericClosure { startSet = 1; }", TypeError, hintfmt("value is %s while a list was expected", "an integer"), hintfmt("while evaluating the 'startSet' attribute passed as argument to builtins.genericClosure")); - // Okay: "genericClosure { startSet = []; }" - ASSERT_TRACE2("genericClosure { startSet = [{ key = 1;}]; operator = true; }", TypeError, hintfmt("value is %s while a function was expected", "a Boolean"), @@ -68,16 +126,17 @@ namespace nix { ASSERT_TRACE2("genericClosure { startSet = [{ key = 1;}]; operator = item: true; }", TypeError, hintfmt("value is %s while a list was expected", "a Boolean"), - hintfmt("while evaluating the return value of the `operator` passed to builtins.genericClosure")); // TODO: inconsistent naming + hintfmt("while evaluating the return value of the `operator` passed to builtins.genericClosure")); ASSERT_TRACE2("genericClosure { startSet = [{ key = 1;}]; operator = item: [ true ]; }", TypeError, hintfmt("value is %s while a set was expected", "a Boolean"), hintfmt("while evaluating one of the elements generated by (or initially passed to) builtins.genericClosure")); - ASSERT_TRACE1("genericClosure { startSet = [{ key = 1;}]; operator = item: [ {} ]; }", + ASSERT_TRACE2("genericClosure { startSet = [{ key = 1;}]; operator = item: [ {} ]; }", TypeError, - hintfmt("attribute '%s' missing %s", "key", normaltxt("in one of the attrsets generated by (or initially passed to) builtins.genericClosure"))); + hintfmt("attribute '%s' missing", "key"), + hintfmt("in one of the attrsets generated by (or initially passed to) builtins.genericClosure")); ASSERT_TRACE2("genericClosure { startSet = [{ key = 1;}]; operator = item: [{ key = ''a''; }]; }", EvalError, @@ -91,4 +150,1149 @@ namespace nix { } + + TEST_F(ErrorTraceTest, replaceStrings) { + ASSERT_TRACE2("replaceStrings 0 0 {}", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.replaceStrings")); + + ASSERT_TRACE2("replaceStrings [] 0 {}", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the second argument passed to builtins.replaceStrings")); + + ASSERT_TRACE1("replaceStrings [ 0 ] [] {}", + EvalError, + hintfmt("'from' and 'to' arguments passed to builtins.replaceStrings have different lengths")); + + ASSERT_TRACE2("replaceStrings [ 1 ] [ \"new\" ] {}", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating one of the strings to replace passed to builtins.replaceStrings")); + + ASSERT_TRACE2("replaceStrings [ \"old\" ] [ true ] {}", + TypeError, + hintfmt("value is %s while a string was expected", "a Boolean"), + hintfmt("while evaluating one of the replacement strings passed to builtins.replaceStrings")); + + ASSERT_TRACE2("replaceStrings [ \"old\" ] [ \"new\" ] {}", + TypeError, + hintfmt("value is %s while a string was expected", "a set"), + hintfmt("while evaluating the third argument passed to builtins.replaceStrings")); + + } + + + TEST_F(ErrorTraceTest, scopedImport) { + } + + + TEST_F(ErrorTraceTest, import) { + } + + + TEST_F(ErrorTraceTest, typeOf) { + } + + + TEST_F(ErrorTraceTest, isNull) { + } + + + TEST_F(ErrorTraceTest, isFunction) { + } + + + TEST_F(ErrorTraceTest, isInt) { + } + + + TEST_F(ErrorTraceTest, isFloat) { + } + + + TEST_F(ErrorTraceTest, isString) { + } + + + TEST_F(ErrorTraceTest, isBool) { + } + + + TEST_F(ErrorTraceTest, isPath) { + } + + + TEST_F(ErrorTraceTest, break) { + } + + + TEST_F(ErrorTraceTest, abort) { + } + + + TEST_F(ErrorTraceTest, throw) { + } + + + TEST_F(ErrorTraceTest, addErrorContext) { + } + + + TEST_F(ErrorTraceTest, ceil) { + ASSERT_TRACE2("ceil \"foo\"", + TypeError, + hintfmt("value is %s while a float was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.ceil")); + + } + + + TEST_F(ErrorTraceTest, floor) { + ASSERT_TRACE2("floor \"foo\"", + TypeError, + hintfmt("value is %s while a float was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.floor")); + + } + + + TEST_F(ErrorTraceTest, tryEval) { + } + + + TEST_F(ErrorTraceTest, getEnv) { + ASSERT_TRACE2("getEnv [ ]", + TypeError, + hintfmt("value is %s while a string was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.getEnv")); + + } + + + TEST_F(ErrorTraceTest, seq) { + } + + + TEST_F(ErrorTraceTest, deepSeq) { + } + + + TEST_F(ErrorTraceTest, trace) { + } + + + TEST_F(ErrorTraceTest, placeholder) { + ASSERT_TRACE2("placeholder []", + TypeError, + hintfmt("value is %s while a string was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.placeholder")); + + } + + + TEST_F(ErrorTraceTest, toPath) { + ASSERT_TRACE2("toPath []", + TypeError, + hintfmt("cannot coerce %s to a string", "a list"), + hintfmt("while evaluating the first argument passed to builtins.toPath")); + + ASSERT_TRACE2("toPath \"foo\"", + EvalError, + hintfmt("string '%s' doesn't represent an absolute path", "foo"), + hintfmt("while evaluating the first argument passed to builtins.toPath")); + + } + + + TEST_F(ErrorTraceTest, storePath) { + ASSERT_TRACE2("storePath true", + TypeError, + hintfmt("cannot coerce %s to a string", "a Boolean"), + hintfmt("while evaluating the first argument passed to builtins.storePath")); + + } + + + TEST_F(ErrorTraceTest, pathExists) { + ASSERT_TRACE2("pathExists []", + TypeError, + hintfmt("cannot coerce %s to a string", "a list"), + hintfmt("while realising the context of a path")); + + ASSERT_TRACE2("pathExists \"zorglub\"", + EvalError, + hintfmt("string '%s' doesn't represent an absolute path", "zorglub"), + hintfmt("while realising the context of a path")); + + } + + + TEST_F(ErrorTraceTest, baseNameOf) { + ASSERT_TRACE2("baseNameOf []", + TypeError, + hintfmt("cannot coerce %s to a string", "a list"), + hintfmt("while evaluating the first argument passed to builtins.baseNameOf")); + + } + + + TEST_F(ErrorTraceTest, dirOf) { + } + + + TEST_F(ErrorTraceTest, readFile) { + } + + + TEST_F(ErrorTraceTest, findFile) { + } + + + TEST_F(ErrorTraceTest, hashFile) { + } + + + TEST_F(ErrorTraceTest, readDir) { + } + + + TEST_F(ErrorTraceTest, toXML) { + } + + + TEST_F(ErrorTraceTest, toJSON) { + } + + + TEST_F(ErrorTraceTest, fromJSON) { + } + + + TEST_F(ErrorTraceTest, toFile) { + } + + + TEST_F(ErrorTraceTest, filterSource) { + ASSERT_TRACE2("filterSource [] []", + TypeError, + hintfmt("cannot coerce %s to a string", "a list"), + hintfmt("while evaluating the second argument (the path to filter) passed to builtins.filterSource")); + + ASSERT_TRACE2("filterSource [] \"foo\"", + EvalError, + hintfmt("string '%s' doesn't represent an absolute path", "foo"), + hintfmt("while evaluating the second argument (the path to filter) passed to builtins.filterSource")); + + ASSERT_TRACE2("filterSource [] ./.", + TypeError, + hintfmt("value is %s while a function was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.filterSource")); + + // Usupported by store "dummy" + + // ASSERT_TRACE2("filterSource (_: 1) ./.", + // TypeError, + // hintfmt("attempt to call something which is not a function but %s", "an integer"), + // hintfmt("while adding path '/home/layus/projects/nix'")); + + // ASSERT_TRACE2("filterSource (_: _: 1) ./.", + // TypeError, + // hintfmt("value is %s while a Boolean was expected", "an integer"), + // hintfmt("while evaluating the return value of the path filter function")); + + } + + + TEST_F(ErrorTraceTest, path) { + } + + + TEST_F(ErrorTraceTest, attrNames) { + ASSERT_TRACE2("attrNames []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the argument passed to builtins.attrNames")); + + } + + + TEST_F(ErrorTraceTest, attrValues) { + ASSERT_TRACE2("attrValues []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the argument passed to builtins.attrValues")); + + } + + + TEST_F(ErrorTraceTest, getAttr) { + ASSERT_TRACE2("getAttr [] []", + TypeError, + hintfmt("value is %s while a string was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.getAttr")); + + ASSERT_TRACE2("getAttr \"foo\" []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the second argument passed to builtins.getAttr")); + + ASSERT_TRACE2("getAttr \"foo\" {}", + TypeError, + hintfmt("attribute '%s' missing", "foo"), + hintfmt("in the attribute set under consideration")); + + } + + + TEST_F(ErrorTraceTest, unsafeGetAttrPos) { + } + + + TEST_F(ErrorTraceTest, hasAttr) { + ASSERT_TRACE2("hasAttr [] []", + TypeError, + hintfmt("value is %s while a string was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.hasAttr")); + + ASSERT_TRACE2("hasAttr \"foo\" []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the second argument passed to builtins.hasAttr")); + + } + + + TEST_F(ErrorTraceTest, isAttrs) { + } + + + TEST_F(ErrorTraceTest, removeAttrs) { + ASSERT_TRACE2("removeAttrs \"\" \"\"", + TypeError, + hintfmt("value is %s while a set was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.removeAttrs")); + + ASSERT_TRACE2("removeAttrs \"\" [ 1 ]", + TypeError, + hintfmt("value is %s while a set was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.removeAttrs")); + + ASSERT_TRACE2("removeAttrs \"\" [ \"1\" ]", + TypeError, + hintfmt("value is %s while a set was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.removeAttrs")); + + } + + + TEST_F(ErrorTraceTest, listToAttrs) { + ASSERT_TRACE2("listToAttrs 1", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the argument passed to builtins.listToAttrs")); + + ASSERT_TRACE2("listToAttrs [ 1 ]", + TypeError, + hintfmt("value is %s while a set was expected", "an integer"), + hintfmt("while evaluating an element of the list passed to builtins.listToAttrs")); + + ASSERT_TRACE2("listToAttrs [ {} ]", + TypeError, + hintfmt("attribute '%s' missing", "name"), + hintfmt("in a {name=...; value=...;} pair")); + + ASSERT_TRACE2("listToAttrs [ { name = 1; } ]", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the `name` attribute of an element of the list passed to builtins.listToAttrs")); + + ASSERT_TRACE2("listToAttrs [ { name = \"foo\"; } ]", + TypeError, + hintfmt("attribute '%s' missing", "value"), + hintfmt("in a {name=...; value=...;} pair")); + + } + + + TEST_F(ErrorTraceTest, intersectAttrs) { + ASSERT_TRACE2("intersectAttrs [] []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.intersectAttrs")); + + ASSERT_TRACE2("intersectAttrs {} []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the second argument passed to builtins.intersectAttrs")); + + } + + + TEST_F(ErrorTraceTest, catAttrs) { + ASSERT_TRACE2("catAttrs [] {}", + TypeError, + hintfmt("value is %s while a string was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.catAttrs")); + + ASSERT_TRACE2("catAttrs \"foo\" {}", + TypeError, + hintfmt("value is %s while a list was expected", "a set"), + hintfmt("while evaluating the second argument passed to builtins.catAttrs")); + + ASSERT_TRACE2("catAttrs \"foo\" [ 1 ]", + TypeError, + hintfmt("value is %s while a set was expected", "an integer"), + hintfmt("while evaluating an element in the list passed as second argument to builtins.catAttrs")); + + ASSERT_TRACE2("catAttrs \"foo\" [ { foo = 1; } 1 { bar = 5;} ]", + TypeError, + hintfmt("value is %s while a set was expected", "an integer"), + hintfmt("while evaluating an element in the list passed as second argument to builtins.catAttrs")); + + } + + + TEST_F(ErrorTraceTest, functionArgs) { + ASSERT_TRACE1("functionArgs {}", + TypeError, + hintfmt("'functionArgs' requires a function")); + + } + + + TEST_F(ErrorTraceTest, mapAttrs) { + ASSERT_TRACE2("mapAttrs [] []", + TypeError, + hintfmt("value is %s while a set was expected", "a list"), + hintfmt("while evaluating the second argument passed to builtins.mapAttrs")); + + // XXX: defered + // ASSERT_TRACE2("mapAttrs \"\" { foo.bar = 1; }", + // TypeError, + // hintfmt("attempt to call something which is not a function but %s", "a string"), + // hintfmt("while evaluating the attribute 'foo'")); + + // ASSERT_TRACE2("mapAttrs (x: x + \"1\") { foo.bar = 1; }", + // TypeError, + // hintfmt("attempt to call something which is not a function but %s", "a string"), + // hintfmt("while evaluating the attribute 'foo'")); + + // ASSERT_TRACE2("mapAttrs (x: y: x + 1) { foo.bar = 1; }", + // TypeError, + // hintfmt("cannot coerce %s to a string", "an integer"), + // hintfmt("while evaluating a path segment")); + + } + + + TEST_F(ErrorTraceTest, zipAttrsWith) { + ASSERT_TRACE2("zipAttrsWith [] [ 1 ]", + TypeError, + hintfmt("value is %s while a function was expected", "a list"), + hintfmt("while evaluating the first argument passed to builtins.zipAttrsWith")); + + ASSERT_TRACE2("zipAttrsWith (_: 1) [ 1 ]", + TypeError, + hintfmt("value is %s while a set was expected", "an integer"), + hintfmt("while evaluating a value of the list passed as second argument to builtins.zipAttrsWith")); + + // XXX: How to properly tell that the fucntion takes two arguments ? + // The same question also applies to sort, and maybe others. + // Due to lazyness, we only create a thunk, and it fails later on. + // ASSERT_TRACE2("zipAttrsWith (_: 1) [ { foo = 1; } ]", + // TypeError, + // hintfmt("attempt to call something which is not a function but %s", "an integer"), + // hintfmt("while evaluating the attribute 'foo'")); + + // XXX: Also deferred deeply + // ASSERT_TRACE2("zipAttrsWith (a: b: a + b) [ { foo = 1; } { foo = 2; } ]", + // TypeError, + // hintfmt("cannot coerce %s to a string", "a list"), + // hintfmt("while evaluating a path segment")); + + } + + + TEST_F(ErrorTraceTest, isList) { + } + + + TEST_F(ErrorTraceTest, elemAt) { + ASSERT_TRACE2("elemAt \"foo\" (-1)", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.elemAt")); + + ASSERT_TRACE1("elemAt [] (-1)", + Error, + hintfmt("list index %d is out of bounds", -1)); + + ASSERT_TRACE1("elemAt [\"foo\"] 3", + Error, + hintfmt("list index %d is out of bounds", 3)); + + } + + + TEST_F(ErrorTraceTest, head) { + ASSERT_TRACE2("head 1", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.elemAt")); + + ASSERT_TRACE1("head []", + Error, + hintfmt("list index %d is out of bounds", 0)); + + } + + + TEST_F(ErrorTraceTest, tail) { + ASSERT_TRACE2("tail 1", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.tail")); + + ASSERT_TRACE1("tail []", + Error, + hintfmt("'tail' called on an empty list")); + + } + + + TEST_F(ErrorTraceTest, map) { + ASSERT_TRACE2("map 1 \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.map")); + + ASSERT_TRACE2("map 1 [ 1 ]", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.map")); + + } + + + TEST_F(ErrorTraceTest, filter) { + ASSERT_TRACE2("filter 1 \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.filter")); + + ASSERT_TRACE2("filter 1 [ \"foo\" ]", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.filter")); + + ASSERT_TRACE2("filter (_: 5) [ \"foo\" ]", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the return value of the filtering function passed to builtins.filter")); + + } + + + TEST_F(ErrorTraceTest, elem) { + ASSERT_TRACE2("elem 1 \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.elem")); + + } + + + TEST_F(ErrorTraceTest, concatLists) { + ASSERT_TRACE2("concatLists 1", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.concatLists")); + + ASSERT_TRACE2("concatLists [ 1 ]", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating a value of the list passed to builtins.concatLists")); + + ASSERT_TRACE2("concatLists [ [1] \"foo\" ]", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating a value of the list passed to builtins.concatLists")); + + } + + + TEST_F(ErrorTraceTest, length) { + ASSERT_TRACE2("length 1", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.length")); + + ASSERT_TRACE2("length \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the first argument passed to builtins.length")); + + } + + + TEST_F(ErrorTraceTest, foldlPrime) { + ASSERT_TRACE2("foldl' 1 \"foo\" true", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.foldlStrict")); + + ASSERT_TRACE2("foldl' (_: 1) \"foo\" true", + TypeError, + hintfmt("value is %s while a list was expected", "a Boolean"), + hintfmt("while evaluating the third argument passed to builtins.foldlStrict")); + + ASSERT_TRACE1("foldl' (_: 1) \"foo\" [ true ]", + TypeError, + hintfmt("attempt to call something which is not a function but %s", "an integer")); + + ASSERT_TRACE2("foldl' (a: b: a && b) \"foo\" [ true ]", + TypeError, + hintfmt("value is %s while a Boolean was expected", "a string"), + hintfmt("in the left operand of the AND (&&) operator")); + + } + + + TEST_F(ErrorTraceTest, any) { + ASSERT_TRACE2("any 1 \"foo\"", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.any")); + + ASSERT_TRACE2("any (_: 1) \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.any")); + + ASSERT_TRACE2("any (_: 1) [ \"foo\" ]", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the return value of the function passed to builtins.any")); + + } + + + TEST_F(ErrorTraceTest, all) { + ASSERT_TRACE2("all 1 \"foo\"", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.all")); + + ASSERT_TRACE2("all (_: 1) \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.all")); + + ASSERT_TRACE2("all (_: 1) [ \"foo\" ]", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the return value of the function passed to builtins.all")); + + } + + + TEST_F(ErrorTraceTest, genList) { + ASSERT_TRACE2("genList 1 \"foo\"", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.genList")); + + ASSERT_TRACE2("genList 1 2", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.genList", "an integer")); + + // XXX: defered + // ASSERT_TRACE2("genList (x: x + \"foo\") 2 #TODO", + // TypeError, + // hintfmt("cannot add %s to an integer", "a string"), + // hintfmt("while evaluating anonymous lambda")); + + ASSERT_TRACE1("genList false (-3)", + EvalError, + hintfmt("cannot create list of size %d", -3)); + + } + + + TEST_F(ErrorTraceTest, sort) { + ASSERT_TRACE2("sort 1 \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.sort")); + + ASSERT_TRACE2("sort 1 [ \"foo\" ]", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.sort")); + + ASSERT_TRACE1("sort (_: 1) [ \"foo\" \"bar\" ]", + TypeError, + hintfmt("attempt to call something which is not a function but %s", "an integer")); + + ASSERT_TRACE2("sort (_: _: 1) [ \"foo\" \"bar\" ]", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the return value of the sorting function passed to builtins.sort")); + + // XXX: Trace too deep, need better asserts + // ASSERT_TRACE1("sort (a: b: a <= b) [ \"foo\" {} ] # TODO", + // TypeError, + // hintfmt("cannot compare %s with %s", "a string", "a set")); + + // ASSERT_TRACE1("sort (a: b: a <= b) [ {} {} ] # TODO", + // TypeError, + // hintfmt("cannot compare %s with %s; values of that type are incomparable", "a set", "a set")); + + } + + + TEST_F(ErrorTraceTest, partition) { + ASSERT_TRACE2("partition 1 \"foo\"", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.partition")); + + ASSERT_TRACE2("partition (_: 1) \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.partition")); + + ASSERT_TRACE2("partition (_: 1) [ \"foo\" ]", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the return value of the partition function passed to builtins.partition")); + + } + + + TEST_F(ErrorTraceTest, groupBy) { + ASSERT_TRACE2("groupBy 1 \"foo\"", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.groupBy")); + + ASSERT_TRACE2("groupBy (_: 1) \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.groupBy")); + + ASSERT_TRACE2("groupBy (x: x) [ \"foo\" \"bar\" 1 ]", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the return value of the grouping function passed to builtins.groupBy")); + + } + + + TEST_F(ErrorTraceTest, concatMap) { + ASSERT_TRACE2("concatMap 1 \"foo\"", + TypeError, + hintfmt("value is %s while a function was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.concatMap")); + + ASSERT_TRACE2("concatMap (x: 1) \"foo\"", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the second argument passed to builtins.concatMap")); + + ASSERT_TRACE2("concatMap (x: 1) [ \"foo\" ] # TODO", + TypeError, + hintfmt("value is %s while a list was expected", "an integer"), + hintfmt("while evaluating the return value of the function passed to buitlins.concatMap")); + + ASSERT_TRACE2("concatMap (x: \"foo\") [ 1 2 ] # TODO", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the return value of the function passed to buitlins.concatMap")); + + } + + + TEST_F(ErrorTraceTest, add) { + ASSERT_TRACE2("add \"foo\" 1", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the first argument of the addition")); + + ASSERT_TRACE2("add 1 \"foo\"", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the second argument of the addition")); + + } + + + TEST_F(ErrorTraceTest, sub) { + ASSERT_TRACE2("sub \"foo\" 1", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the first argument of the subtraction")); + + ASSERT_TRACE2("sub 1 \"foo\"", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the second argument of the subtraction")); + + } + + + TEST_F(ErrorTraceTest, mul) { + ASSERT_TRACE2("mul \"foo\" 1", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the first argument of the multiplication")); + + ASSERT_TRACE2("mul 1 \"foo\"", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the second argument of the multiplication")); + + } + + + TEST_F(ErrorTraceTest, div) { + ASSERT_TRACE2("div \"foo\" 1 # TODO: an integer was expected -> a number", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the first operand of the division")); + + ASSERT_TRACE2("div 1 \"foo\"", + TypeError, + hintfmt("value is %s while a float was expected", "a string"), + hintfmt("while evaluating the second operand of the division")); + + ASSERT_TRACE1("div \"foo\" 0", + EvalError, + hintfmt("division by zero")); + + } + + + TEST_F(ErrorTraceTest, bitAnd) { + ASSERT_TRACE2("bitAnd 1.1 2", + TypeError, + hintfmt("value is %s while an integer was expected", "a float"), + hintfmt("while evaluating the first argument passed to builtins.bitAnd")); + + ASSERT_TRACE2("bitAnd 1 2.2", + TypeError, + hintfmt("value is %s while an integer was expected", "a float"), + hintfmt("while evaluating the second argument passed to builtins.bitAnd")); + + } + + + TEST_F(ErrorTraceTest, bitOr) { + ASSERT_TRACE2("bitOr 1.1 2", + TypeError, + hintfmt("value is %s while an integer was expected", "a float"), + hintfmt("while evaluating the first argument passed to builtins.bitOr")); + + ASSERT_TRACE2("bitOr 1 2.2", + TypeError, + hintfmt("value is %s while an integer was expected", "a float"), + hintfmt("while evaluating the second argument passed to builtins.bitOr")); + + } + + + TEST_F(ErrorTraceTest, bitXor) { + ASSERT_TRACE2("bitXor 1.1 2", + TypeError, + hintfmt("value is %s while an integer was expected", "a float"), + hintfmt("while evaluating the first argument passed to builtins.bitXor")); + + ASSERT_TRACE2("bitXor 1 2.2", + TypeError, + hintfmt("value is %s while an integer was expected", "a float"), + hintfmt("while evaluating the second argument passed to builtins.bitXor")); + + } + + + TEST_F(ErrorTraceTest, lessThan) { + ASSERT_TRACE1("lessThan 1 \"foo\"", + EvalError, + hintfmt("cannot compare %s with %s", "an integer", "a string")); + + ASSERT_TRACE1("lessThan {} {}", + EvalError, + hintfmt("cannot compare %s with %s; values of that type are incomparable", "a set", "a set")); + + ASSERT_TRACE2("lessThan [ 1 2 ] [ \"foo\" ]", + EvalError, + hintfmt("cannot compare %s with %s", "an integer", "a string"), + hintfmt("while comparing two list elements")); + + } + + + TEST_F(ErrorTraceTest, toString) { + ASSERT_TRACE2("toString { a = 1; }", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating the first argument passed to builtins.toString")); + + } + + + TEST_F(ErrorTraceTest, substring) { + ASSERT_TRACE2("substring {} \"foo\" true", + TypeError, + hintfmt("value is %s while an integer was expected", "a set"), + hintfmt("while evaluating the first argument (the start offset) passed to builtins.substring")); + + ASSERT_TRACE2("substring 3 \"foo\" true", + TypeError, + hintfmt("value is %s while an integer was expected", "a string"), + hintfmt("while evaluating the second argument (the substring length) passed to builtins.substring")); + + ASSERT_TRACE2("substring 0 3 {}", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating the third argument (the string) passed to builtins.substring")); + + ASSERT_TRACE1("substring (-3) 3 \"sometext\"", + EvalError, + hintfmt("negative start position in 'substring'")); + + } + + + TEST_F(ErrorTraceTest, stringLength) { + ASSERT_TRACE2("stringLength {} # TODO: context is missing ???", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating the argument passed to builtins.stringLength")); + + } + + + TEST_F(ErrorTraceTest, hashString) { + ASSERT_TRACE2("hashString 1 {}", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.hashString")); + + ASSERT_TRACE1("hashString \"foo\" \"content\"", + UsageError, + hintfmt("unknown hash algorithm '%s'", "foo")); + + ASSERT_TRACE2("hashString \"sha256\" {}", + TypeError, + hintfmt("value is %s while a string was expected", "a set"), + hintfmt("while evaluating the second argument passed to builtins.hashString")); + + } + + + TEST_F(ErrorTraceTest, match) { + ASSERT_TRACE2("match 1 {}", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.match")); + + ASSERT_TRACE2("match \"foo\" {}", + TypeError, + hintfmt("value is %s while a string was expected", "a set"), + hintfmt("while evaluating the second argument passed to builtins.match")); + + ASSERT_TRACE1("match \"(.*\" \"\"", + EvalError, + hintfmt("invalid regular expression '%s'", "(.*")); + + } + + + TEST_F(ErrorTraceTest, split) { + ASSERT_TRACE2("split 1 {}", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.split")); + + ASSERT_TRACE2("split \"foo\" {}", + TypeError, + hintfmt("value is %s while a string was expected", "a set"), + hintfmt("while evaluating the second argument passed to builtins.split")); + + ASSERT_TRACE1("split \"f(o*o\" \"1foo2\"", + EvalError, + hintfmt("invalid regular expression '%s'", "f(o*o")); + + } + + + TEST_F(ErrorTraceTest, concatStringsSep) { + ASSERT_TRACE2("concatStringsSep 1 {}", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument (the separator string) passed to builtins.concatStringsSep")); + + ASSERT_TRACE2("concatStringsSep \"foo\" {}", + TypeError, + hintfmt("value is %s while a list was expected", "a set"), + hintfmt("while evaluating the second argument (the list of strings to concat) passed to builtins.concatStringsSep")); + + ASSERT_TRACE2("concatStringsSep \"foo\" [ 1 2 {} ] # TODO: coerce to string is buggy", + TypeError, + hintfmt("cannot coerce %s to a string", "an integer"), + hintfmt("while evaluating one element of the list of strings to concat passed to builtins.concatStringsSep")); + + } + + + TEST_F(ErrorTraceTest, parseDrvName) { + ASSERT_TRACE2("parseDrvName 1", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.parseDrvName")); + + } + + + TEST_F(ErrorTraceTest, compareVersions) { + ASSERT_TRACE2("compareVersions 1 {}", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.compareVersions")); + + ASSERT_TRACE2("compareVersions \"abd\" {}", + TypeError, + hintfmt("value is %s while a string was expected", "a set"), + hintfmt("while evaluating the second argument passed to builtins.compareVersions")); + + } + + + TEST_F(ErrorTraceTest, splitVersion) { + ASSERT_TRACE2("splitVersion 1", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the first argument passed to builtins.splitVersion")); + + } + + + TEST_F(ErrorTraceTest, traceVerbose) { + } + + + /* // Needs different ASSERTs + TEST_F(ErrorTraceTest, derivationStrict) { + ASSERT_TRACE2("derivationStrict \"\"", + TypeError, + hintfmt("value is %s while a set was expected", "a string"), + hintfmt("while evaluating the argument passed to builtins.derivationStrict")); + + ASSERT_TRACE2("derivationStrict {}", + TypeError, + hintfmt("attribute '%s' missing", "name"), + hintfmt("in the attrset passed as argument to builtins.derivationStrict")); + + ASSERT_TRACE2("derivationStrict { name = 1; }", + TypeError, + hintfmt("value is %s while a string was expected", "an integer"), + hintfmt("while evaluating the `name` attribute passed to builtins.derivationStrict")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; }", + TypeError, + hintfmt("required attribute 'builder' missing"), + hintfmt("while evaluating derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; __structuredAttrs = 15; }", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the `__structuredAttrs` attribute passed to builtins.derivationStrict")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; __ignoreNulls = 15; }", + TypeError, + hintfmt("value is %s while a Boolean was expected", "an integer"), + hintfmt("while evaluating the `__ignoreNulls` attribute passed to builtins.derivationStrict")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; outputHashMode = 15; }", + TypeError, + hintfmt("invalid value '15' for 'outputHashMode' attribute"), + hintfmt("while evaluating the attribute 'outputHashMode' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; outputHashMode = \"custom\"; }", + TypeError, + hintfmt("invalid value 'custom' for 'outputHashMode' attribute"), + hintfmt("while evaluating the attribute 'outputHashMode' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = {}; }", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating the attribute 'system' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = {}; }", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating the attribute 'outputs' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"drv\"; }", + TypeError, + hintfmt("invalid derivation output name 'drv'"), + hintfmt("while evaluating the attribute 'outputs' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = []; }", + TypeError, + hintfmt("derivation cannot have an empty set of outputs"), + hintfmt("while evaluating the attribute 'outputs' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = [ \"drv\" ]; }", + TypeError, + hintfmt("invalid derivation output name 'drv'"), + hintfmt("while evaluating the attribute 'outputs' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = [ \"out\" \"out\" ]; }", + TypeError, + hintfmt("duplicate derivation output 'out'"), + hintfmt("while evaluating the attribute 'outputs' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; __contentAddressed = \"true\"; }", + TypeError, + hintfmt("value is %s while a Boolean was expected", "a string"), + hintfmt("while evaluating the attribute '__contentAddressed' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; __impure = \"true\"; }", + TypeError, + hintfmt("value is %s while a Boolean was expected", "a string"), + hintfmt("while evaluating the attribute '__impure' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; __impure = \"true\"; }", + TypeError, + hintfmt("value is %s while a Boolean was expected", "a string"), + hintfmt("while evaluating the attribute '__impure' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; args = \"foo\"; }", + TypeError, + hintfmt("value is %s while a list was expected", "a string"), + hintfmt("while evaluating the attribute 'args' of derivation 'foo'")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; args = [ {} ]; }", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating an element of the argument list")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; args = [ \"a\" {} ]; }", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating an element of the argument list")); + + ASSERT_TRACE2("derivationStrict { name = \"foo\"; builder = 1; system = 1; outputs = \"out\"; FOO = {}; }", + TypeError, + hintfmt("cannot coerce %s to a string", "a set"), + hintfmt("while evaluating the attribute 'FOO' of derivation 'foo'")); + + } + */ + } /* namespace nix */ diff --git a/src/libexpr/value.hh b/src/libexpr/value.hh index 7d3f6d700..508dbe218 100644 --- a/src/libexpr/value.hh +++ b/src/libexpr/value.hh @@ -89,7 +89,7 @@ class ExternalValueBase /* Coerce the value to a string. Defaults to uncoercable, i.e. throws an * error. */ - virtual std::string coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore, std::string_view errorCtx) const; + virtual std::string coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore) const; /* Compare to another value of the same type. Defaults to uncomparable, * i.e. always false. diff --git a/src/libfetchers/tarball.cc b/src/libfetchers/tarball.cc index 155b86cc4..302046610 100644 --- a/src/libfetchers/tarball.cc +++ b/src/libfetchers/tarball.cc @@ -72,15 +72,13 @@ DownloadFileResult downloadFile( auto hash = hashString(htSHA256, res.data); ValidPathInfo info { *store, - { - .name = name, - .info = FixedOutputInfo { - { - .method = FileIngestionMethod::Flat, - .hash = hash, - }, - .references = {}, + name, + FixedOutputInfo { + { + .method = FileIngestionMethod::Flat, + .hash = hash, }, + .references = {}, }, hashString(htSHA256, sink.s), }; diff --git a/src/libstore/binary-cache-store.cc b/src/libstore/binary-cache-store.cc index ac41add2c..9058bb8b1 100644 --- a/src/libstore/binary-cache-store.cc +++ b/src/libstore/binary-cache-store.cc @@ -307,17 +307,15 @@ StorePath BinaryCacheStore::addToStoreFromDump(Source & dump, std::string_view n return addToStoreCommon(dump, repair, CheckSigs, [&](HashResult nar) { ValidPathInfo info { *this, - { - .name = std::string { name }, - .info = FixedOutputInfo { - { - .method = method, - .hash = nar.first, - }, - .references = { - .others = references, - .self = false, - }, + name, + FixedOutputInfo { + { + .method = method, + .hash = nar.first, + }, + .references = { + .others = references, + .self = false, }, }, nar.first, @@ -427,17 +425,15 @@ StorePath BinaryCacheStore::addToStore( return addToStoreCommon(*source, repair, CheckSigs, [&](HashResult nar) { ValidPathInfo info { *this, - { - .name = std::string { name }, - .info = FixedOutputInfo { - { - .method = method, - .hash = h, - }, - .references = { - .others = references, - .self = false, - }, + name, + FixedOutputInfo { + { + .method = method, + .hash = h, + }, + .references = { + .others = references, + .self = false, }, }, nar.first, @@ -465,12 +461,10 @@ StorePath BinaryCacheStore::addTextToStore( return addToStoreCommon(source, repair, CheckSigs, [&](HashResult nar) { ValidPathInfo info { *this, - { - .name = std::string { name }, - .info = TextInfo { - { .hash = textHash }, - references, - }, + std::string { name }, + TextInfo { + { .hash = textHash }, + references, }, nar.first, }; diff --git a/src/libstore/build/local-derivation-goal.cc b/src/libstore/build/local-derivation-goal.cc index 88cb33d0f..6f3048b63 100644 --- a/src/libstore/build/local-derivation-goal.cc +++ b/src/libstore/build/local-derivation-goal.cc @@ -2484,13 +2484,11 @@ DrvOutputs LocalDerivationGoal::registerOutputs() auto got = caSink.finish().first; ValidPathInfo newInfo0 { worker.store, - { - .name = outputPathName(drv->name, outputName), - .info = contentAddressFromMethodHashAndRefs( - outputHash.method, - std::move(got), - rewriteRefs()), - }, + outputPathName(drv->name, outputName), + contentAddressFromMethodHashAndRefs( + outputHash.method, + std::move(got), + rewriteRefs()), Hash::dummy, }; if (*scratchPath != newInfo0.path) { diff --git a/src/libstore/build/substitution-goal.cc b/src/libstore/build/substitution-goal.cc index 994cb4ac2..87fed495c 100644 --- a/src/libstore/build/substitution-goal.cc +++ b/src/libstore/build/substitution-goal.cc @@ -95,10 +95,9 @@ void PathSubstitutionGoal::tryNext() subs.pop_front(); if (ca) { - subPath = sub->makeFixedOutputPathFromCA({ - .name = std::string { storePath.name() }, - .info = caWithoutRefs(*ca), - }); + subPath = sub->makeFixedOutputPathFromCA( + std::string { storePath.name() }, + caWithoutRefs(*ca)); if (sub->storeDir == worker.store.storeDir) assert(subPath == storePath); } else if (sub->storeDir != worker.store.storeDir) { diff --git a/src/libstore/content-address.hh b/src/libstore/content-address.hh index a251aeb01..729b1078d 100644 --- a/src/libstore/content-address.hh +++ b/src/libstore/content-address.hh @@ -147,11 +147,4 @@ Hash getContentAddressHash(const ContentAddressWithReferences & ca); std::string printMethodAlgo(const ContentAddressWithReferences &); -struct StorePathDescriptor { - std::string name; - ContentAddressWithReferences info; - - GENERATE_CMP(StorePathDescriptor, me->name, me->info); -}; - } diff --git a/src/libstore/derivations.cc b/src/libstore/derivations.cc index 2a1fbe0f6..b19ee5ed1 100644 --- a/src/libstore/derivations.cc +++ b/src/libstore/derivations.cc @@ -35,10 +35,9 @@ std::optional<StorePath> DerivationOutput::path(const Store & store, std::string StorePath DerivationOutput::CAFixed::path(const Store & store, std::string_view drvName, std::string_view outputName) const { - return store.makeFixedOutputPathFromCA(StorePathDescriptor { - .name = outputPathName(drvName, outputName), - .info = ca, - }); + return store.makeFixedOutputPathFromCA( + outputPathName(drvName, outputName), + ca); } diff --git a/src/libstore/derivations.hh b/src/libstore/derivations.hh index c57a6f808..803ab3cd6 100644 --- a/src/libstore/derivations.hh +++ b/src/libstore/derivations.hh @@ -13,6 +13,7 @@ namespace nix { +class Store; /* Abstract syntax of derivations. */ diff --git a/src/libstore/local-store.cc b/src/libstore/local-store.cc index 654aee973..94c005130 100644 --- a/src/libstore/local-store.cc +++ b/src/libstore/local-store.cc @@ -1136,10 +1136,9 @@ void LocalStore::querySubstitutablePathInfos(const StorePathCAMap & paths, Subst // Recompute store path so that we can use a different store root. if (path.second) { - subPath = makeFixedOutputPathFromCA({ - .name = std::string { path.first.name() }, - .info = caWithoutRefs(*path.second), - }); + subPath = makeFixedOutputPathFromCA( + path.first.name(), + caWithoutRefs(*path.second)); if (sub->storeDir == storeDir) assert(subPath == path.first); if (subPath != path.first) @@ -1417,21 +1416,18 @@ StorePath LocalStore::addToStoreFromDump(Source & source0, std::string_view name auto [hash, size] = hashSink->finish(); - auto desc = StorePathDescriptor { - std::string { name }, - FixedOutputInfo { - { - .method = method, - .hash = hash, - }, - .references = { - .others = references, - .self = false, - }, + ContentAddressWithReferences desc = FixedOutputInfo { + { + .method = method, + .hash = hash, + }, + .references = { + .others = references, + .self = false, }, }; - auto dstPath = makeFixedOutputPathFromCA(desc); + auto dstPath = makeFixedOutputPathFromCA(name, desc); addTempRoot(dstPath); @@ -1475,7 +1471,12 @@ StorePath LocalStore::addToStoreFromDump(Source & source0, std::string_view name optimisePath(realPath, repair); - ValidPathInfo info { *this, std::move(desc), narHash.first }; + ValidPathInfo info { + *this, + name, + std::move(desc), + narHash.first + }; info.narSize = narHash.second; registerValidPath(info); } diff --git a/src/libstore/make-content-addressed.cc b/src/libstore/make-content-addressed.cc index 09f615439..3ee64c77a 100644 --- a/src/libstore/make-content-addressed.cc +++ b/src/libstore/make-content-addressed.cc @@ -49,15 +49,13 @@ std::map<StorePath, StorePath> makeContentAddressed( ValidPathInfo info { dstStore, - StorePathDescriptor { - .name = std::string { path.name() }, - .info = FixedOutputInfo { - { - .method = FileIngestionMethod::Recursive, - .hash = narModuloHash, - }, - .references = std::move(refs), + path.name(), + FixedOutputInfo { + { + .method = FileIngestionMethod::Recursive, + .hash = narModuloHash, }, + .references = std::move(refs), }, Hash::dummy, }; diff --git a/src/libstore/misc.cc b/src/libstore/misc.cc index dace653d6..acd4e0929 100644 --- a/src/libstore/misc.cc +++ b/src/libstore/misc.cc @@ -331,7 +331,7 @@ OutputPathMap resolveDerivedPath(Store & store, const DerivedPath::Built & bfd, [&](const OutputsSpec::Names & names) { return static_cast<std::set<std::string>>(names); }, - }, bfd.outputs); + }, bfd.outputs.raw()); for (auto & output : outputNames) { auto outputHash = get(outputHashes, output); if (!outputHash) diff --git a/src/libstore/nar-info.hh b/src/libstore/nar-info.hh index f1e3aabbd..a4dccb397 100644 --- a/src/libstore/nar-info.hh +++ b/src/libstore/nar-info.hh @@ -16,8 +16,8 @@ struct NarInfo : ValidPathInfo uint64_t fileSize = 0; NarInfo() = delete; - NarInfo(const Store & store, StorePathDescriptor && ca, Hash narHash) - : ValidPathInfo(store, std::move(ca), narHash) + NarInfo(const Store & store, std::string && name, ContentAddressWithReferences && ca, Hash narHash) + : ValidPathInfo(store, std::move(name), std::move(ca), narHash) { } NarInfo(StorePath && path, Hash narHash) : ValidPathInfo(std::move(path), narHash) { } NarInfo(const ValidPathInfo & info) : ValidPathInfo(info) { } diff --git a/src/libstore/outputs-spec.cc b/src/libstore/outputs-spec.cc index d0f39a854..e26c38138 100644 --- a/src/libstore/outputs-spec.cc +++ b/src/libstore/outputs-spec.cc @@ -1,8 +1,10 @@ +#include <regex> +#include <nlohmann/json.hpp> + #include "util.hh" +#include "regex-combinators.hh" #include "outputs-spec.hh" -#include "nlohmann/json.hpp" - -#include <regex> +#include "path-regex.hh" namespace nix { @@ -18,10 +20,14 @@ bool OutputsSpec::contains(const std::string & outputName) const }, raw()); } +static std::string outputSpecRegexStr = + regex::either( + regex::group(R"(\*)"), + regex::group(regex::list(nameRegexStr))); std::optional<OutputsSpec> OutputsSpec::parseOpt(std::string_view s) { - static std::regex regex(R"((\*)|([a-z]+(,[a-z]+)*))"); + static std::regex regex(std::string { outputSpecRegexStr }); std::smatch match; std::string s2 { s }; // until some improves std::regex @@ -42,7 +48,7 @@ OutputsSpec OutputsSpec::parse(std::string_view s) { std::optional spec = parseOpt(s); if (!spec) - throw Error("Invalid outputs specifier: '%s'", s); + throw Error("invalid outputs specifier '%s'", s); return *spec; } @@ -65,7 +71,7 @@ std::pair<std::string_view, ExtendedOutputsSpec> ExtendedOutputsSpec::parse(std: { std::optional spec = parseOpt(s); if (!spec) - throw Error("Invalid extended outputs specifier: '%s'", s); + throw Error("invalid extended outputs specifier '%s'", s); return *spec; } @@ -163,7 +169,7 @@ void adl_serializer<OutputsSpec>::to_json(json & json, OutputsSpec t) { [&](const OutputsSpec::Names & names) { json = names; }, - }, t); + }, t.raw()); } @@ -183,7 +189,7 @@ void adl_serializer<ExtendedOutputsSpec>::to_json(json & json, ExtendedOutputsSp [&](const ExtendedOutputsSpec::Explicit & e) { adl_serializer<OutputsSpec>::to_json(json, e); }, - }, t); + }, t.raw()); } } diff --git a/src/libstore/path-info.cc b/src/libstore/path-info.cc index 93f91e702..5944afd06 100644 --- a/src/libstore/path-info.cc +++ b/src/libstore/path-info.cc @@ -21,48 +21,45 @@ void ValidPathInfo::sign(const Store & store, const SecretKey & secretKey) sigs.insert(secretKey.signDetached(fingerprint(store))); } -std::optional<StorePathDescriptor> ValidPathInfo::fullStorePathDescriptorOpt() const +std::optional<ContentAddressWithReferences> ValidPathInfo::contentAddressWithReferenences() const { if (! ca) return std::nullopt; - return StorePathDescriptor { - .name = std::string { path.name() }, - .info = std::visit(overloaded { - [&](const TextHash & th) -> ContentAddressWithReferences { - assert(references.count(path) == 0); - return TextInfo { - th, - .references = references, - }; - }, - [&](const FixedOutputHash & foh) -> ContentAddressWithReferences { - auto refs = references; - bool hasSelfReference = false; - if (refs.count(path)) { - hasSelfReference = true; - refs.erase(path); - } - return FixedOutputInfo { - foh, - .references = { - .others = std::move(refs), - .self = hasSelfReference, - }, - }; - }, - }, *ca), - }; + return std::visit(overloaded { + [&](const TextHash & th) -> ContentAddressWithReferences { + assert(references.count(path) == 0); + return TextInfo { + th, + .references = references, + }; + }, + [&](const FixedOutputHash & foh) -> ContentAddressWithReferences { + auto refs = references; + bool hasSelfReference = false; + if (refs.count(path)) { + hasSelfReference = true; + refs.erase(path); + } + return FixedOutputInfo { + foh, + .references = { + .others = std::move(refs), + .self = hasSelfReference, + }, + }; + }, + }, *ca); } bool ValidPathInfo::isContentAddressed(const Store & store) const { - auto fullCaOpt = fullStorePathDescriptorOpt(); + auto fullCaOpt = contentAddressWithReferenences(); if (! fullCaOpt) return false; - auto caPath = store.makeFixedOutputPathFromCA(*fullCaOpt); + auto caPath = store.makeFixedOutputPathFromCA(path.name(), *fullCaOpt); bool res = caPath == path; @@ -102,9 +99,10 @@ Strings ValidPathInfo::shortRefs() const ValidPathInfo::ValidPathInfo( const Store & store, - StorePathDescriptor && info, + std::string_view name, + ContentAddressWithReferences && ca, Hash narHash) - : path(store.makeFixedOutputPathFromCA(info)) + : path(store.makeFixedOutputPathFromCA(name, ca)) , narHash(narHash) { std::visit(overloaded { @@ -118,7 +116,7 @@ ValidPathInfo::ValidPathInfo( this->references.insert(path); this->ca = std::move((FixedOutputHash &&) foi); }, - }, std::move(info.info)); + }, std::move(ca)); } diff --git a/src/libstore/path-info.hh b/src/libstore/path-info.hh index 476df79c2..663d94540 100644 --- a/src/libstore/path-info.hh +++ b/src/libstore/path-info.hh @@ -77,7 +77,7 @@ struct ValidPathInfo void sign(const Store & store, const SecretKey & secretKey); - std::optional<StorePathDescriptor> fullStorePathDescriptorOpt() const; + std::optional<ContentAddressWithReferences> contentAddressWithReferenences() const; /* Return true iff the path is verifiably content-addressed. */ bool isContentAddressed(const Store & store) const; @@ -100,7 +100,7 @@ struct ValidPathInfo ValidPathInfo(const StorePath & path, Hash narHash) : path(path), narHash(narHash) { }; ValidPathInfo(const Store & store, - StorePathDescriptor && ca, Hash narHash); + std::string_view name, ContentAddressWithReferences && ca, Hash narHash); virtual ~ValidPathInfo() { } diff --git a/src/libstore/path-regex.hh b/src/libstore/path-regex.hh new file mode 100644 index 000000000..6893c3876 --- /dev/null +++ b/src/libstore/path-regex.hh @@ -0,0 +1,7 @@ +#pragma once + +namespace nix { + +static constexpr std::string_view nameRegexStr = R"([0-9a-zA-Z\+\-\._\?=]+)"; + +} diff --git a/src/libstore/path.cc b/src/libstore/path.cc index 392db225e..46be54281 100644 --- a/src/libstore/path.cc +++ b/src/libstore/path.cc @@ -8,8 +8,10 @@ static void checkName(std::string_view path, std::string_view name) { if (name.empty()) throw BadStorePath("store path '%s' has an empty name", path); - if (name.size() > 211) - throw BadStorePath("store path '%s' has a name longer than 211 characters", path); + if (name.size() > StorePath::MaxPathLen) + throw BadStorePath("store path '%s' has a name longer than '%d characters", + StorePath::MaxPathLen, path); + // See nameRegexStr for the definition for (auto c : name) if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') diff --git a/src/libstore/path.hh b/src/libstore/path.hh index 7f13c11e9..1e5579b90 100644 --- a/src/libstore/path.hh +++ b/src/libstore/path.hh @@ -6,7 +6,6 @@ namespace nix { -class Store; struct Hash; class StorePath @@ -18,6 +17,8 @@ public: /* Size of the hash part of store paths, in base-32 characters. */ constexpr static size_t HashLen = 32; // i.e. 160 bits + constexpr static size_t MaxPathLen = 211; + StorePath() = delete; StorePath(std::string_view baseName); diff --git a/src/libstore/realisation.hh b/src/libstore/realisation.hh index 9429c7004..48d0283de 100644 --- a/src/libstore/realisation.hh +++ b/src/libstore/realisation.hh @@ -10,6 +10,8 @@ namespace nix { +class Store; + struct DrvOutput { // The hash modulo of the derivation Hash drvHash; diff --git a/src/libstore/store-api.cc b/src/libstore/store-api.cc index c39e50d14..3c0c26706 100644 --- a/src/libstore/store-api.cc +++ b/src/libstore/store-api.cc @@ -209,17 +209,17 @@ StorePath Store::makeTextPath(std::string_view name, const TextInfo & info) cons } -StorePath Store::makeFixedOutputPathFromCA(const StorePathDescriptor & desc) const +StorePath Store::makeFixedOutputPathFromCA(std::string_view name, const ContentAddressWithReferences & ca) const { // New template return std::visit(overloaded { [&](const TextInfo & ti) { - return makeTextPath(desc.name, ti); + return makeTextPath(name, ti); }, [&](const FixedOutputInfo & foi) { - return makeFixedOutputPath(desc.name, foi); + return makeFixedOutputPath(name, foi); } - }, desc.info); + }, ca); } @@ -437,15 +437,13 @@ ValidPathInfo Store::addToStoreSlow(std::string_view name, const Path & srcPath, ValidPathInfo info { *this, - StorePathDescriptor { - std::string { name }, - FixedOutputInfo { - { - .method = method, - .hash = hash, - }, - .references = {}, + name, + FixedOutputInfo { + { + .method = method, + .hash = hash, }, + .references = {}, }, narHash, }; @@ -997,7 +995,8 @@ void copyStorePath( if (info->ca && info->references.empty()) { auto info2 = make_ref<ValidPathInfo>(*info); info2->path = dstStore.makeFixedOutputPathFromCA( - info->fullStorePathDescriptorOpt().value()); + info->path.name(), + info->contentAddressWithReferenences().value()); if (dstStore.storeDir == srcStore.storeDir) assert(info->path == info2->path); info = info2; @@ -1110,7 +1109,8 @@ std::map<StorePath, StorePath> copyPaths( auto storePathForDst = storePathForSrc; if (currentPathInfo.ca && currentPathInfo.references.empty()) { storePathForDst = dstStore.makeFixedOutputPathFromCA( - currentPathInfo.fullStorePathDescriptorOpt().value()); + currentPathInfo.path.name(), + currentPathInfo.contentAddressWithReferenences().value()); if (dstStore.storeDir == srcStore.storeDir) assert(storePathForDst == storePathForSrc); if (storePathForDst != storePathForSrc) diff --git a/src/libstore/store-api.hh b/src/libstore/store-api.hh index d77aea338..2d252db84 100644 --- a/src/libstore/store-api.hh +++ b/src/libstore/store-api.hh @@ -216,7 +216,7 @@ public: StorePath makeTextPath(std::string_view name, const TextInfo & info) const; - StorePath makeFixedOutputPathFromCA(const StorePathDescriptor & info) const; + StorePath makeFixedOutputPathFromCA(std::string_view name, const ContentAddressWithReferences & ca) const; /* This is the preparatory part of addToStore(); it computes the store path to which srcPath is to be copied. Returns the store diff --git a/src/libstore/tests/libstoretests.hh b/src/libstore/tests/libstoretests.hh new file mode 100644 index 000000000..05397659b --- /dev/null +++ b/src/libstore/tests/libstoretests.hh @@ -0,0 +1,23 @@ +#include <gtest/gtest.h> +#include <gmock/gmock.h> + +#include "store-api.hh" + +namespace nix { + +class LibStoreTest : public ::testing::Test { + public: + static void SetUpTestSuite() { + initLibStore(); + } + + protected: + LibStoreTest() + : store(openStore("dummy://")) + { } + + ref<Store> store; +}; + + +} /* namespace nix */ diff --git a/src/libstore/tests/local.mk b/src/libstore/tests/local.mk index f74295d97..a2cf8a0cf 100644 --- a/src/libstore/tests/local.mk +++ b/src/libstore/tests/local.mk @@ -12,4 +12,4 @@ libstore-tests_CXXFLAGS += -I src/libstore -I src/libutil libstore-tests_LIBS = libstore libutil -libstore-tests_LDFLAGS := $(GTEST_LIBS) +libstore-tests_LDFLAGS := -lrapidcheck $(GTEST_LIBS) diff --git a/src/libstore/tests/outputs-spec.cc b/src/libstore/tests/outputs-spec.cc index c9c2cafd0..06e4cabbd 100644 --- a/src/libstore/tests/outputs-spec.cc +++ b/src/libstore/tests/outputs-spec.cc @@ -40,6 +40,20 @@ TEST(OutputsSpec, names_out) { ASSERT_EQ(expected.to_string(), str); } +TEST(OutputsSpec, names_underscore) { + std::string_view str = "a_b"; + OutputsSpec expected = OutputsSpec::Names { "a_b" }; + ASSERT_EQ(OutputsSpec::parse(str), expected); + ASSERT_EQ(expected.to_string(), str); +} + +TEST(OutputsSpec, names_numberic) { + std::string_view str = "01"; + OutputsSpec expected = OutputsSpec::Names { "01" }; + ASSERT_EQ(OutputsSpec::parse(str), expected); + ASSERT_EQ(expected.to_string(), str); +} + TEST(OutputsSpec, names_out_bin) { OutputsSpec expected = OutputsSpec::Names { "out", "bin" }; ASSERT_EQ(OutputsSpec::parse("out,bin"), expected); diff --git a/src/libstore/tests/path.cc b/src/libstore/tests/path.cc new file mode 100644 index 000000000..8ea252c92 --- /dev/null +++ b/src/libstore/tests/path.cc @@ -0,0 +1,144 @@ +#include <regex> + +#include <nlohmann/json.hpp> +#include <gtest/gtest.h> +#include <rapidcheck/gtest.h> + +#include "path-regex.hh" +#include "store-api.hh" + +#include "libstoretests.hh" + +namespace nix { + +#define STORE_DIR "/nix/store/" +#define HASH_PART "g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q" + +class StorePathTest : public LibStoreTest +{ +}; + +static std::regex nameRegex { std::string { nameRegexStr } }; + +#define TEST_DONT_PARSE(NAME, STR) \ + TEST_F(StorePathTest, bad_ ## NAME) { \ + std::string_view str = \ + STORE_DIR HASH_PART "-" STR; \ + ASSERT_THROW( \ + store->parseStorePath(str), \ + BadStorePath); \ + std::string name { STR }; \ + EXPECT_FALSE(std::regex_match(name, nameRegex)); \ + } + +TEST_DONT_PARSE(empty, "") +TEST_DONT_PARSE(garbage, "&*()") +TEST_DONT_PARSE(double_star, "**") +TEST_DONT_PARSE(star_first, "*,foo") +TEST_DONT_PARSE(star_second, "foo,*") +TEST_DONT_PARSE(bang, "foo!o") + +#undef TEST_DONT_PARSE + +#define TEST_DO_PARSE(NAME, STR) \ + TEST_F(StorePathTest, good_ ## NAME) { \ + std::string_view str = \ + STORE_DIR HASH_PART "-" STR; \ + auto p = store->parseStorePath(str); \ + std::string name { p.name() }; \ + EXPECT_TRUE(std::regex_match(name, nameRegex)); \ + } + +// 0-9 a-z A-Z + - . _ ? = + +TEST_DO_PARSE(numbers, "02345") +TEST_DO_PARSE(lower_case, "foo") +TEST_DO_PARSE(upper_case, "FOO") +TEST_DO_PARSE(plus, "foo+bar") +TEST_DO_PARSE(dash, "foo-dev") +TEST_DO_PARSE(underscore, "foo_bar") +TEST_DO_PARSE(period, "foo.txt") +TEST_DO_PARSE(question_mark, "foo?why") +TEST_DO_PARSE(equals_sign, "foo=foo") + +#undef TEST_DO_PARSE + +// For rapidcheck +void showValue(const StorePath & p, std::ostream & os) { + os << p.to_string(); +} + +} + +namespace rc { +using namespace nix; + +template<> +struct Arbitrary<StorePath> { + static Gen<StorePath> arbitrary(); +}; + +Gen<StorePath> Arbitrary<StorePath>::arbitrary() +{ + auto len = *gen::inRange<size_t>(1, StorePath::MaxPathLen); + + std::string pre { HASH_PART "-" }; + pre.reserve(pre.size() + len); + + for (size_t c = 0; c < len; ++c) { + switch (auto i = *gen::inRange<uint8_t>(0, 10 + 2 * 26 + 6)) { + case 0 ... 9: + pre += '0' + i; + case 10 ... 35: + pre += 'A' + (i - 10); + break; + case 36 ... 61: + pre += 'a' + (i - 36); + break; + case 62: + pre += '+'; + break; + case 63: + pre += '-'; + break; + case 64: + pre += '.'; + break; + case 65: + pre += '_'; + break; + case 66: + pre += '?'; + break; + case 67: + pre += '='; + break; + default: + assert(false); + } + } + + return gen::just(StorePath { pre }); +} + +} // namespace rc + +namespace nix { + +RC_GTEST_FIXTURE_PROP( + StorePathTest, + prop_regex_accept, + (const StorePath & p)) +{ + RC_ASSERT(std::regex_match(std::string { p.name() }, nameRegex)); +} + +RC_GTEST_FIXTURE_PROP( + StorePathTest, + prop_round_rip, + (const StorePath & p)) +{ + RC_ASSERT(p == store->parseStorePath(store->printStorePath(p))); +} + +} diff --git a/src/libutil/monitor-fd.hh b/src/libutil/monitor-fd.hh index 5ee0b88ef..9518cf8aa 100644 --- a/src/libutil/monitor-fd.hh +++ b/src/libutil/monitor-fd.hh @@ -22,27 +22,38 @@ public: { thread = std::thread([fd]() { while (true) { - /* Wait indefinitely until a POLLHUP occurs. */ - struct pollfd fds[1]; - fds[0].fd = fd; - /* This shouldn't be necessary, but macOS doesn't seem to - like a zeroed out events field. - See rdar://37537852. - */ - fds[0].events = POLLHUP; - auto count = poll(fds, 1, -1); - if (count == -1) abort(); // can't happen - /* This shouldn't happen, but can on macOS due to a bug. - See rdar://37550628. - - This may eventually need a delay or further - coordination with the main thread if spinning proves - too harmful. - */ - if (count == 0) continue; - assert(fds[0].revents & POLLHUP); - triggerInterrupt(); - break; + /* Wait indefinitely until a POLLHUP occurs. */ + struct pollfd fds[1]; + fds[0].fd = fd; + /* Polling for no specific events (i.e. just waiting + for an error/hangup) doesn't work on macOS + anymore. So wait for read events and ignore + them. */ + fds[0].events = + #ifdef __APPLE__ + POLLRDNORM + #else + 0 + #endif + ; + auto count = poll(fds, 1, -1); + if (count == -1) abort(); // can't happen + /* This shouldn't happen, but can on macOS due to a bug. + See rdar://37550628. + + This may eventually need a delay or further + coordination with the main thread if spinning proves + too harmful. + */ + if (count == 0) continue; + if (fds[0].revents & POLLHUP) { + triggerInterrupt(); + break; + } + /* This will only happen on macOS. We sleep a bit to + avoid waking up too often if the client is sending + input. */ + sleep(1); } }); }; diff --git a/src/libutil/regex-combinators.hh b/src/libutil/regex-combinators.hh new file mode 100644 index 000000000..0b997b25a --- /dev/null +++ b/src/libutil/regex-combinators.hh @@ -0,0 +1,30 @@ +#pragma once + +#include <string_view> + +namespace nix::regex { + +// TODO use constexpr string building like +// https://github.com/akrzemi1/static_string/blob/master/include/ak_toolkit/static_string.hpp + +static inline std::string either(std::string_view a, std::string_view b) +{ + return std::string { a } + "|" + b; +} + +static inline std::string group(std::string_view a) +{ + return std::string { "(" } + a + ")"; +} + +static inline std::string many(std::string_view a) +{ + return std::string { "(?:" } + a + ")*"; +} + +static inline std::string list(std::string_view a) +{ + return std::string { a } + many(group("," + a)); +} + +} diff --git a/src/nix/add-to-store.cc b/src/nix/add-to-store.cc index 0b58818c3..81dbc09a6 100644 --- a/src/nix/add-to-store.cc +++ b/src/nix/add-to-store.cc @@ -43,15 +43,13 @@ struct CmdAddToStore : MixDryRun, StoreCommand ValidPathInfo info { *store, - StorePathDescriptor { - .name = *namePart, - .info = FixedOutputInfo { - { - .method = std::move(ingestionMethod), - .hash = std::move(hash), - }, - .references = {}, + std::move(*namePart), + FixedOutputInfo { + { + .method = std::move(ingestionMethod), + .hash = std::move(hash), }, + .references = {}, }, narHash, }; diff --git a/src/nix/flake.cc b/src/nix/flake.cc index d16d88ef8..020c1b182 100644 --- a/src/nix/flake.cc +++ b/src/nix/flake.cc @@ -655,6 +655,19 @@ struct CmdFlakeCheck : FlakeCommand } } + else if ( + name == "lib" + || name == "darwinConfigurations" + || name == "darwinModules" + || name == "flakeModule" + || name == "flakeModules" + || name == "herculesCI" + || name == "homeConfigurations" + || name == "nixopsConfigurations" + ) + // Known but unchecked community attribute + ; + else warn("unknown flake output '%s'", name); diff --git a/src/nix/profile.cc b/src/nix/profile.cc index aac8e5c81..345505532 100644 --- a/src/nix/profile.cc +++ b/src/nix/profile.cc @@ -200,17 +200,15 @@ struct ProfileManifest ValidPathInfo info { *store, - StorePathDescriptor { - "profile", - FixedOutputInfo { - { - .method = FileIngestionMethod::Recursive, - .hash = narHash, - }, - .references = { - .others = std::move(references), - .self = false, - }, + "profile", + FixedOutputInfo { + { + .method = FileIngestionMethod::Recursive, + .hash = narHash, + }, + .references = { + .others = std::move(references), + .self = false, }, }, narHash, diff --git a/tests/build.sh b/tests/build.sh index 036fb037e..a00fb5232 100644 --- a/tests/build.sh +++ b/tests/build.sh @@ -42,20 +42,21 @@ nix build -f multiple-outputs.nix --json 'a^*' --no-link | jq --exit-status ' nix build -f multiple-outputs.nix --json e --no-link | jq --exit-status ' (.[0] | (.drvPath | match(".*multiple-outputs-e.drv")) and - (.outputs | keys == ["a", "b"])) + (.outputs | keys == ["a_a", "b"])) ' # But not when it's overriden. -nix build -f multiple-outputs.nix --json e^a --no-link | jq --exit-status ' +nix build -f multiple-outputs.nix --json e^a_a --no-link +nix build -f multiple-outputs.nix --json e^a_a --no-link | jq --exit-status ' (.[0] | (.drvPath | match(".*multiple-outputs-e.drv")) and - (.outputs | keys == ["a"])) + (.outputs | keys == ["a_a"])) ' nix build -f multiple-outputs.nix --json 'e^*' --no-link | jq --exit-status ' (.[0] | (.drvPath | match(".*multiple-outputs-e.drv")) and - (.outputs | keys == ["a", "b", "c"])) + (.outputs | keys == ["a_a", "b", "c"])) ' # Test building from raw store path to drv not expression. @@ -88,7 +89,7 @@ nix build "$drv^first,second" --no-link --json | jq --exit-status ' (.outputs | (keys | length == 2) and (.first | match(".*multiple-outputs-a-first")) and - (.second | match(".*multiple-outputs-a-second")))) + (.second | match(".*multiple-outputs-a-second")))) ' nix build "$drv^*" --no-link --json | jq --exit-status ' @@ -97,14 +98,14 @@ nix build "$drv^*" --no-link --json | jq --exit-status ' (.outputs | (keys | length == 2) and (.first | match(".*multiple-outputs-a-first")) and - (.second | match(".*multiple-outputs-a-second")))) + (.second | match(".*multiple-outputs-a-second")))) ' # Make sure that `--impure` works (regression test for https://github.com/NixOS/nix/issues/6488) nix build --impure -f multiple-outputs.nix --json e --no-link | jq --exit-status ' (.[0] | (.drvPath | match(".*multiple-outputs-e.drv")) and - (.outputs | keys == ["a", "b"])) + (.outputs | keys == ["a_a", "b"])) ' testNormalization () { diff --git a/tests/lang/eval-okay-context-introspection.exp b/tests/lang/eval-okay-context-introspection.exp index 27ba77dda..03b400cc8 100644 --- a/tests/lang/eval-okay-context-introspection.exp +++ b/tests/lang/eval-okay-context-introspection.exp @@ -1 +1 @@ -true +[ true true true true true true ] diff --git a/tests/lang/eval-okay-context-introspection.nix b/tests/lang/eval-okay-context-introspection.nix index 43178bd2e..50a78d946 100644 --- a/tests/lang/eval-okay-context-introspection.nix +++ b/tests/lang/eval-okay-context-introspection.nix @@ -18,7 +18,24 @@ let }; }; - legit-context = builtins.getContext "${path}${drv.outPath}${drv.foo.outPath}${drv.drvPath}"; + combo-path = "${path}${drv.outPath}${drv.foo.outPath}${drv.drvPath}"; + legit-context = builtins.getContext combo-path; - constructed-context = builtins.getContext (builtins.appendContext "" desired-context); -in legit-context == constructed-context + reconstructed-path = builtins.appendContext + (builtins.unsafeDiscardStringContext combo-path) + desired-context; + + # Eta rule for strings with context. + etaRule = str: + str == builtins.appendContext + (builtins.unsafeDiscardStringContext str) + (builtins.getContext str); + +in [ + (legit-context == desired-context) + (reconstructed-path == combo-path) + (etaRule "foo") + (etaRule drv.drvPath) + (etaRule drv.foo.outPath) + (etaRule (builtins.unsafeDiscardOutputDependency drv.drvPath)) +] diff --git a/tests/lang/eval-okay-readDir.exp b/tests/lang/eval-okay-readDir.exp index bf8d2c14e..6413f6d4f 100644 --- a/tests/lang/eval-okay-readDir.exp +++ b/tests/lang/eval-okay-readDir.exp @@ -1 +1 @@ -{ bar = "regular"; foo = "directory"; } +{ bar = "regular"; foo = "directory"; ldir = "symlink"; linked = "symlink"; } diff --git a/tests/lang/eval-okay-readFileType.exp b/tests/lang/eval-okay-readFileType.exp new file mode 100644 index 000000000..6413f6d4f --- /dev/null +++ b/tests/lang/eval-okay-readFileType.exp @@ -0,0 +1 @@ +{ bar = "regular"; foo = "directory"; ldir = "symlink"; linked = "symlink"; } diff --git a/tests/lang/eval-okay-readFileType.nix b/tests/lang/eval-okay-readFileType.nix new file mode 100644 index 000000000..174fb6c3a --- /dev/null +++ b/tests/lang/eval-okay-readFileType.nix @@ -0,0 +1,6 @@ +{ + bar = builtins.readFileType ./readDir/bar; + foo = builtins.readFileType ./readDir/foo; + linked = builtins.readFileType ./readDir/linked; + ldir = builtins.readFileType ./readDir/ldir; +} diff --git a/tests/lang/readDir/ldir b/tests/lang/readDir/ldir new file mode 120000 index 000000000..191028156 --- /dev/null +++ b/tests/lang/readDir/ldir @@ -0,0 +1 @@ +foo
\ No newline at end of file diff --git a/tests/lang/readDir/linked b/tests/lang/readDir/linked new file mode 120000 index 000000000..c503f86a0 --- /dev/null +++ b/tests/lang/readDir/linked @@ -0,0 +1 @@ +foo/git-hates-directories
\ No newline at end of file diff --git a/tests/multiple-outputs.nix b/tests/multiple-outputs.nix index 1429bc648..413d392e4 100644 --- a/tests/multiple-outputs.nix +++ b/tests/multiple-outputs.nix @@ -91,9 +91,9 @@ rec { e = mkDerivation { name = "multiple-outputs-e"; - outputs = [ "a" "b" "c" ]; - meta.outputsToInstall = [ "a" "b" ]; - buildCommand = "mkdir $a $b $c"; + outputs = [ "a_a" "b" "c" ]; + meta.outputsToInstall = [ "a_a" "b" ]; + buildCommand = "mkdir $a_a $b $c"; }; independent = mkDerivation { @@ -117,4 +117,14 @@ rec { ''; }; + invalid-output-name-1 = mkDerivation { + name = "invalid-output-name-1"; + outputs = [ "out/"]; + }; + + invalid-output-name-2 = mkDerivation { + name = "invalid-output-name-2"; + outputs = [ "x" "foo$"]; + }; + } diff --git a/tests/multiple-outputs.sh b/tests/multiple-outputs.sh index 0d45ad35b..66be6fa64 100644 --- a/tests/multiple-outputs.sh +++ b/tests/multiple-outputs.sh @@ -83,3 +83,6 @@ nix-store --gc --keep-derivations --keep-outputs nix-store --gc --print-roots rm -rf $NIX_STORE_DIR/.links rmdir $NIX_STORE_DIR + +nix build -f multiple-outputs.nix invalid-output-name-1 2>&1 | grep 'contains illegal character' +nix build -f multiple-outputs.nix invalid-output-name-2 2>&1 | grep 'contains illegal character' |