diff options
| -rw-r--r-- | doc/manual/src/SUMMARY.md | 1 | ||||
| -rw-r--r-- | doc/manual/src/expressions/builtin-constants.md | 20 | ||||
| -rw-r--r-- | doc/manual/src/expressions/builtins.md | 878 | ||||
| -rw-r--r-- | src/libexpr/primops.cc | 1178 | ||||
| -rw-r--r-- | src/libexpr/primops/fetchTree.cc | 175 |
5 files changed, 1274 insertions, 978 deletions
diff --git a/doc/manual/src/SUMMARY.md b/doc/manual/src/SUMMARY.md index 4089caf8a..8281f683f 100644 --- a/doc/manual/src/SUMMARY.md +++ b/doc/manual/src/SUMMARY.md @@ -38,6 +38,7 @@ - [Operators](expressions/language-operators.md) - [Derivations](expressions/derivations.md) - [Advanced Attributes](expressions/advanced-attributes.md) + - [Built-in Constants](expressions/builtin-constants.md) - [Built-in Functions](expressions/builtins.md) - [Advanced Topics](advanced-topics/advanced-topics.md) - [Remote Builds](advanced-topics/distributed-builds.md) diff --git a/doc/manual/src/expressions/builtin-constants.md b/doc/manual/src/expressions/builtin-constants.md new file mode 100644 index 000000000..3345a715b --- /dev/null +++ b/doc/manual/src/expressions/builtin-constants.md @@ -0,0 +1,20 @@ +# Built-in Constants + +Here are the constants built into the Nix expression evaluator: + + - `builtins` + The set `builtins` contains all the built-in functions and values. + You can use `builtins` to test for the availability of features in + the Nix installation, e.g., + + ```nix + if builtins ? getEnv then builtins.getEnv "PATH" else "" + ``` + + This allows a Nix expression to fall back gracefully on older Nix + installations that don’t have the desired built-in function. + + - `builtins.currentSystem` + The built-in value `currentSystem` evaluates to the Nix platform + identifier for the Nix installation on which the expression is being + evaluated, such as `"i686-linux"` or `"x86_64-darwin"`. diff --git a/doc/manual/src/expressions/builtins.md b/doc/manual/src/expressions/builtins.md index c258fb3b3..ae3bb150c 100644 --- a/doc/manual/src/expressions/builtins.md +++ b/doc/manual/src/expressions/builtins.md @@ -1,374 +1,20 @@ # Built-in Functions -This section lists the functions and constants built into the Nix -expression evaluator. (The built-in function `derivation` is discussed -above.) Some built-ins, such as `derivation`, are always in scope of -every Nix expression; you can just access them right away. But to -prevent polluting the namespace too much, most built-ins are not in +This section lists the functions built into the Nix expression +evaluator. (The built-in function `derivation` is discussed above.) +Some built-ins, such as `derivation`, are always in scope of every Nix +expression; you can just access them right away. But to prevent +polluting the namespace too much, most built-ins are not in scope. Instead, you can access them through the `builtins` built-in value, which is a set that contains all built-in functions and values. For instance, `derivation` is also available as `builtins.derivation`. - - `builtins.add` *e1* *e2* - Return the sum of the numbers *e1* and *e2*. - - - `builtins.all` *pred* *list* - Return `true` if the function *pred* returns `true` for all elements - of *list*, and `false` otherwise. - - - `builtins.any` *pred* *list* - Return `true` if the function *pred* returns `true` for at least one - element of *list*, and `false` otherwise. - - - `builtins.attrNames` *set* - Return the names of the attributes in the set *set* in an - alphabetically sorted list. For instance, `builtins.attrNames { y - = 1; x = "foo"; }` evaluates to `[ "x" "y" ]`. - - - `builtins.attrValues` *set* - Return the values of the attributes in the set *set* in the order - corresponding to the sorted attribute names. - - - `baseNameOf` *s* - Return the *base name* of the string *s*, that is, everything - following the final slash in the string. This is similar to the GNU - `basename` command. - - - `builtins.bitAnd` *e1* *e2* - Return the bitwise AND of the integers *e1* and *e2*. - - - `builtins.bitOr` *e1* *e2* - Return the bitwise OR of the integers *e1* and *e2*. - - - `builtins.bitXor` *e1* *e2* - Return the bitwise XOR of the integers *e1* and *e2*. - - - `builtins` - The set `builtins` contains all the built-in functions and values. - You can use `builtins` to test for the availability of features in - the Nix installation, e.g., - - ```nix - if builtins ? getEnv then builtins.getEnv "PATH" else "" - ``` - - This allows a Nix expression to fall back gracefully on older Nix - installations that don’t have the desired built-in function. - - - `builtins.compareVersions` *s1* *s2* - Compare two strings representing versions and return `-1` if - version *s1* is older than version *s2*, `0` if they are the same, - and `1` if *s1* is newer than *s2*. The version comparison - algorithm is the same as the one used by [`nix-env - -u`](../command-ref/nix-env.md#operation---upgrade). - - - `builtins.concatLists` *lists* - Concatenate a list of lists into a single list. - - - `builtins.concatStringsSep` *separator* *list* - Concatenate a list of strings with a separator between each - element, e.g. `concatStringsSep "/" ["usr" "local" "bin"] == - "usr/local/bin"` - - - `builtins.currentSystem` - The built-in value `currentSystem` evaluates to the Nix platform - identifier for the Nix installation on which the expression is being - evaluated, such as `"i686-linux"` or `"x86_64-darwin"`. - - - `builtins.deepSeq` *e1* *e2* - This is like `seq e1 e2`, except that *e1* is evaluated *deeply*: - if it’s a list or set, its elements or attributes are also - evaluated recursively. - - `derivation` *attrs*; `builtins.derivation` *attrs* - `derivation` is described in [its own section](derivations.md). - - - `dirOf` *s*; `builtins.dirOf` *s* - Return the directory part of the string *s*, that is, everything - before the final slash in the string. This is similar to the GNU - `dirname` command. - - - `builtins.div` *e1* *e2* - Return the quotient of the numbers *e1* and *e2*. - - - `builtins.elem` *x* *xs* - Return `true` if a value equal to *x* occurs in the list *xs*, and - `false` otherwise. - - - `builtins.elemAt` *xs* *n* - Return element *n* from the list *xs*. Elements are counted starting - from 0. A fatal error occurs if the index is out of bounds. - - - `builtins.fetchurl` *url* - Download the specified URL and return the path of the downloaded - file. This function is not available if [restricted evaluation - mode](../command-ref/conf-file.md) is enabled. - - - `fetchTarball` *url*; `builtins.fetchTarball` *url* - Download the specified URL, unpack it and return the path of the - unpacked tree. The file must be a tape archive (`.tar`) compressed - with `gzip`, `bzip2` or `xz`. The top-level path component of the - files in the tarball is removed, so it is best if the tarball - contains a single directory at top level. The typical use of the - function is to obtain external Nix expression dependencies, such as - a particular version of Nixpkgs, e.g. - - ```nix - with import (fetchTarball https://github.com/NixOS/nixpkgs/archive/nixos-14.12.tar.gz) {}; - - stdenv.mkDerivation { … } - ``` - - The fetched tarball is cached for a certain amount of time (1 hour - by default) in `~/.cache/nix/tarballs/`. You can change the cache - timeout either on the command line with `--option tarball-ttl number - of seconds` or in the Nix configuration file with this option: ` - number of seconds to cache `. - - Note that when obtaining the hash with ` nix-prefetch-url ` the - option `--unpack` is required. - - This function can also verify the contents against a hash. In that - case, the function takes a set instead of a URL. The set requires - the attribute `url` and the attribute `sha256`, e.g. - - ```nix - with import (fetchTarball { - url = "https://github.com/NixOS/nixpkgs/archive/nixos-14.12.tar.gz"; - sha256 = "1jppksrfvbk5ypiqdz4cddxdl8z6zyzdb2srq8fcffr327ld5jj2"; - }) {}; - - stdenv.mkDerivation { … } - ``` - - This function is not available if [restricted evaluation - mode](../command-ref/conf-file.md) is enabled. - - - `builtins.fetchGit` *args* - Fetch a path from git. *args* can be a URL, in which case the HEAD - of the repo at that URL is fetched. Otherwise, it can be an - attribute with the following attributes (all except `url` optional): - - - url - The URL of the repo. - - - name - The name of the directory the repo should be exported to in the - store. Defaults to the basename of the URL. - - - rev - The git revision to fetch. Defaults to the tip of `ref`. - - - ref - The git ref to look for the requested revision under. This is - often a branch or tag name. Defaults to `HEAD`. - - By default, the `ref` value is prefixed with `refs/heads/`. As - of Nix 2.3.0 Nix will not prefix `refs/heads/` if `ref` starts - with `refs/`. - - - submodules - A Boolean parameter that specifies whether submodules should be - checked out. Defaults to `false`. - - Here are some examples of how to use `fetchGit`. - - - To fetch a private repository over SSH: - - ```nix - builtins.fetchGit { - url = "git@github.com:my-secret/repository.git"; - ref = "master"; - rev = "adab8b916a45068c044658c4158d81878f9ed1c3"; - } - ``` - - - To fetch an arbitrary reference: - - ```nix - builtins.fetchGit { - url = "https://github.com/NixOS/nix.git"; - ref = "refs/heads/0.5-release"; - } - ``` - - - If the revision you're looking for is in the default branch of - the git repository you don't strictly need to specify the branch - name in the `ref` attribute. - - However, if the revision you're looking for is in a future - branch for the non-default branch you will need to specify the - the `ref` attribute as well. - - ```nix - builtins.fetchGit { - url = "https://github.com/nixos/nix.git"; - rev = "841fcbd04755c7a2865c51c1e2d3b045976b7452"; - ref = "1.11-maintenance"; - } - ``` - - > **Note** - > - > It is nice to always specify the branch which a revision - > belongs to. Without the branch being specified, the fetcher - > might fail if the default branch changes. Additionally, it can - > be confusing to try a commit from a non-default branch and see - > the fetch fail. If the branch is specified the fault is much - > more obvious. - - - If the revision you're looking for is in the default branch of - the git repository you may omit the `ref` attribute. - - ```nix - builtins.fetchGit { - url = "https://github.com/nixos/nix.git"; - rev = "841fcbd04755c7a2865c51c1e2d3b045976b7452"; - } - ``` - - - To fetch a specific tag: - - ```nix - builtins.fetchGit { - url = "https://github.com/nixos/nix.git"; - ref = "refs/tags/1.9"; - } - ``` - - - To fetch the latest version of a remote branch: - - ```nix - builtins.fetchGit { - url = "ssh://git@github.com/nixos/nix.git"; - ref = "master"; - } - ``` - - > **Note** - > - > Nix will refetch the branch in accordance with - > the option `tarball-ttl`. - - > **Note** - > - > This behavior is disabled in *Pure evaluation mode*. - - - `builtins.filter` *f* *xs* - Return a list consisting of the elements of *xs* for which the - function *f* returns `true`. - - - `builtins.filterSource` *e1* *e2* - This function allows you to copy sources into the Nix store while - filtering certain files. For instance, suppose that you want to use - the directory `source-dir` as an input to a Nix expression, e.g. - - ```nix - stdenv.mkDerivation { - ... - src = ./source-dir; - } - ``` - - However, if `source-dir` is a Subversion working copy, then all - those annoying `.svn` subdirectories will also be copied to the - store. Worse, the contents of those directories may change a lot, - causing lots of spurious rebuilds. With `filterSource` you can - filter out the `.svn` directories: - - ```nix - src = builtins.filterSource - (path: type: type != "directory" || baseNameOf path != ".svn") - ./source-dir; - ``` - - Thus, the first argument *e1* must be a predicate function that is - called for each regular file, directory or symlink in the source - tree *e2*. If the function returns `true`, the file is copied to the - Nix store, otherwise it is omitted. The function is called with two - arguments. The first is the full path of the file. The second is a - string that identifies the type of the file, which is either - `"regular"`, `"directory"`, `"symlink"` or `"unknown"` (for other - kinds of files such as device nodes or fifos — but note that those - cannot be copied to the Nix store, so if the predicate returns - `true` for them, the copy will fail). If you exclude a directory, - the entire corresponding subtree of *e2* will be excluded. - - - `builtins.foldl’` *op* *nul* *list* - Reduce a list by applying a binary operator, from left to right, - e.g. `foldl’ op nul [x0 x1 x2 ...] = op (op (op nul x0) x1) x2) - ...`. The operator is applied strictly, i.e., its arguments are - evaluated first. For example, `foldl’ (x: y: x + y) 0 [1 2 3]` - evaluates to 6. - - `builtins.functionArgs` *f* - Return a set containing the names of the formal arguments expected - by the function *f*. The value of each attribute is a Boolean - denoting whether the corresponding argument has a default value. For - instance, `functionArgs ({ x, y ? 123}: ...) = { x = false; y = - true; }`. - - "Formal argument" here refers to the attributes pattern-matched by - the function. Plain lambdas are not included, e.g. `functionArgs (x: - ...) = { }`. - - - `builtins.fromJSON` *e* - Convert a JSON string to a Nix value. For example, - - ```nix - builtins.fromJSON ''{"x": [1, 2, 3], "y": null}'' - ``` - - returns the value `{ x = [ 1 2 3 ]; y = null; }`. - - - `builtins.genList` *generator* *length* - Generate list of size *length*, with each element *i* equal to the - value returned by *generator* `i`. For example, - - ```nix - builtins.genList (x: x * x) 5 - ``` - - returns the list `[ 0 1 4 9 16 ]`. - - - `builtins.getAttr` *s* *set* - `getAttr` returns the attribute named *s* from *set*. Evaluation - aborts if the attribute doesn’t exist. This is a dynamic version of - the `.` operator, since *s* is an expression rather than an - identifier. - - - `builtins.getEnv` *s* - `getEnv` returns the value of the environment variable *s*, or an - empty string if the variable doesn’t exist. This function should be - used with care, as it can introduce all sorts of nasty environment - dependencies in your Nix expression. - - `getEnv` is used in Nix Packages to locate the file - `~/.nixpkgs/config.nix`, which contains user-local settings for Nix - Packages. (That is, it does a `getEnv "HOME"` to locate the user’s - home directory.) - - - `builtins.hasAttr` *s* *set* - `hasAttr` returns `true` if *set* has an attribute named *s*, and - `false` otherwise. This is a dynamic version of the `?` operator, - since *s* is an expression rather than an identifier. - - - `builtins.hashString` *type* *s* - Return a base-16 representation of the cryptographic hash of string - *s*. The hash algorithm specified by *type* must be one of `"md5"`, - `"sha1"`, `"sha256"` or `"sha512"`. - - - `builtins.hashFile` *type* *p* - Return a base-16 representation of the cryptographic hash of the - file at path *p*. The hash algorithm specified by *type* must be one - of `"md5"`, `"sha1"`, `"sha256"` or `"sha512"`. - - - `builtins.head` *list* - Return the first element of a list; abort evaluation if the argument - isn’t a list or is an empty list. You can test whether a list is - empty by comparing it with `[]`. + `derivation` is described in [its own section](derivations.md). - `import` *path*; `builtins.import` *path* + Load, parse and return the Nix expression in the file *path*. If *path* is a directory, the file ` default.nix ` in that directory is loaded. Evaluation aborts if the file doesn’t exist or contains @@ -376,535 +22,47 @@ For instance, `derivation` is also available as `builtins.derivation`. system: you can put any Nix expression (such as a set or a function) in a separate file, and use it from Nix expressions in other files. - + > **Note** - > + > > Unlike some languages, `import` is a regular function in Nix. > Paths using the angle bracket syntax (e.g., `import` *\<foo\>*) > are [normal path values](language-values.md). - + A Nix expression loaded by `import` must not contain any *free variables* (identifiers that are not defined in the Nix expression itself and are not built-in). Therefore, it cannot refer to variables that are in scope at the call site. For instance, if you have a calling expression - + ```nix rec { x = 123; y = import ./foo.nix; } ``` - + then the following `foo.nix` will give an error: - + ```nix x + 456 ``` - + since `x` is not in scope in `foo.nix`. If you want `x` to be available in `foo.nix`, you should pass it as a function argument: - + ```nix rec { x = 123; y = import ./foo.nix x; } ``` - + and - + ```nix x: x + 456 ``` - + (The function argument doesn’t have to be called `x` in `foo.nix`; any name would work.) - - - `builtins.intersectAttrs` *e1* *e2* - Return a set consisting of the attributes in the set *e2* that also - exist in the set *e1*. - - - `builtins.isAttrs` *e* - Return `true` if *e* evaluates to a set, and `false` otherwise. - - - `builtins.isList` *e* - Return `true` if *e* evaluates to a list, and `false` otherwise. - - - `builtins.isFunction` *e* - Return `true` if *e* evaluates to a function, and `false` otherwise. - - - `builtins.isString` *e* - Return `true` if *e* evaluates to a string, and `false` otherwise. - - - `builtins.isInt` *e* - Return `true` if *e* evaluates to an int, and `false` otherwise. - - - `builtins.isFloat` *e* - Return `true` if *e* evaluates to a float, and `false` otherwise. - - - `builtins.isBool` *e* - Return `true` if *e* evaluates to a bool, and `false` otherwise. - - - `builtins.isPath` *e* - Return `true` if *e* evaluates to a path, and `false` otherwise. - - - `isNull` *e*; `builtins.isNull` *e* - Return `true` if *e* evaluates to `null`, and `false` otherwise. - - > **Warning** - > - > This function is *deprecated*; just write `e == null` instead. - - - `builtins.length` *e* - Return the length of the list *e*. - - - `builtins.lessThan` *e1* *e2* - Return `true` if the number *e1* is less than the number *e2*, and - `false` otherwise. Evaluation aborts if either *e1* or *e2* does not - evaluate to a number. - - - `builtins.listToAttrs` *e* - Construct a set from a list specifying the names and values of each - attribute. Each element of the list should be a set consisting of a - string-valued attribute `name` specifying the name of the attribute, - and an attribute `value` specifying its value. Example: - - ```nix - builtins.listToAttrs - [ { name = "foo"; value = 123; } - { name = "bar"; value = 456; } - ] - ``` - - evaluates to - - ```nix - { foo = 123; bar = 456; } - ``` - - - `map` *f* *list*; `builtins.map` *f* *list* - Apply the function *f* to each element in the list *list*. For - example, - - ```nix - map (x: "foo" + x) [ "bar" "bla" "abc" ] - ``` - - evaluates to `[ "foobar" "foobla" "fooabc" ]`. - - - `builtins.match` *regex* *str* - Returns a list if the [extended POSIX regular - expression](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04) - *regex* matches *str* precisely, otherwise returns `null`. Each item - in the list is a regex group. - - ```nix - builtins.match "ab" "abc" - ``` - - Evaluates to `null`. - - ```nix - builtins.match "abc" "abc" - ``` - - Evaluates to `[ ]`. - - ```nix - builtins.match "a(b)(c)" "abc" - ``` - - Evaluates to `[ "b" "c" ]`. - - ```nix - builtins.match "[[:space:]]+([[:upper:]]+)[[:space:]]+" " FOO " - ``` - - Evaluates to `[ "foo" ]`. - - - `builtins.mul` *e1* *e2* - Return the product of the numbers *e1* and *e2*. - - - `builtins.parseDrvName` *s* - Split the string *s* into a package name and version. The package - name is everything up to but not including the first dash followed - by a digit, and the version is everything following that dash. The - result is returned in a set `{ name, version }`. Thus, - `builtins.parseDrvName "nix-0.12pre12876"` returns `{ name = - "nix"; version = "0.12pre12876"; }`. - - - `builtins.path` *args* - An enrichment of the built-in path type, based on the attributes - present in *args*. All are optional except `path`: - - - path - The underlying path. - - - name - The name of the path when added to the store. This can used to - reference paths that have nix-illegal characters in their names, - like `@`. - - - filter - A function of the type expected by `builtins.filterSource`, - with the same semantics. - - - recursive - When `false`, when `path` is added to the store it is with a - flat hash, rather than a hash of the NAR serialization of the - file. Thus, `path` must refer to a regular file, not a - directory. This allows similar behavior to `fetchurl`. Defaults - to `true`. - - - sha256 - When provided, this is the expected hash of the file at the - path. Evaluation will fail if the hash is incorrect, and - providing a hash allows `builtins.path` to be used even when the - `pure-eval` nix config option is on. - - - `builtins.pathExists` *path* - Return `true` if the path *path* exists at evaluation time, and - `false` otherwise. - - - `builtins.placeholder` *output* - Return a placeholder string for the specified *output* that will be - substituted by the corresponding output path at build time. Typical - outputs would be `"out"`, `"bin"` or `"dev"`. - - - `builtins.readDir` *path* - Return the contents of the directory *path* as a set mapping - directory entries to the corresponding file type. For instance, if - directory `A` contains a regular file `B` and another directory - `C`, then `builtins.readDir ./A` will return the set - - ```nix - { B = "regular"; C = "directory"; } - ``` - - The possible values for the file type are `"regular"`, - `"directory"`, `"symlink"` and `"unknown"`. - - - `builtins.readFile` *path* - Return the contents of the file *path* as a string. - - - `removeAttrs` *set* *list*; `builtins.removeAttrs` *set* *list* - Remove the attributes listed in *list* from *set*. The attributes - don’t have to exist in *set*. For instance, - - ```nix - removeAttrs { x = 1; y = 2; z = 3; } [ "a" "x" "z" ] - ``` - - evaluates to `{ y = 2; }`. - - - `builtins.replaceStrings` *from* *to* *s* - Given string *s*, replace every occurrence of the strings in *from* - with the corresponding string in *to*. For example, - - ```nix - builtins.replaceStrings ["oo" "a"] ["a" "i"] "foobar" - ``` - - evaluates to `"fabir"`. - - - `builtins.seq` *e1* *e2* - Evaluate *e1*, then evaluate and return *e2*. This ensures that a - computation is strict in the value of *e1*. - - - `builtins.sort` *comparator* *list* - Return *list* in sorted order. It repeatedly calls the function - *comparator* with two elements. The comparator should return `true` - if the first element is less than the second, and `false` otherwise. - For example, - - ```nix - builtins.sort builtins.lessThan [ 483 249 526 147 42 77 ] - ``` - - produces the list `[ 42 77 147 249 483 526 ]`. - - This is a stable sort: it preserves the relative order of elements - deemed equal by the comparator. - - - `builtins.split` *regex* *str* - Returns a list composed of non matched strings interleaved with the - lists of the [extended POSIX regular - expression](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04) - *regex* matches of *str*. Each item in the lists of matched - sequences is a regex group. - - ```nix - builtins.split "(a)b" "abc" - ``` - - Evaluates to `[ "" [ "a" ] "c" ]`. - - ```nix - builtins.split "([ac])" "abc" - ``` - - Evaluates to `[ "" [ "a" ] "b" [ "c" ] "" ]`. - - ```nix - builtins.split "(a)|(c)" "abc" - ``` - - Evaluates to `[ "" [ "a" null ] "b" [ null "c" ] "" ]`. - - ```nix - builtins.split "([[:upper:]]+)" " FOO " - ``` - - Evaluates to `[ " " [ "FOO" ] " " ]`. - - - `builtins.splitVersion` *s* - Split a string representing a version into its components, by the - same version splitting logic underlying the version comparison in - [`nix-env -u`](../command-ref/nix-env.md#operation---upgrade). - - - `builtins.stringLength` *e* - Return the length of the string *e*. If *e* is not a string, - evaluation is aborted. - - - `builtins.sub` *e1* *e2* - Return the difference between the numbers *e1* and *e2*. - - - `builtins.substring` *start* *len* *s* - Return the substring of *s* from character position *start* - (zero-based) up to but not including *start + len*. If *start* is - greater than the length of the string, an empty string is returned, - and if *start + len* lies beyond the end of the string, only the - substring up to the end of the string is returned. *start* must be - non-negative. For example, - - ```nix - builtins.substring 0 3 "nixos" - ``` - - evaluates to `"nix"`. - - - `builtins.tail` *list* - Return the second to last elements of a list; abort evaluation if - the argument isn’t a list or is an empty list. - - - `throw` *s*; `builtins.throw` *s* - Throw an error message *s*. This usually aborts Nix expression - evaluation, but in `nix-env -qa` and other commands that try to - evaluate a set of derivations to get information about those - derivations, a derivation that throws an error is silently skipped - (which is not the case for `abort`). - - - `builtins.toFile` *name* *s* - Store the string *s* in a file in the Nix store and return its - path. The file has suffix *name*. This file can be used as an - input to derivations. One application is to write builders - “inline”. For instance, the following Nix expression combines the - [Nix expression for GNU Hello](expression-syntax.md) and its - [build script](build-script.md) into one file: - - ```nix - { stdenv, fetchurl, perl }: - - stdenv.mkDerivation { - name = "hello-2.1.1"; - - builder = builtins.toFile "builder.sh" " - source $stdenv/setup - - PATH=$perl/bin:$PATH - - tar xvfz $src - cd hello-* - ./configure --prefix=$out - make - make install - "; - - src = fetchurl { - url = "http://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz"; - sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465"; - }; - inherit perl; - } - ``` - - It is even possible for one file to refer to another, e.g., - - ```nix - builder = let - configFile = builtins.toFile "foo.conf" " - # This is some dummy configuration file. - ... - "; - in builtins.toFile "builder.sh" " - source $stdenv/setup - ... - cp ${configFile} $out/etc/foo.conf - "; - ``` - - Note that `${configFile}` is an - [antiquotation](language-values.md), so the result of the - expression `configFile` - (i.e., a path like `/nix/store/m7p7jfny445k...-foo.conf`) will be - spliced into the resulting string. - - It is however *not* allowed to have files mutually referring to each - other, like so: - - ```nix - let - foo = builtins.toFile "foo" "...${bar}..."; - bar = builtins.toFile "bar" "...${foo}..."; - in foo - ``` - - This is not allowed because it would cause a cyclic dependency in - the computation of the cryptographic hashes for `foo` and `bar`. - - It is also not possible to reference the result of a derivation. If - you are using Nixpkgs, the `writeTextFile` function is able to do - that. - - - `builtins.toJSON` *e* - Return a string containing a JSON representation of *e*. Strings, - integers, floats, booleans, nulls and lists are mapped to their JSON - equivalents. Sets (except derivations) are represented as objects. - Derivations are translated to a JSON string containing the - derivation’s output path. Paths are copied to the store and - represented as a JSON string of the resulting store path. - - - `builtins.toPath` *s* - DEPRECATED. Use `/. + "/path"` to convert a string into an absolute - path. For relative paths, use `./. + "/path"`. - - - `toString` *e*; `builtins.toString` *e* - Convert the expression *e* to a string. *e* can be: - - - A string (in which case the string is returned unmodified). - - - A path (e.g., `toString /foo/bar` yields `"/foo/bar"`. - - - A set containing `{ __toString = self: ...; }`. - - - An integer. - - - A list, in which case the string representations of its elements - are joined with spaces. - - - A Boolean (`false` yields `""`, `true` yields `"1"`). - - - `null`, which yields the empty string. - - - `builtins.toXML` *e* - Return a string containing an XML representation of *e*. The main - application for `toXML` is to communicate information with the - builder in a more structured format than plain environment - variables. - - Here is an example where this is the case: - - ```nix - { stdenv, fetchurl, libxslt, jira, uberwiki }: - - stdenv.mkDerivation (rec { - name = "web-server"; - - buildInputs = [ libxslt ]; - - builder = builtins.toFile "builder.sh" " - source $stdenv/setup - mkdir $out - echo "$servlets" | xsltproc ${stylesheet} - > $out/server-conf.xml ① - "; - - stylesheet = builtins.toFile "stylesheet.xsl" ② - "<?xml version='1.0' encoding='UTF-8'?> - <xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/XSL/Transform' version='1.0'> - <xsl:template match='/'> - <Configure> - <xsl:for-each select='/expr/list/attrs'> - <Call name='addWebApplication'> - <Arg><xsl:value-of select=\"attr[@name = 'path']/string/@value\" /></Arg> - <Arg><xsl:value-of select=\"attr[@name = 'war']/path/@value\" /></Arg> - </Call> - </xsl:for-each> - </Configure> - </xsl:template> - </xsl:stylesheet> - "; - - servlets = builtins.toXML [ ③ - { path = "/bugtracker"; war = jira + "/lib/atlassian-jira.war"; } - { path = "/wiki"; war = uberwiki + "/uberwiki.war"; } - ]; - }) - ``` - - The builder is supposed to generate the configuration file for a - [Jetty servlet container](http://jetty.mortbay.org/). A servlet - container contains a number of servlets (`*.war` files) each - exported under a specific URI prefix. So the servlet configuration - is a list of sets containing the `path` and `war` of the servlet - (①). This kind of information is difficult to communicate with the - normal method of passing information through an environment - variable, which just concatenates everything together into a - string (which might just work in this case, but wouldn’t work if - fields are optional or contain lists themselves). Instead the Nix - expression is converted to an XML representation with `toXML`, - which is unambiguous and can easily be processed with the - appropriate tools. For instance, in the example an XSLT stylesheet - (at point ②) is applied to it (at point ①) to generate the XML - configuration file for the Jetty server. The XML representation - produced at point ③ by `toXML` is as follows: - - ```xml - <?xml version='1.0' encoding='utf-8'?> - <expr> - <list> - <attrs> - <attr name="path"> - <string value="/bugtracker" /> - </attr> - <attr name="war"> - <path value="/nix/store/d1jh9pasa7k2...-jira/lib/atlassian-jira.war" /> - </attr> - </attrs> - <attrs> - <attr name="path"> - <string value="/wiki" /> - </attr> - <attr name="war"> - <path value="/nix/store/y6423b1yi4sx...-uberwiki/uberwiki.war" /> - </attr> - </attrs> - </list> - </expr> - ``` - - Note that we used the `toFile` built-in to write the builder and - the stylesheet “inline” in the Nix expression. The path of the - stylesheet is spliced into the builder using the syntax `xsltproc - ${stylesheet}`. - - - `builtins.trace` *e1* *e2* - Evaluate *e1* and print its abstract syntax representation on - standard error. Then return *e2*. This function is useful for - debugging. - - - `builtins.tryEval` *e* - Try to shallowly evaluate *e*. Return a set containing the - attributes `success` (`true` if *e* evaluated successfully, - `false` if an error was thrown) and `value`, equalling *e* if - successful and `false` otherwise. Note that this doesn't evaluate - *e* deeply, so ` let e = { x = throw ""; }; in (builtins.tryEval - e).success ` will be `true`. Using ` builtins.deepSeq ` one can - get the expected result: `let e = { x = throw ""; }; in - (builtins.tryEval (builtins.deepSeq e e)).success` will be - `false`. - - - `builtins.typeOf` *e* - Return a string representing the type of the value *e*, namely - `"int"`, `"bool"`, `"string"`, `"path"`, `"null"`, `"set"`, - `"list"`, `"lambda"` or `"float"`. diff --git a/src/libexpr/primops.cc b/src/libexpr/primops.cc index 902a37e6b..78892053a 100644 --- a/src/libexpr/primops.cc +++ b/src/libexpr/primops.cc @@ -275,6 +275,16 @@ static void prim_typeOf(EvalState & state, const Pos & pos, Value * * args, Valu mkString(v, state.symbols.create(t)); } +static RegisterPrimOp primop_typeOf({ + .name = "__typeOf", + .args = {"e"}, + .doc = R"( + Return a string representing the type of the value *e*, namely + `"int"`, `"bool"`, `"string"`, `"path"`, `"null"`, `"set"`, + `"list"`, `"lambda"` or `"float"`. + )", + .fun = prim_typeOf, +}); /* Determine whether the argument is the null value. */ static void prim_isNull(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -283,6 +293,18 @@ static void prim_isNull(EvalState & state, const Pos & pos, Value * * args, Valu mkBool(v, args[0]->type == tNull); } +static RegisterPrimOp primop_isNull({ + .name = "isNull", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to `null`, and `false` otherwise. + + > **Warning** + > + > This function is *deprecated*; just write `e == null` instead. + )", + .fun = prim_isNull, +}); /* Determine whether the argument is a function. */ static void prim_isFunction(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -302,6 +324,14 @@ static void prim_isFunction(EvalState & state, const Pos & pos, Value * * args, mkBool(v, res); } +static RegisterPrimOp primop_isFunction({ + .name = "__isFunction", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a function, and `false` otherwise. + )", + .fun = prim_isFunction, +}); /* Determine whether the argument is an integer. */ static void prim_isInt(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -310,6 +340,15 @@ static void prim_isInt(EvalState & state, const Pos & pos, Value * * args, Value mkBool(v, args[0]->type == tInt); } +static RegisterPrimOp primop_isInt({ + .name = "__isInt", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to an integer, and `false` otherwise. + )", + .fun = prim_isInt, +}); + /* Determine whether the argument is a float. */ static void prim_isFloat(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -317,6 +356,15 @@ static void prim_isFloat(EvalState & state, const Pos & pos, Value * * args, Val mkBool(v, args[0]->type == tFloat); } +static RegisterPrimOp primop_isFloat({ + .name = "__isFloat", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a float, and `false` otherwise. + )", + .fun = prim_isFloat, +}); + /* Determine whether the argument is a string. */ static void prim_isString(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -324,6 +372,14 @@ static void prim_isString(EvalState & state, const Pos & pos, Value * * args, Va mkBool(v, args[0]->type == tString); } +static RegisterPrimOp primop_isString({ + .name = "__isString", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a string, and `false` otherwise. + )", + .fun = prim_isString, +}); /* Determine whether the argument is a Boolean. */ static void prim_isBool(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -332,6 +388,15 @@ static void prim_isBool(EvalState & state, const Pos & pos, Value * * args, Valu mkBool(v, args[0]->type == tBool); } +static RegisterPrimOp primop_isBool({ + .name = "__isBool", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a bool, and `false` otherwise. + )", + .fun = prim_isBool, +}); + /* Determine whether the argument is a path. */ static void prim_isPath(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -339,6 +404,15 @@ static void prim_isPath(EvalState & state, const Pos & pos, Value * * args, Valu mkBool(v, args[0]->type == tPath); } +static RegisterPrimOp primop_isPath({ + .name = "__isPath", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a path, and `false` otherwise. + )", + .fun = prim_isPath, +}); + struct CompareValues { bool operator () (const Value * v1, const Value * v2) const @@ -459,14 +533,23 @@ static RegisterPrimOp primop_abort({ } }); - -static void prim_throw(EvalState & state, const Pos & pos, Value * * args, Value & v) -{ - PathSet context; - string s = state.coerceToString(pos, *args[0], context); - throw ThrownError(s); -} - +static RegisterPrimOp primop_throw({ + .name = "throw", + .args = {"s"}, + .doc = R"( + Throw an error message *s*. This usually aborts Nix expression + evaluation, but in `nix-env -qa` and other commands that try to + evaluate a set of derivations to get information about those + derivations, a derivation that throws an error is silently skipped + (which is not the case for `abort`). + )", + .fun = [](EvalState & state, const Pos & pos, Value * * args, Value & v) + { + PathSet context; + string s = state.coerceToString(pos, *args[0], context); + throw ThrownError(s); + } +}); static void prim_addErrorContext(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -497,6 +580,22 @@ static void prim_tryEval(EvalState & state, const Pos & pos, Value * * args, Val v.attrs->sort(); } +static RegisterPrimOp primop_tryEval({ + .name = "__tryEval", + .args = {"e"}, + .doc = R"( + Try to shallowly evaluate *e*. Return a set containing the + attributes `success` (`true` if *e* evaluated successfully, + `false` if an error was thrown) and `value`, equalling *e* if + successful and `false` otherwise. Note that this doesn't evaluate + *e* deeply, so ` let e = { x = throw ""; }; in (builtins.tryEval + e).success ` will be `true`. Using ` builtins.deepSeq ` one can + get the expected result: `let e = { x = throw ""; }; in + (builtins.tryEval (builtins.deepSeq e e)).success` will be + `false`. + )", + .fun = prim_tryEval, +}); /* Return an environment variable. Use with care. */ static void prim_getEnv(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -505,6 +604,22 @@ static void prim_getEnv(EvalState & state, const Pos & pos, Value * * args, Valu mkString(v, evalSettings.restrictEval || evalSettings.pureEval ? "" : getEnv(name).value_or("")); } +static RegisterPrimOp primop_getEnv({ + .name = "__getEnv", + .args = {"s"}, + .doc = R"( + `getEnv` returns the value of the environment variable *s*, or an + empty string if the variable doesn’t exist. This function should be + used with care, as it can introduce all sorts of nasty environment + dependencies in your Nix expression. + + `getEnv` is used in Nix Packages to locate the file + `~/.nixpkgs/config.nix`, which contains user-local settings for Nix + Packages. (That is, it does a `getEnv "HOME"` to locate the user’s + home directory.) + )", + .fun = prim_getEnv, +}); /* Evaluate the first argument, then return the second argument. */ static void prim_seq(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -514,6 +629,15 @@ static void prim_seq(EvalState & state, const Pos & pos, Value * * args, Value & v = *args[1]; } +static RegisterPrimOp primop_seq({ + .name = "__seq", + .args = {"e1", "e2"}, + .doc = R"( + Evaluate *e1*, then evaluate and return *e2*. This ensures that a + computation is strict in the value of *e1*. + )", + .fun = prim_seq, +}); /* Evaluate the first argument deeply (i.e. recursing into lists and attrsets), then return the second argument. */ @@ -524,6 +648,16 @@ static void prim_deepSeq(EvalState & state, const Pos & pos, Value * * args, Val v = *args[1]; } +static RegisterPrimOp primop_deepSeq({ + .name = "__deepSeq", + .args = {"e1", "e2"}, + .doc = R"( + This is like `seq e1 e2`, except that *e1* is evaluated *deeply*: + if it’s a list or set, its elements or attributes are also + evaluated recursively. + )", + .fun = prim_deepSeq, +}); /* Evaluate the first expression and print it on standard error. Then return the second expression. Useful for debugging. */ @@ -538,6 +672,17 @@ static void prim_trace(EvalState & state, const Pos & pos, Value * * args, Value v = *args[1]; } +static RegisterPrimOp primop_trace({ + .name = "__trace", + .args = {"e1", "e2"}, + .doc = R"( + Evaluate *e1* and print its abstract syntax representation on + standard error. Then return *e2*. This function is useful for + debugging. + )", + .fun = prim_trace, +}); + /************************************************************* * Derivations @@ -879,6 +1024,17 @@ static void prim_placeholder(EvalState & state, const Pos & pos, Value * * args, mkString(v, hashPlaceholder(state.forceStringNoCtx(*args[0], pos))); } +static RegisterPrimOp primop_placeholder({ + .name = "placeholder", + .args = {"output"}, + .doc = R"( + Return a placeholder string for the specified *output* that will be + substituted by the corresponding output path at build time. Typical + outputs would be `"out"`, `"bin"` or `"dev"`. + )", + .fun = prim_placeholder, +}); + /************************************************************* * Paths @@ -893,6 +1049,15 @@ static void prim_toPath(EvalState & state, const Pos & pos, Value * * args, Valu mkString(v, canonPath(path), context); } +static RegisterPrimOp primop_toPath({ + .name = "__toPath", + .args = {"s"}, + .doc = R"( + DEPRECATED. Use `/. + "/path"` to convert a string into an absolute + path. For relative paths, use `./. + "/path"`. + )", + .fun = prim_toPath, +}); /* Allow a valid store path to be used in an expression. This is useful in some generated expressions such as in nix-push, which @@ -904,6 +1069,9 @@ static void prim_toPath(EvalState & state, const Pos & pos, Value * * args, Valu corner cases. */ static void prim_storePath(EvalState & state, const Pos & pos, Value * * args, Value & v) { + if (evalSettings.pureEval) + throw EvalError("builtins.storePath' is not allowed in pure evaluation mode"); + PathSet context; Path path = state.checkSourcePath(state.coerceToPath(pos, *args[0], context)); /* Resolve symlinks in ‘path’, unless ‘path’ itself is a symlink @@ -949,6 +1117,15 @@ static void prim_pathExists(EvalState & state, const Pos & pos, Value * * args, } } +static RegisterPrimOp primop_pathExists({ + .name = "__pathExists", + .args = {"path"}, + .doc = R"( + Return `true` if the path *path* exists at evaluation time, and + `false` otherwise. + )", + .fun = prim_pathExists, +}); /* Return the base name of the given string, i.e., everything following the last slash. */ @@ -958,6 +1135,16 @@ static void prim_baseNameOf(EvalState & state, const Pos & pos, Value * * args, mkString(v, baseNameOf(state.coerceToString(pos, *args[0], context, false, false)), context); } +static RegisterPrimOp primop_baseNameOf({ + .name = "baseNameOf", + .args = {"s"}, + .doc = R"( + Return the *base name* of the string *s*, that is, everything + following the final slash in the string. This is similar to the GNU + `basename` command. + )", + .fun = prim_baseNameOf, +}); /* Return the directory of the given path, i.e., everything before the last slash. Return either a path or a string depending on the type @@ -969,6 +1156,16 @@ static void prim_dirOf(EvalState & state, const Pos & pos, Value * * args, Value if (args[0]->type == tPath) mkPath(v, dir.c_str()); else mkString(v, dir, context); } +static RegisterPrimOp primop_dirOf({ + .name = "dirOf", + .args = {"s"}, + .doc = R"( + Return the directory part of the string *s*, that is, everything + before the final slash in the string. This is similar to the GNU + `dirname` command. + )", + .fun = prim_dirOf, +}); /* Return the contents of a file as a string. */ static void prim_readFile(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -989,6 +1186,14 @@ static void prim_readFile(EvalState & state, const Pos & pos, Value * * args, Va mkString(v, s.c_str()); } +static RegisterPrimOp primop_readFile({ + .name = "__readFile", + .args = {"path"}, + .doc = R"( + Return the contents of the file *path* as a string. + )", + .fun = prim_readFile, +}); /* Find a file in the Nix search path. Used to implement <x> paths, which are desugared to 'findFile __nixPath "x"'. */ @@ -1051,6 +1256,17 @@ static void prim_hashFile(EvalState & state, const Pos & pos, Value * * args, Va mkString(v, hashFile(*ht, state.checkSourcePath(p)).to_string(Base16, false), context); } +static RegisterPrimOp primop_hashFile({ + .name = "__hashFile", + .args = {"type", "p"}, + .doc = R"( + Return a base-16 representation of the cryptographic hash of the + file at path *p*. The hash algorithm specified by *type* must be one + of `"md5"`, `"sha1"`, `"sha256"` or `"sha512"`. + )", + .fun = prim_hashFile, +}); + /* Read a directory (without . or ..) */ static void prim_readDir(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1082,6 +1298,25 @@ static void prim_readDir(EvalState & state, const Pos & pos, Value * * args, Val v.attrs->sort(); } +static RegisterPrimOp primop_readDir({ + .name = "__readDir", + .args = {"path"}, + .doc = R"( + Return the contents of the directory *path* as a set mapping + directory entries to the corresponding file type. For instance, if + directory `A` contains a regular file `B` and another directory + `C`, then `builtins.readDir ./A` will return the set + + ```nix + { B = "regular"; C = "directory"; } + ``` + + The possible values for the file type are `"regular"`, + `"directory"`, `"symlink"` and `"unknown"`. + )", + .fun = prim_readDir, +}); + /************************************************************* * Creating files @@ -1099,6 +1334,102 @@ static void prim_toXML(EvalState & state, const Pos & pos, Value * * args, Value mkString(v, out.str(), context); } +static RegisterPrimOp primop_toXML({ + .name = "__toXML", + .args = {"e"}, + .doc = R"( + Return a string containing an XML representation of *e*. The main + application for `toXML` is to communicate information with the + builder in a more structured format than plain environment + variables. + + Here is an example where this is the case: + + ```nix + { stdenv, fetchurl, libxslt, jira, uberwiki }: + + stdenv.mkDerivation (rec { + name = "web-server"; + + buildInputs = [ libxslt ]; + + builder = builtins.toFile "builder.sh" " + source $stdenv/setup + mkdir $out + echo "$servlets" | xsltproc ${stylesheet} - > $out/server-conf.xml ① + "; + + stylesheet = builtins.toFile "stylesheet.xsl" ② + "<?xml version='1.0' encoding='UTF-8'?> + <xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/XSL/Transform' version='1.0'> + <xsl:template match='/'> + <Configure> + <xsl:for-each select='/expr/list/attrs'> + <Call name='addWebApplication'> + <Arg><xsl:value-of select=\"attr[@name = 'path']/string/@value\" /></Arg> + <Arg><xsl:value-of select=\"attr[@name = 'war']/path/@value\" /></Arg> + </Call> + </xsl:for-each> + </Configure> + </xsl:template> + </xsl:stylesheet> + "; + + servlets = builtins.toXML [ ③ + { path = "/bugtracker"; war = jira + "/lib/atlassian-jira.war"; } + { path = "/wiki"; war = uberwiki + "/uberwiki.war"; } + ]; + }) + ``` + + The builder is supposed to generate the configuration file for a + [Jetty servlet container](http://jetty.mortbay.org/). A servlet + container contains a number of servlets (`*.war` files) each + exported under a specific URI prefix. So the servlet configuration + is a list of sets containing the `path` and `war` of the servlet + (①). This kind of information is difficult to communicate with the + normal method of passing information through an environment + variable, which just concatenates everything together into a + string (which might just work in this case, but wouldn’t work if + fields are optional or contain lists themselves). Instead the Nix + expression is converted to an XML representation with `toXML`, + which is unambiguous and can easily be processed with the + appropriate tools. For instance, in the example an XSLT stylesheet + (at point ②) is applied to it (at point ①) to generate the XML + configuration file for the Jetty server. The XML representation + produced at point ③ by `toXML` is as follows: + + ```xml + <?xml version='1.0' encoding='utf-8'?> + <expr> + <list> + <attrs> + <attr name="path"> + <string value="/bugtracker" /> + </attr> + <attr name="war"> + <path value="/nix/store/d1jh9pasa7k2...-jira/lib/atlassian-jira.war" /> + </attr> + </attrs> + <attrs> + <attr name="path"> + <string value="/wiki" /> + </attr> + <attr name="war"> + <path value="/nix/store/y6423b1yi4sx...-uberwiki/uberwiki.war" /> + </attr> + </attrs> + </list> + </expr> + ``` + + Note that we used the `toFile` built-in to write the builder and + the stylesheet “inline” in the Nix expression. The path of the + stylesheet is spliced into the builder using the syntax `xsltproc + ${stylesheet}`. + )", + .fun = prim_toXML, +}); /* Convert the argument (which can be any Nix expression) to a JSON string. Not all Nix expressions can be sensibly or completely @@ -1111,6 +1442,19 @@ static void prim_toJSON(EvalState & state, const Pos & pos, Value * * args, Valu mkString(v, out.str(), context); } +static RegisterPrimOp primop_toJSON({ + .name = "__toJSON", + .args = {"e"}, + .doc = R"( + Return a string containing a JSON representation of *e*. Strings, + integers, floats, booleans, nulls and lists are mapped to their JSON + equivalents. Sets (except derivations) are represented as objects. + Derivations are translated to a JSON string containing the + derivation’s output path. Paths are copied to the store and + represented as a JSON string of the resulting store path. + )", + .fun = prim_toJSON, +}); /* Parse a JSON string to a value. */ static void prim_fromJSON(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1119,6 +1463,20 @@ static void prim_fromJSON(EvalState & state, const Pos & pos, Value * * args, Va parseJSON(state, s, v); } +static RegisterPrimOp primop_fromJSON({ + .name = "__fromJSON", + .args = {"e"}, + .doc = R"( + Convert a JSON string to a Nix value. For example, + + ```nix + builtins.fromJSON ''{"x": [1, 2, 3], "y": null}'' + ``` + + returns the value `{ x = [ 1 2 3 ]; y = null; }`. + )", + .fun = prim_fromJSON, +}); /* Store a string in the Nix store as a source file that can be used as an input by derivations. */ @@ -1153,6 +1511,83 @@ static void prim_toFile(EvalState & state, const Pos & pos, Value * * args, Valu mkString(v, storePath, {storePath}); } +static RegisterPrimOp primop_toFile({ + .name = "__toFile", + .args = {"name", "s"}, + .doc = R"( + Store the string *s* in a file in the Nix store and return its + path. The file has suffix *name*. This file can be used as an + input to derivations. One application is to write builders + “inline”. For instance, the following Nix expression combines the + [Nix expression for GNU Hello](expression-syntax.md) and its + [build script](build-script.md) into one file: + + ```nix + { stdenv, fetchurl, perl }: + + stdenv.mkDerivation { + name = "hello-2.1.1"; + + builder = builtins.toFile "builder.sh" " + source $stdenv/setup + + PATH=$perl/bin:$PATH + + tar xvfz $src + cd hello-* + ./configure --prefix=$out + make + make install + "; + + src = fetchurl { + url = "http://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz"; + sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465"; + }; + inherit perl; + } + ``` + + It is even possible for one file to refer to another, e.g., + + ```nix + builder = let + configFile = builtins.toFile "foo.conf" " + # This is some dummy configuration file. + ... + "; + in builtins.toFile "builder.sh" " + source $stdenv/setup + ... + cp ${configFile} $out/etc/foo.conf + "; + ``` + + Note that `${configFile}` is an + [antiquotation](language-values.md), so the result of the + expression `configFile` + (i.e., a path like `/nix/store/m7p7jfny445k...-foo.conf`) will be + spliced into the resulting string. + + It is however *not* allowed to have files mutually referring to each + other, like so: + + ```nix + let + foo = builtins.toFile "foo" "...${bar}..."; + bar = builtins.toFile "bar" "...${foo}..."; + in foo + ``` + + This is not allowed because it would cause a cyclic dependency in + the computation of the cryptographic hashes for `foo` and `bar`. + + It is also not possible to reference the result of a derivation. If + you are using Nixpkgs, the `writeTextFile` function is able to do + that. + )", + .fun = prim_toFile, +}); static void addPath(EvalState & state, const Pos & pos, const string & name, const Path & path_, Value * filterFun, FileIngestionMethod method, const std::optional<Hash> expectedHash, Value & v) @@ -1223,6 +1658,48 @@ static void prim_filterSource(EvalState & state, const Pos & pos, Value * * args addPath(state, pos, std::string(baseNameOf(path)), path, args[0], FileIngestionMethod::Recursive, std::nullopt, v); } +static RegisterPrimOp primop_filterSource({ + .name = "__filterSource", + .args = {"e1", "e2"}, + .doc = R"( + This function allows you to copy sources into the Nix store while + filtering certain files. For instance, suppose that you want to use + the directory `source-dir` as an input to a Nix expression, e.g. + + ```nix + stdenv.mkDerivation { + ... + src = ./source-dir; + } + ``` + + However, if `source-dir` is a Subversion working copy, then all + those annoying `.svn` subdirectories will also be copied to the + store. Worse, the contents of those directories may change a lot, + causing lots of spurious rebuilds. With `filterSource` you can + filter out the `.svn` directories: + + ```nix + src = builtins.filterSource + (path: type: type != "directory" || baseNameOf path != ".svn") + ./source-dir; + ``` + + Thus, the first argument *e1* must be a predicate function that is + called for each regular file, directory or symlink in the source + tree *e2*. If the function returns `true`, the file is copied to the + Nix store, otherwise it is omitted. The function is called with two + arguments. The first is the full path of the file. The second is a + string that identifies the type of the file, which is either + `"regular"`, `"directory"`, `"symlink"` or `"unknown"` (for other + kinds of files such as device nodes or fifos — but note that those + cannot be copied to the Nix store, so if the predicate returns + `true` for them, the copy will fail). If you exclude a directory, + the entire corresponding subtree of *e2* will be excluded. + )", + .fun = prim_filterSource, +}); + static void prim_path(EvalState & state, const Pos & pos, Value * * args, Value & v) { state.forceAttrs(*args[0], pos); @@ -1268,6 +1745,41 @@ static void prim_path(EvalState & state, const Pos & pos, Value * * args, Value addPath(state, pos, name, path, filterFun, method, expectedHash, v); } +static RegisterPrimOp primop_path({ + .name = "__path", + .args = {"args"}, + .doc = R"( + An enrichment of the built-in path type, based on the attributes + present in *args*. All are optional except `path`: + + - path + The underlying path. + + - name + The name of the path when added to the store. This can used to + reference paths that have nix-illegal characters in their names, + like `@`. + + - filter + A function of the type expected by `builtins.filterSource`, + with the same semantics. + + - recursive + When `false`, when `path` is added to the store it is with a + flat hash, rather than a hash of the NAR serialization of the + file. Thus, `path` must refer to a regular file, not a + directory. This allows similar behavior to `fetchurl`. Defaults + to `true`. + + - sha256 + When provided, this is the expected hash of the file at the + path. Evaluation will fail if the hash is incorrect, and + providing a hash allows `builtins.path` to be used even when the + `pure-eval` nix config option is on. + )", + .fun = prim_path, +}); + /************************************************************* * Sets @@ -1290,6 +1802,16 @@ static void prim_attrNames(EvalState & state, const Pos & pos, Value * * args, V [](Value * v1, Value * v2) { return strcmp(v1->string.s, v2->string.s) < 0; }); } +static RegisterPrimOp primop_attrNames({ + .name = "__attrNames", + .args = {"set"}, + .doc = R"( + Return the names of the attributes in the set *set* in an + alphabetically sorted list. For instance, `builtins.attrNames { y + = 1; x = "foo"; }` evaluates to `[ "x" "y" ]`. + )", + .fun = prim_attrNames, +}); /* Return the values of the attributes in a set as a list, in the same order as attrNames. */ @@ -1310,6 +1832,15 @@ static void prim_attrValues(EvalState & state, const Pos & pos, Value * * args, v.listElems()[i] = ((Attr *) v.listElems()[i])->value; } +static RegisterPrimOp primop_attrValues({ + .name = "__attrValues", + .args = {"set"}, + .doc = R"( + Return the values of the attributes in the set *set* in the order + corresponding to the sorted attribute names. + )", + .fun = prim_attrValues, +}); /* Dynamic version of the `.' operator. */ void prim_getAttr(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1329,9 +1860,20 @@ void prim_getAttr(EvalState & state, const Pos & pos, Value * * args, Value & v) v = *i->value; } +static RegisterPrimOp primop_getAttr({ + .name = "__getAttr", + .args = {"s", "set"}, + .doc = R"( + `getAttr` returns the attribute named *s* from *set*. Evaluation + aborts if the attribute doesn’t exist. This is a dynamic version of + the `.` operator, since *s* is an expression rather than an + identifier. + )", + .fun = prim_getAttr, +}); /* Return position information of the specified attribute. */ -void prim_unsafeGetAttrPos(EvalState & state, const Pos & pos, Value * * args, Value & v) +static void prim_unsafeGetAttrPos(EvalState & state, const Pos & pos, Value * * args, Value & v) { string attr = state.forceStringNoCtx(*args[0], pos); state.forceAttrs(*args[1], pos); @@ -1342,7 +1884,6 @@ void prim_unsafeGetAttrPos(EvalState & state, const Pos & pos, Value * * args, V state.mkPos(v, i->pos); } - /* Dynamic version of the `?' operator. */ static void prim_hasAttr(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1351,6 +1892,16 @@ static void prim_hasAttr(EvalState & state, const Pos & pos, Value * * args, Val mkBool(v, args[1]->attrs->find(state.symbols.create(attr)) != args[1]->attrs->end()); } +static RegisterPrimOp primop_hasAttr({ + .name = "__hasAttr", + .args = {"s", "set"}, + .doc = R"( + `hasAttr` returns `true` if *set* has an attribute named *s*, and + `false` otherwise. This is a dynamic version of the `?` operator, + since *s* is an expression rather than an identifier. + )", + .fun = prim_hasAttr, +}); /* Determine whether the argument is a set. */ static void prim_isAttrs(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1359,6 +1910,14 @@ static void prim_isAttrs(EvalState & state, const Pos & pos, Value * * args, Val mkBool(v, args[0]->type == tAttrs); } +static RegisterPrimOp primop_isAttrs({ + .name = "__isAttrs", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a set, and `false` otherwise. + )", + .fun = prim_isAttrs, +}); static void prim_removeAttrs(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1382,6 +1941,21 @@ static void prim_removeAttrs(EvalState & state, const Pos & pos, Value * * args, } } +static RegisterPrimOp primop_removeAttrs({ + .name = "removeAttrs", + .args = {"set", "list"}, + .doc = R"( + Remove the attributes listed in *list* from *set*. The attributes + don’t have to exist in *set*. For instance, + + ```nix + removeAttrs { x = 1; y = 2; z = 3; } [ "a" "x" "z" ] + ``` + + evaluates to `{ y = 2; }`. + )", + .fun = prim_removeAttrs, +}); /* Builds a set from a list specifying (name, value) pairs. To be precise, a list [{name = "name1"; value = value1;} ... {name = @@ -1423,6 +1997,30 @@ static void prim_listToAttrs(EvalState & state, const Pos & pos, Value * * args, v.attrs->sort(); } +static RegisterPrimOp primop_listToAttrs({ + .name = "__listToAttrs", + .args = {"e"}, + .doc = R"( + Construct a set from a list specifying the names and values of each + attribute. Each element of the list should be a set consisting of a + string-valued attribute `name` specifying the name of the attribute, + and an attribute `value` specifying its value. Example: + + ```nix + builtins.listToAttrs + [ { name = "foo"; value = 123; } + { name = "bar"; value = 456; } + ] + ``` + + evaluates to + + ```nix + { foo = 123; bar = 456; } + ``` + )", + .fun = prim_listToAttrs, +}); /* Return the right-biased intersection of two sets as1 and as2, i.e. a set that contains every attribute from as2 that is also a @@ -1441,6 +2039,15 @@ static void prim_intersectAttrs(EvalState & state, const Pos & pos, Value * * ar } } +static RegisterPrimOp primop_intersectAttrs({ + .name = "__intersectAttrs", + .args = {"e1", "e2"}, + .doc = R"( + Return a set consisting of the attributes in the set *e2* that also + exist in the set *e1*. + )", + .fun = prim_intersectAttrs, +}); /* Collect each attribute named `attr' from a list of attribute sets. Sets that don't contain the named attribute are ignored. @@ -1470,7 +2077,6 @@ static void prim_catAttrs(EvalState & state, const Pos & pos, Value * * args, Va v.listElems()[n] = res[n]; } - /* Return a set containing the names of the formal arguments expected by the function `f'. The value of each attribute is a Boolean denoting whether the corresponding argument has a default value. For instance, @@ -1508,6 +2114,22 @@ static void prim_functionArgs(EvalState & state, const Pos & pos, Value * * args v.attrs->sort(); } +static RegisterPrimOp primop_functionArgs({ + .name = "__functionArgs", + .args = {"f"}, + .doc = R"( + Return a set containing the names of the formal arguments expected + by the function *f*. The value of each attribute is a Boolean + denoting whether the corresponding argument has a default value. For + instance, `functionArgs ({ x, y ? 123}: ...) = { x = false; y = + true; }`. + + "Formal argument" here refers to the attributes pattern-matched by + the function. Plain lambdas are not included, e.g. `functionArgs (x: + ...) = { }`. + )", + .fun = prim_functionArgs, +}); /* Apply a function to every element of an attribute set. */ static void prim_mapAttrs(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1526,7 +2148,6 @@ static void prim_mapAttrs(EvalState & state, const Pos & pos, Value * * args, Va } - /************************************************************* * Lists *************************************************************/ @@ -1539,6 +2160,14 @@ static void prim_isList(EvalState & state, const Pos & pos, Value * * args, Valu mkBool(v, args[0]->isList()); } +static RegisterPrimOp primop_isList({ + .name = "__isList", + .args = {"e"}, + .doc = R"( + Return `true` if *e* evaluates to a list, and `false` otherwise. + )", + .fun = prim_isList, +}); static void elemAt(EvalState & state, const Pos & pos, Value & list, int n, Value & v) { @@ -1552,13 +2181,21 @@ static void elemAt(EvalState & state, const Pos & pos, Value & list, int n, Valu v = *list.listElems()[n]; } - /* Return the n-1'th element of a list. */ static void prim_elemAt(EvalState & state, const Pos & pos, Value * * args, Value & v) { elemAt(state, pos, *args[0], state.forceInt(*args[1], pos), v); } +static RegisterPrimOp primop_elemAt({ + .name = "__elemAt", + .args = {"xs", "n"}, + .doc = R"( + Return element *n* from the list *xs*. Elements are counted starting + from 0. A fatal error occurs if the index is out of bounds. + )", + .fun = prim_elemAt, +}); /* Return the first element of a list. */ static void prim_head(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1566,6 +2203,16 @@ static void prim_head(EvalState & state, const Pos & pos, Value * * args, Value elemAt(state, pos, *args[0], 0, v); } +static RegisterPrimOp primop_head({ + .name = "__head", + .args = {"list"}, + .doc = R"( + Return the first element of a list; abort evaluation if the argument + isn’t a list or is an empty list. You can test whether a list is + empty by comparing it with `[]`. + )", + .fun = prim_head, +}); /* Return a list consisting of everything but the first element of a list. Warning: this function takes O(n) time, so you probably @@ -1584,6 +2231,21 @@ static void prim_tail(EvalState & state, const Pos & pos, Value * * args, Value v.listElems()[n] = args[0]->listElems()[n + 1]; } +static RegisterPrimOp primop_tail({ + .name = "__tail", + .args = {"list"}, + .doc = R"( + Return the second to last elements of a list; abort evaluation if + the argument isn’t a list or is an empty list. + + > **Warning** + > + > This function should generally be avoided since it's inefficient: + > unlike Haskell's `tail`, it takes O(n) time, so recursing over a + > list by repeatedly calling `tail` takes O(n^2) time. + )", + .fun = prim_tail, +}); /* Apply a function to every element of a list. */ static void prim_map(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1597,6 +2259,21 @@ static void prim_map(EvalState & state, const Pos & pos, Value * * args, Value & *args[0], *args[1]->listElems()[n]); } +static RegisterPrimOp primop_map({ + .name = "map", + .args = {"f", "list"}, + .doc = R"( + Apply the function *f* to each element in the list *list*. For + example, + + ```nix + map (x: "foo" + x) [ "bar" "bla" "abc" ] + ``` + + evaluates to `[ "foobar" "foobla" "fooabc" ]`. + )", + .fun = prim_map, +}); /* Filter a list using a predicate; that is, return a list containing every element from the list for which the predicate function @@ -1628,6 +2305,15 @@ static void prim_filter(EvalState & state, const Pos & pos, Value * * args, Valu } } +static RegisterPrimOp primop_filter({ + .name = "__filter", + .args = {"f", "list"}, + .doc = R"( + Return a list consisting of the elements of *list* for which the + function *f* returns `true`. + )", + .fun = prim_filter, +}); /* Return true if a list contains a given element. */ static void prim_elem(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1642,6 +2328,15 @@ static void prim_elem(EvalState & state, const Pos & pos, Value * * args, Value mkBool(v, res); } +static RegisterPrimOp primop_elem({ + .name = "__elem", + .args = {"x", "xs"}, + .doc = R"( + Return `true` if a value equal to *x* occurs in the list *xs*, and + `false` otherwise. + )", + .fun = prim_elem, +}); /* Concatenate a list of lists. */ static void prim_concatLists(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1650,6 +2345,14 @@ static void prim_concatLists(EvalState & state, const Pos & pos, Value * * args, state.concatLists(v, args[0]->listSize(), args[0]->listElems(), pos); } +static RegisterPrimOp primop_concatLists({ + .name = "__concatLists", + .args = {"lists"}, + .doc = R"( + Concatenate a list of lists into a single list. + )", + .fun = prim_concatLists, +}); /* Return the length of a list. This is an O(1) time operation. */ static void prim_length(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1658,6 +2361,14 @@ static void prim_length(EvalState & state, const Pos & pos, Value * * args, Valu mkInt(v, args[0]->listSize()); } +static RegisterPrimOp primop_length({ + .name = "__length", + .args = {"e"}, + .doc = R"( + Return the length of the list *e*. + )", + .fun = prim_length, +}); /* Reduce a list by applying a binary operator, from left to right. The operator is applied strictly. */ @@ -1682,6 +2393,18 @@ static void prim_foldlStrict(EvalState & state, const Pos & pos, Value * * args, } } +static RegisterPrimOp primop_foldlStrict({ + .name = "__foldl'", + .args = {"op", "nul", "list"}, + .doc = R"( + Reduce a list by applying a binary operator, from left to right, + e.g. `foldl’ op nul [x0 x1 x2 ...] = op (op (op nul x0) x1) x2) + ...`. The operator is applied strictly, i.e., its arguments are + evaluated first. For example, `foldl’ (x: y: x + y) 0 [1 2 3]` + evaluates to 6. + )", + .fun = prim_foldlStrict, +}); static void anyOrAll(bool any, EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1707,12 +2430,30 @@ static void prim_any(EvalState & state, const Pos & pos, Value * * args, Value & anyOrAll(true, state, pos, args, v); } +static RegisterPrimOp primop_any({ + .name = "__any", + .args = {"pred", "list"}, + .doc = R"( + Return `true` if the function *pred* returns `true` for at least one + element of *list*, and `false` otherwise. + )", + .fun = prim_any, +}); static void prim_all(EvalState & state, const Pos & pos, Value * * args, Value & v) { anyOrAll(false, state, pos, args, v); } +static RegisterPrimOp primop_all({ + .name = "__all", + .args = {"pred", "list"}, + .doc = R"( + Return `true` if the function *pred* returns `true` for all elements + of *list*, and `false` otherwise. + )", + .fun = prim_all, +}); static void prim_genList(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1733,6 +2474,21 @@ static void prim_genList(EvalState & state, const Pos & pos, Value * * args, Val } } +static RegisterPrimOp primop_genList({ + .name = "__genList", + .args = {"generator", "length"}, + .doc = R"( + Generate list of size *length*, with each element *i* equal to the + value returned by *generator* `i`. For example, + + ```nix + builtins.genList (x: x * x) 5 + ``` + + returns the list `[ 0 1 4 9 16 ]`. + )", + .fun = prim_genList, +}); static void prim_lessThan(EvalState & state, const Pos & pos, Value * * args, Value & v); @@ -1768,6 +2524,26 @@ static void prim_sort(EvalState & state, const Pos & pos, Value * * args, Value std::stable_sort(v.listElems(), v.listElems() + len, comparator); } +static RegisterPrimOp primop_sort({ + .name = "__sort", + .args = {"comparator", "list"}, + .doc = R"( + Return *list* in sorted order. It repeatedly calls the function + *comparator* with two elements. The comparator should return `true` + if the first element is less than the second, and `false` otherwise. + For example, + + ```nix + builtins.sort builtins.lessThan [ 483 249 526 147 42 77 ] + ``` + + produces the list `[ 42 77 147 249 483 526 ]`. + + This is a stable sort: it preserves the relative order of elements + deemed equal by the comparator. + )", + .fun = prim_sort, +}); static void prim_partition(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1851,6 +2627,14 @@ static void prim_add(EvalState & state, const Pos & pos, Value * * args, Value & mkInt(v, state.forceInt(*args[0], pos) + state.forceInt(*args[1], pos)); } +static RegisterPrimOp primop_add({ + .name = "__add", + .args = {"e1", "e2"}, + .doc = R"( + Return the sum of the numbers *e1* and *e2*. + )", + .fun = prim_add, +}); static void prim_sub(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1862,6 +2646,14 @@ static void prim_sub(EvalState & state, const Pos & pos, Value * * args, Value & mkInt(v, state.forceInt(*args[0], pos) - state.forceInt(*args[1], pos)); } +static RegisterPrimOp primop_sub({ + .name = "__sub", + .args = {"e1", "e2"}, + .doc = R"( + Return the difference between the numbers *e1* and *e2*. + )", + .fun = prim_sub, +}); static void prim_mul(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1873,6 +2665,14 @@ static void prim_mul(EvalState & state, const Pos & pos, Value * * args, Value & mkInt(v, state.forceInt(*args[0], pos) * state.forceInt(*args[1], pos)); } +static RegisterPrimOp primop_mul({ + .name = "__mul", + .args = {"e1", "e2"}, + .doc = R"( + Return the product of the numbers *e1* and *e2*. + )", + .fun = prim_mul, +}); static void prim_div(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1902,21 +2702,57 @@ static void prim_div(EvalState & state, const Pos & pos, Value * * args, Value & } } +static RegisterPrimOp primop_div({ + .name = "__div", + .args = {"e1", "e2"}, + .doc = R"( + Return the quotient of the numbers *e1* and *e2*. + )", + .fun = prim_div, +}); + static void prim_bitAnd(EvalState & state, const Pos & pos, Value * * args, Value & v) { mkInt(v, state.forceInt(*args[0], pos) & state.forceInt(*args[1], pos)); } +static RegisterPrimOp primop_bitAnd({ + .name = "__bitAnd", + .args = {"e1", "e2"}, + .doc = R"( + Return the bitwise AND of the integers *e1* and *e2*. + )", + .fun = prim_bitAnd, +}); + static void prim_bitOr(EvalState & state, const Pos & pos, Value * * args, Value & v) { mkInt(v, state.forceInt(*args[0], pos) | state.forceInt(*args[1], pos)); } +static RegisterPrimOp primop_bitOr({ + .name = "__bitOr", + .args = {"e1", "e2"}, + .doc = R"( + Return the bitwise OR of the integers *e1* and *e2*. + )", + .fun = prim_bitOr, +}); + static void prim_bitXor(EvalState & state, const Pos & pos, Value * * args, Value & v) { mkInt(v, state.forceInt(*args[0], pos) ^ state.forceInt(*args[1], pos)); } +static RegisterPrimOp primop_bitXor({ + .name = "__bitXor", + .args = {"e1", "e2"}, + .doc = R"( + Return the bitwise XOR of the integers *e1* and *e2*. + )", + .fun = prim_bitXor, +}); + static void prim_lessThan(EvalState & state, const Pos & pos, Value * * args, Value & v) { state.forceValue(*args[0], pos); @@ -1925,6 +2761,17 @@ static void prim_lessThan(EvalState & state, const Pos & pos, Value * * args, Va mkBool(v, comp(args[0], args[1])); } +static RegisterPrimOp primop_lessThan({ + .name = "__lessThan", + .args = {"e1", "e2"}, + .doc = R"( + Return `true` if the number *e1* is less than the number *e2*, and + `false` otherwise. Evaluation aborts if either *e1* or *e2* does not + evaluate to a number. + )", + .fun = prim_lessThan, +}); + /************************************************************* * String manipulation @@ -1941,6 +2788,29 @@ static void prim_toString(EvalState & state, const Pos & pos, Value * * args, Va mkString(v, s, context); } +static RegisterPrimOp primop_toString({ + .name = "toString", + .args = {"e"}, + .doc = R"( + Convert the expression *e* to a string. *e* can be: + + - A string (in which case the string is returned unmodified). + + - A path (e.g., `toString /foo/bar` yields `"/foo/bar"`. + + - A set containing `{ __toString = self: ...; }`. + + - An integer. + + - A list, in which case the string representations of its elements + are joined with spaces. + + - A Boolean (`false` yields `""`, `true` yields `"1"`). + + - `null`, which yields the empty string. + )", + .fun = prim_toString, +}); /* `substring start len str' returns the substring of `str' starting at character position `min(start, stringLength str)' inclusive and @@ -1962,6 +2832,25 @@ static void prim_substring(EvalState & state, const Pos & pos, Value * * args, V mkString(v, (unsigned int) start >= s.size() ? "" : string(s, start, len), context); } +static RegisterPrimOp primop_substring({ + .name = "__substring", + .args = {"start", "len", "s"}, + .doc = R"( + Return the substring of *s* from character position *start* + (zero-based) up to but not including *start + len*. If *start* is + greater than the length of the string, an empty string is returned, + and if *start + len* lies beyond the end of the string, only the + substring up to the end of the string is returned. *start* must be + non-negative. For example, + + ```nix + builtins.substring 0 3 "nixos" + ``` + + evaluates to `"nix"`. + )", + .fun = prim_substring, +}); static void prim_stringLength(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -1970,6 +2859,15 @@ static void prim_stringLength(EvalState & state, const Pos & pos, Value * * args mkInt(v, s.size()); } +static RegisterPrimOp primop_stringLength({ + .name = "__stringLength", + .args = {"e"}, + .doc = R"( + Return the length of the string *e*. If *e* is not a string, + evaluation is aborted. + )", + .fun = prim_stringLength, +}); /* Return the cryptographic hash of a string in base-16. */ static void prim_hashString(EvalState & state, const Pos & pos, Value * * args, Value & v) @@ -1988,6 +2886,16 @@ static void prim_hashString(EvalState & state, const Pos & pos, Value * * args, mkString(v, hashString(*ht, s).to_string(Base16, false), context); } +static RegisterPrimOp primop_hashString({ + .name = "__hashString", + .args = {"type", "s"}, + .doc = R"( + Return a base-16 representation of the cryptographic hash of string + *s*. The hash algorithm specified by *type* must be one of `"md5"`, + `"sha1"`, `"sha256"` or `"sha512"`. + )", + .fun = prim_hashString, +}); /* Match a regular expression against a string and return either ‘null’ or a list containing substring matches. */ @@ -2036,6 +2944,41 @@ void prim_match(EvalState & state, const Pos & pos, Value * * args, Value & v) } } +static RegisterPrimOp primop_match({ + .name = "__match", + .args = {"regex", "str"}, + .doc = R"s( + Returns a list if the [extended POSIX regular + expression](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04) + *regex* matches *str* precisely, otherwise returns `null`. Each item + in the list is a regex group. + + ```nix + builtins.match "ab" "abc" + ``` + + Evaluates to `null`. + + ```nix + builtins.match "abc" "abc" + ``` + + Evaluates to `[ ]`. + + ```nix + builtins.match "a(b)(c)" "abc" + ``` + + Evaluates to `[ "b" "c" ]`. + + ```nix + builtins.match "[[:space:]]+([[:upper:]]+)[[:space:]]+" " FOO " + ``` + + Evaluates to `[ "foo" ]`. + )s", + .fun = prim_match, +}); /* Split a string with a regular expression, and return a list of the non-matching parts interleaved by the lists of the matching groups. */ @@ -2109,8 +3052,44 @@ static void prim_split(EvalState & state, const Pos & pos, Value * * args, Value } } +static RegisterPrimOp primop_split({ + .name = "__split", + .args = {"regex", "str"}, + .doc = R"s( + Returns a list composed of non matched strings interleaved with the + lists of the [extended POSIX regular + expression](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04) + *regex* matches of *str*. Each item in the lists of matched + sequences is a regex group. + + ```nix + builtins.split "(a)b" "abc" + ``` + + Evaluates to `[ "" [ "a" ] "c" ]`. + + ```nix + builtins.split "([ac])" "abc" + ``` + + Evaluates to `[ "" [ "a" ] "b" [ "c" ] "" ]`. + + ```nix + builtins.split "(a)|(c)" "abc" + ``` + + Evaluates to `[ "" [ "a" null ] "b" [ null "c" ] "" ]`. -static void prim_concatStringSep(EvalState & state, const Pos & pos, Value * * args, Value & v) + ```nix + builtins.split "([[:upper:]]+)" " FOO " + ``` + + Evaluates to `[ " " [ "FOO" ] " " ]`. + )s", + .fun = prim_split, +}); + +static void prim_concatStringsSep(EvalState & state, const Pos & pos, Value * * args, Value & v) { PathSet context; @@ -2129,6 +3108,16 @@ static void prim_concatStringSep(EvalState & state, const Pos & pos, Value * * a mkString(v, res, context); } +static RegisterPrimOp primop_concatStringsSep({ + .name = "__concatStringsSep", + .args = {"separator", "list"}, + .doc = R"( + Concatenate a list of strings with a separator between each + element, e.g. `concatStringsSep "/" ["usr" "local" "bin"] == + "usr/local/bin"`. + )", + .fun = prim_concatStringsSep, +}); static void prim_replaceStrings(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -2188,6 +3177,22 @@ static void prim_replaceStrings(EvalState & state, const Pos & pos, Value * * ar mkString(v, res, context); } +static RegisterPrimOp primop_replaceStrings({ + .name = "__replaceStrings", + .args = {"from", "to", "s"}, + .doc = R"( + Given string *s*, replace every occurrence of the strings in *from* + with the corresponding string in *to*. For example, + + ```nix + builtins.replaceStrings ["oo" "a"] ["a" "i"] "foobar" + ``` + + evaluates to `"fabir"`. + )", + .fun = prim_replaceStrings, +}); + /************************************************************* * Versions @@ -2204,6 +3209,19 @@ static void prim_parseDrvName(EvalState & state, const Pos & pos, Value * * args v.attrs->sort(); } +static RegisterPrimOp primop_parseDrvName({ + .name = "__parseDrvName", + .args = {"s"}, + .doc = R"( + Split the string *s* into a package name and version. The package + name is everything up to but not including the first dash followed + by a digit, and the version is everything following that dash. The + result is returned in a set `{ name, version }`. Thus, + `builtins.parseDrvName "nix-0.12pre12876"` returns `{ name = + "nix"; version = "0.12pre12876"; }`. + )", + .fun = prim_parseDrvName, +}); static void prim_compareVersions(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -2212,6 +3230,18 @@ static void prim_compareVersions(EvalState & state, const Pos & pos, Value * * a mkInt(v, compareVersions(version1, version2)); } +static RegisterPrimOp primop_compareVersions({ + .name = "__compareVersions", + .args = {"s1", "s2"}, + .doc = R"( + Compare two strings representing versions and return `-1` if + version *s1* is older than version *s2*, `0` if they are the same, + and `1` if *s1* is newer than *s2*. The version comparison + algorithm is the same as the one used by [`nix-env + -u`](../command-ref/nix-env.md#operation---upgrade). + )", + .fun = prim_compareVersions, +}); static void prim_splitVersion(EvalState & state, const Pos & pos, Value * * args, Value & v) { @@ -2232,6 +3262,17 @@ static void prim_splitVersion(EvalState & state, const Pos & pos, Value * * args } } +static RegisterPrimOp primop_splitVersion({ + .name = "__splitVersion", + .args = {"s"}, + .doc = R"( + Split a string representing a version into its components, by the + same version splitting logic underlying the version comparison in + [`nix-env -u`](../command-ref/nix-env.md#operation---upgrade). + )", + .fun = prim_splitVersion, +}); + /************************************************************* * Primop registration @@ -2282,15 +3323,6 @@ void EvalState::createBaseEnv() mkNull(v); addConstant("null", v); - auto vThrow = addPrimOp("throw", 1, prim_throw); - - auto addPurityError = [&](const std::string & name) { - Value * v2 = allocValue(); - mkString(*v2, fmt("'%s' is not allowed in pure evaluation mode", name)); - mkApp(v, *vThrow, *v2); - addConstant(name, v); - }; - if (!evalSettings.pureEval) { mkInt(v, time(0)); addConstant("__currentTime", v); @@ -2325,115 +3357,24 @@ void EvalState::createBaseEnv() addPrimOp("__importNative", 2, prim_importNative); addPrimOp("__exec", 1, prim_exec); } - addPrimOp("__typeOf", 1, prim_typeOf); - addPrimOp("isNull", 1, prim_isNull); - addPrimOp("__isFunction", 1, prim_isFunction); - addPrimOp("__isString", 1, prim_isString); - addPrimOp("__isInt", 1, prim_isInt); - addPrimOp("__isFloat", 1, prim_isFloat); - addPrimOp("__isBool", 1, prim_isBool); - addPrimOp("__isPath", 1, prim_isPath); addPrimOp("__genericClosure", 1, prim_genericClosure); addPrimOp("__addErrorContext", 2, prim_addErrorContext); - addPrimOp("__tryEval", 1, prim_tryEval); - addPrimOp("__getEnv", 1, prim_getEnv); - - // Strictness - addPrimOp("__seq", 2, prim_seq); - addPrimOp("__deepSeq", 2, prim_deepSeq); - - // Debugging - addPrimOp("__trace", 2, prim_trace); // Paths - addPrimOp("__toPath", 1, prim_toPath); - if (evalSettings.pureEval) - addPurityError("__storePath"); - else - addPrimOp("__storePath", 1, prim_storePath); - addPrimOp("__pathExists", 1, prim_pathExists); - addPrimOp("baseNameOf", 1, prim_baseNameOf); - addPrimOp("dirOf", 1, prim_dirOf); - addPrimOp("__readFile", 1, prim_readFile); - addPrimOp("__readDir", 1, prim_readDir); + addPrimOp("__storePath", 1, prim_storePath); addPrimOp("__findFile", 2, prim_findFile); - addPrimOp("__hashFile", 2, prim_hashFile); - - // Creating files - addPrimOp("__toXML", 1, prim_toXML); - addPrimOp("__toJSON", 1, prim_toJSON); - addPrimOp("__fromJSON", 1, prim_fromJSON); - addPrimOp("__toFile", 2, prim_toFile); - addPrimOp("__filterSource", 2, prim_filterSource); - addPrimOp("__path", 1, prim_path); // Sets - addPrimOp("__attrNames", 1, prim_attrNames); - addPrimOp("__attrValues", 1, prim_attrValues); - addPrimOp("__getAttr", 2, prim_getAttr); addPrimOp("__unsafeGetAttrPos", 2, prim_unsafeGetAttrPos); - addPrimOp("__hasAttr", 2, prim_hasAttr); - addPrimOp("__isAttrs", 1, prim_isAttrs); - addPrimOp("removeAttrs", 2, prim_removeAttrs); - addPrimOp("__listToAttrs", 1, prim_listToAttrs); - addPrimOp("__intersectAttrs", 2, prim_intersectAttrs); addPrimOp("__catAttrs", 2, prim_catAttrs); - addPrimOp("__functionArgs", 1, prim_functionArgs); addPrimOp("__mapAttrs", 2, prim_mapAttrs); // Lists - addPrimOp("__isList", 1, prim_isList); - addPrimOp("__elemAt", 2, prim_elemAt); - addPrimOp("__head", 1, prim_head); - addPrimOp("__tail", 1, prim_tail); - addPrimOp("map", 2, prim_map); - addPrimOp("__filter", 2, prim_filter); - addPrimOp("__elem", 2, prim_elem); - addPrimOp("__concatLists", 1, prim_concatLists); - addPrimOp("__length", 1, prim_length); - addPrimOp("__foldl'", 3, prim_foldlStrict); - addPrimOp("__any", 2, prim_any); - addPrimOp("__all", 2, prim_all); - addPrimOp("__genList", 2, prim_genList); - addPrimOp("__sort", 2, prim_sort); addPrimOp("__partition", 2, prim_partition); addPrimOp("__concatMap", 2, prim_concatMap); - // Integer arithmetic - addPrimOp("__add", 2, prim_add); - addPrimOp("__sub", 2, prim_sub); - addPrimOp("__mul", 2, prim_mul); - addPrimOp("__div", 2, prim_div); - addPrimOp("__bitAnd", 2, prim_bitAnd); - addPrimOp("__bitOr", 2, prim_bitOr); - addPrimOp("__bitXor", 2, prim_bitXor); - addPrimOp("__lessThan", 2, prim_lessThan); - - // String manipulation - addPrimOp("toString", 1, prim_toString); - addPrimOp("__substring", 3, prim_substring); - addPrimOp("__stringLength", 1, prim_stringLength); - addPrimOp("__hashString", 2, prim_hashString); - addPrimOp("__match", 2, prim_match); - addPrimOp("__split", 2, prim_split); - addPrimOp("__concatStringsSep", 2, prim_concatStringSep); - addPrimOp("__replaceStrings", 3, prim_replaceStrings); - - // Versions - addPrimOp("__parseDrvName", 1, prim_parseDrvName); - addPrimOp("__compareVersions", 2, prim_compareVersions); - addPrimOp("__splitVersion", 1, prim_splitVersion); - // Derivations addPrimOp("derivationStrict", 1, prim_derivationStrict); - addPrimOp("placeholder", 1, prim_placeholder); - - /* Add a wrapper around the derivation primop that computes the - `drvPath' and `outPath' attributes lazily. */ - string path = canonPath(settings.nixDataDir + "/nix/corepkgs/derivation.nix", true); - sDerivationNix = symbols.create(path); - evalFile(path, v); - addConstant("derivation", v); /* Add a value containing the current Nix expression search path. */ mkList(v, searchPath.size()); @@ -2458,6 +3399,13 @@ void EvalState::createBaseEnv() .doc = primOp.doc, }); + /* Add a wrapper around the derivation primop that computes the + `drvPath' and `outPath' attributes lazily. */ + string path = canonPath(settings.nixDataDir + "/nix/corepkgs/derivation.nix", true); + sDerivationNix = symbols.create(path); + evalFile(path, v); + addConstant("derivation", v); + /* Now that we've added all primops, sort the `builtins' set, because attribute lookups expect it to be sorted. */ baseEnv.values[0]->attrs->sort(); diff --git a/src/libexpr/primops/fetchTree.cc b/src/libexpr/primops/fetchTree.cc index 0dbf4ae1d..06e8304b8 100644 --- a/src/libexpr/primops/fetchTree.cc +++ b/src/libexpr/primops/fetchTree.cc @@ -226,18 +226,187 @@ static void prim_fetchurl(EvalState & state, const Pos & pos, Value * * args, Va fetch(state, pos, args, v, "fetchurl", false, ""); } +static RegisterPrimOp primop_fetchurl({ + .name = "__fetchurl", + .args = {"url"}, + .doc = R"( + Download the specified URL and return the path of the downloaded + file. This function is not available if [restricted evaluation + mode](../command-ref/conf-file.md) is enabled. + )", + .fun = prim_fetchurl, +}); + static void prim_fetchTarball(EvalState & state, const Pos & pos, Value * * args, Value & v) { fetch(state, pos, args, v, "fetchTarball", true, "source"); } +static RegisterPrimOp primop_fetchTarball({ + .name = "fetchTarball", + .args = {"args"}, + .doc = R"( + Download the specified URL, unpack it and return the path of the + unpacked tree. The file must be a tape archive (`.tar`) compressed + with `gzip`, `bzip2` or `xz`. The top-level path component of the + files in the tarball is removed, so it is best if the tarball + contains a single directory at top level. The typical use of the + function is to obtain external Nix expression dependencies, such as + a particular version of Nixpkgs, e.g. + + ```nix + with import (fetchTarball https://github.com/NixOS/nixpkgs/archive/nixos-14.12.tar.gz) {}; + + stdenv.mkDerivation { … } + ``` + + The fetched tarball is cached for a certain amount of time (1 hour + by default) in `~/.cache/nix/tarballs/`. You can change the cache + timeout either on the command line with `--option tarball-ttl number + of seconds` or in the Nix configuration file with this option: ` + number of seconds to cache `. + + Note that when obtaining the hash with ` nix-prefetch-url ` the + option `--unpack` is required. + + This function can also verify the contents against a hash. In that + case, the function takes a set instead of a URL. The set requires + the attribute `url` and the attribute `sha256`, e.g. + + ```nix + with import (fetchTarball { + url = "https://github.com/NixOS/nixpkgs/archive/nixos-14.12.tar.gz"; + sha256 = "1jppksrfvbk5ypiqdz4cddxdl8z6zyzdb2srq8fcffr327ld5jj2"; + }) {}; + + stdenv.mkDerivation { … } + ``` + + This function is not available if [restricted evaluation + mode](../command-ref/conf-file.md) is enabled. + )", + .fun = prim_fetchTarball, +}); + static void prim_fetchGit(EvalState &state, const Pos &pos, Value **args, Value &v) { fetchTree(state, pos, args, v, "git", true); } -static RegisterPrimOp r2("__fetchurl", 1, prim_fetchurl); -static RegisterPrimOp r3("fetchTarball", 1, prim_fetchTarball); -static RegisterPrimOp r4("fetchGit", 1, prim_fetchGit); +static RegisterPrimOp primop_fetchGit({ + .name = "fetchGit", + .args = {"args"}, + .doc = R"( + Fetch a path from git. *args* can be a URL, in which case the HEAD + of the repo at that URL is fetched. Otherwise, it can be an + attribute with the following attributes (all except `url` optional): + + - url + The URL of the repo. + + - name + The name of the directory the repo should be exported to in the + store. Defaults to the basename of the URL. + + - rev + The git revision to fetch. Defaults to the tip of `ref`. + + - ref + The git ref to look for the requested revision under. This is + often a branch or tag name. Defaults to `HEAD`. + + By default, the `ref` value is prefixed with `refs/heads/`. As + of Nix 2.3.0 Nix will not prefix `refs/heads/` if `ref` starts + with `refs/`. + + - submodules + A Boolean parameter that specifies whether submodules should be + checked out. Defaults to `false`. + + Here are some examples of how to use `fetchGit`. + + - To fetch a private repository over SSH: + + ```nix + builtins.fetchGit { + url = "git@github.com:my-secret/repository.git"; + ref = "master"; + rev = "adab8b916a45068c044658c4158d81878f9ed1c3"; + } + ``` + + - To fetch an arbitrary reference: + + ```nix + builtins.fetchGit { + url = "https://github.com/NixOS/nix.git"; + ref = "refs/heads/0.5-release"; + } + ``` + + - If the revision you're looking for is in the default branch of + the git repository you don't strictly need to specify the branch + name in the `ref` attribute. + + However, if the revision you're looking for is in a future + branch for the non-default branch you will need to specify the + the `ref` attribute as well. + + ```nix + builtins.fetchGit { + url = "https://github.com/nixos/nix.git"; + rev = "841fcbd04755c7a2865c51c1e2d3b045976b7452"; + ref = "1.11-maintenance"; + } + ``` + + > **Note** + > + > It is nice to always specify the branch which a revision + > belongs to. Without the branch being specified, the fetcher + > might fail if the default branch changes. Additionally, it can + > be confusing to try a commit from a non-default branch and see + > the fetch fail. If the branch is specified the fault is much + > more obvious. + + - If the revision you're looking for is in the default branch of + the git repository you may omit the `ref` attribute. + + ```nix + builtins.fetchGit { + url = "https://github.com/nixos/nix.git"; + rev = "841fcbd04755c7a2865c51c1e2d3b045976b7452"; + } + ``` + + - To fetch a specific tag: + + ```nix + builtins.fetchGit { + url = "https://github.com/nixos/nix.git"; + ref = "refs/tags/1.9"; + } + ``` + + - To fetch the latest version of a remote branch: + + ```nix + builtins.fetchGit { + url = "ssh://git@github.com/nixos/nix.git"; + ref = "master"; + } + ``` + + > **Note** + > + > Nix will refetch the branch in accordance with + > the option `tarball-ttl`. + + > **Note** + > + > This behavior is disabled in *Pure evaluation mode*. + )", + .fun = prim_fetchGit, +}); } |