aboutsummaryrefslogtreecommitdiff
path: root/src/libexpr/primops
AgeCommit message (Collapse)Author
2024-03-09Merge pull request #9925 from 9999years/fmt-cleanupeldritch horrors
Cleanup `fmt.hh` (cherry picked from commit 47a1dbb4b8e7913cbb9b4d604728b912e76e4ca0) Change-Id: Id076a45cb39652f437fe3f8bda10c310a9894777
2024-03-09libexpr: Support structured error classeseldritch horrors
While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`. (cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732) Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
2024-03-04Merge pull request #9216 from obsidiansystems/addDrvOutputDependencies-preeldritch horrors
Add `builtins.addDrvOutputDependencies` (cherry picked from commit a58d7f143ec995a45745c2176bfebcc3e011db58) Change-Id: Ia5a1790bf29dfaf29287cc35cdae6b6d650e7a83
2023-11-20fetchTree: clarify docs for shallow flagDavHau
(cherry picked from commit 796a7eb92d2b0caf75685126adc7460a4c39cfec)
2023-08-18Fixing #7479John Ericson
Types converted: - `NixStringContextElem` - `OutputsSpec` - `ExtendedOutputsSpec` - `DerivationOutput` - `DerivationType` Existing ones mostly conforming the pattern cleaned up: - `ContentAddressMethod` - `ContentAddressWithReferences` The `DerivationGoal::derivationType` field had a bogus initialization, now caught, so I made it `std::optional`. I think #8829 can make it non-optional again because it will ensure we always have the derivation when we construct a `DerivationGoal`. See that issue (#7479) for details on the general goal. `git grep 'Raw::Raw'` indicates the two types I didn't yet convert `DerivedPath` and `BuiltPath` (and their `Single` variants) . This is because @roberth and I (can't find issue right now...) plan on reworking them somewhat, so I didn't want to churn them more just yet. Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-08-10Make the Derived Path family of types inductive for dynamic derivationsJohn Ericson
We want to be able to write down `foo.drv^bar.drv^baz`: `foo.drv^bar.drv` is the dynamic derivation (since it is itself a derivation output, `bar.drv` from `foo.drv`). To that end, we create `Single{Derivation,BuiltPath}` types, that are very similar except instead of having multiple outputs (in a set or map), they have a single one. This is for everything to the left of the rightmost `^`. `NixStringContextElem` has an analogous change, and now can reuse `SingleDerivedPath` at the top level. In fact, if we ever get rid of `DrvDeep`, `NixStringContextElem` could be replaced with `SingleDerivedPath` entirely! Important note: some JSON formats have changed. We already can *produce* dynamic derivations, but we can't refer to them directly. Today, we can merely express building or example at the top imperatively over time by building `foo.drv^bar.drv`, and then with a second nix invocation doing `<result-from-first>^baz`, but this is not declarative. The ethos of Nix of being able to write down the full plan everything you want to do, and then execute than plan with a single command, and for that we need the new inductive form of these types. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-31Move evaluator settings (type and global) to separate file/headerJohn Ericson
2023-07-21Merge pull request #8650 from obsidiansystems/content-address-simplerEelco Dolstra
Simplify `ContentAddress`
2023-07-07Simplify `ContentAddress`John Ericson
Whereas `ContentAddressWithReferences` is a sum type complex because different varieties support different notions of reference, and `ContentAddressMethod` is a nested enum to support that, `ContentAddress` can be a simple pair of a method and hash. `ContentAddress` does not need to be a sum type on the outside because the choice of method doesn't effect what type of hashes we can use. Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-07-07fetchClosure: Interleave the examples in the docsRobert Hensing
2023-07-07fetchClosure: Apply suggestions from code reviewRobert Hensing
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-30fetchClosure: Docs and error message improvementsRobert Hensing
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-30doc: Improve `fetchClosure` documentationRobert Hensing
2023-06-30fetchClosure: Refactor: replace enableRewritingRobert Hensing
A single variable is nice and self-contained.
2023-06-30fetchClosure: Split into three casesRobert Hensing
2023-06-30makeContentAddressed: Add single path helperRobert Hensing
2023-06-30fetchClosure: Disallow toPath for inputAddressed = trueRobert Hensing
2023-06-30fetchClosure: Always check that inputAddressed matches the resultRobert Hensing
2023-06-30fetchClosure: Allow input addressed paths in pure modeRobert Hensing
When explicitly requested by the caller, as suggested in the meeting (https://github.com/NixOS/nix/pull/8090#issuecomment-1531139324) > @edolstra: { toPath } vs { fromPath } is too implicit I've opted for the `inputAddressed = true` requirement, because it we did not agree on renaming the path attributes. > @roberth: more explicit > @edolstra: except for the direction; not immediately clear in which direction the rewriting happens This is in fact the most explicit syntax and a bit redundant, which is good, because that redundancy lets us deliver an error message that reminds expression authors that CA provides a better experience to their users.
2023-06-30fetchClosure: Refactor: rename toCA -> enableRewritingRobert Hensing
2023-06-30fetchClosure: Factor out attribute hintRobert Hensing
2023-06-27Automatically document builtin constantsJohn Ericson
This is done in roughly the same way builtin functions are documented. Also auto-link experimental features for primops, subsuming PR #8371. Co-authored-by: Eelco Dolstra <edolstra@gmail.com> Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-24Merge pull request #5385 from Enzime/add/dirty-revThéophane Hufschmitt
Add `dirtyRev` and `dirtyShortRev` to `fetchGit`
2023-06-24Add `dirtyRev` and `dirtyShortRev` to `fetchGit`Michael Hoang
Fixes #4682
2023-06-16Merge pull request #8477 from edolstra/tarball-flake-redirectsEelco Dolstra
Tarball flake improvements
2023-06-13Document fromTOML, hasContext and getContext builtinsMichal Sojka
Until now, these functions were completely missing in the Nix manual. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-13Allow tarball URLs to redirect to a lockable immutable URLEelco Dolstra
Previously, for tarball flakes, we recorded the original URL of the tarball flake, rather than the URL to which it ultimately redirects. Thus, a flake URL like http://example.org/patchelf-latest.tar that redirects to http://example.org/patchelf-<revision>.tar was not really usable. We couldn't record the redirected URL, because sites like GitHub redirect to CDN URLs that we can't rely on to be stable. So now we use the redirected URL only if the server returns the `x-nix-is-immutable` or `x-amz-meta-nix-is-immutable` headers in its response.
2023-06-11Register all PrimOps via the Info structureMichal Sojka
This will allow documenting them (in later commits). Note that we keep the old constructor even if it is no longer used by Nix code, because it is used in tests/plugins/plugintest.cc, which suggests that it might be used by some external plugin.
2023-06-09Parse TOML timestamps (#8120)Andrea Bedini
Currently `fromTOML` throws an exception when encountering a timestamp since the Nix language lacks a way to represent them. This patch changes this beaviour and makes `fromTOML` parse timestamps as attrsets of the format { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; } This is guarded by an experimental feature flag to leave room for iterating on the representation.
2023-05-17add cross-references to pure evaluation modeValentin Gagarin
use consistent wording everywhere. add some details on the configuration option documentation.
2023-04-21Use `std::set<StringContextElem>` not `PathSet` for string contextsJohn Ericson
Motivation `PathSet` is not correct because string contexts have other forms (`Built` and `DrvDeep`) that are not rendered as plain store paths. Instead of wrongly using `PathSet`, or "stringly typed" using `StringSet`, use `std::std<StringContextElem>`. ----- In support of this change, `NixStringContext` is now defined as `std::std<StringContextElem>` not `std:vector<StringContextElem>`. The old definition was just used by a `getContext` method which was only used by the eval cache. It can be deleted altogether since the types are now unified and the preexisting `copyContext` function already suffices. Summarizing the previous paragraph: Old: - `value/context.hh`: `NixStringContext = std::vector<StringContextElem>` - `value.hh`: `NixStringContext Value::getContext(...)` - `value.hh`: `copyContext(...)` New: - `value/context.hh`: `NixStringContext = std::set<StringContextElem>` - `value.hh`: `copyContext(...)` ---- The string representation of string context elements no longer contains the store dir. The diff of `src/libexpr/tests/value/context.cc` should make clear what the new representation is, so we recommend reviewing that file first. This was done for two reasons: Less API churn: `Value::mkString` and friends did not take a `Store` before. But if `NixStringContextElem::{parse, to_string}` *do* take a store (as they did before), then we cannot have the `Value` functions use them (in order to work with the fully-structured `NixStringContext`) without adding that argument. That would have been a lot of churn of threading the store, and this diff is already large enough, so the easier and less invasive thing to do was simply make the element `parse` and `to_string` functions not take the `Store` reference, and the easiest way to do that was to simply drop the store dir. Space usage: Dropping the `/nix/store/` (or similar) from the internal representation will safe space in the heap of the Nix programming being interpreted. If the heap contains many strings with non-trivial contexts, the saving could add up to something significant. ---- The eval cache version is bumped. The eval cache serialization uses `NixStringContextElem::{parse, to_string}`, and since those functions are changed per the above, that means the on-disk representation is also changed. This is simply done by changing the name of the used for the eval cache from `eval-cache-v4` to eval-cache-v5`. ---- To avoid some duplication `EvalCache::mkPathString` is added to abstract over the simple case of turning a store path to a string with just that string in the context. Context This PR picks up where #7543 left off. That one introduced the fully structured `NixStringContextElem` data type, but kept `PathSet context` as an awkward middle ground between internal `char[][]` interpreter heap string contexts and `NixStringContext` fully parsed string contexts. The infelicity of `PathSet context` was specifically called out during Nix team group review, but it was agreeing that fixing it could be left as future work. This is that future work. A possible follow-up step would be to get rid of the `char[][]` evaluator heap representation, too, but it is not yet clear how to do that. To use `NixStringContextElem` there we would need to get the STL containers to GC pointers in the GC build, and I am not sure how to do that. ---- PR #7543 effectively is writing the inverse of a `mkPathString`, `mkOutputString`, and one more such function for the `DrvDeep` case. I would like that PR to have property tests ensuring it is actually the inverse as expected. This PR sets things up nicely so that reworking that PR to be in that more elegant and better tested way is possible. Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-03-30Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
Also improve content-address.hh API docs.
2023-03-27Merge pull request #7609 from obsidiansystems/hide-experimental-settingsJohn Ericson
Hide experimental settings
2023-03-21Fix rendering of fetchGit documentationEelco Dolstra
stripIndentation() doesn't support tabs, so the entire markdown ended up indented and thus rendered as a code block.
2023-03-20Move enabled experimental feature to libutil structJohn Ericson
This is needed in subsequent commits to allow the settings and CLI args infrastructure itself to read this setting.
2023-03-05remove indentation in `fetchGit` attribute listingValentin Gagarin
also reword a confusing sentence and add links to Git terminology
2023-02-28No inheritance for `TextInfo` and `FixedOutputInfo`John Ericson
2023-02-28Revert "Remove some designated initializers"John Ericson
This reverts commit ee9eb83a842eb97d0180fd9d349d30ff27fdb485.
2023-02-28Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-02-27fetchTree: convert fs path to url via ParsedURL::to_stringYorick van Pelt
2023-02-08Documentation: builtins.fetchGit when used on a local path (#7706)Joachim Breitner
* Documentation: builtins.fetchGit when used on a local path Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-02-01Remove some designated initializersJohn Ericson
With the switch to C++20, the rules became more strict, and we can no longer initialize base classes. Make them comments instead. (BTW https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html this offers some new syntax for this use-case. Hopefully this will be adopted and we can eventually use it.)
2023-01-23Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-01-23Merge pull request #7657 from obsidiansystems/fix-7655Robert Hensing
Fix #7655
2023-01-21Fix #7655John Ericson
We had some local variables left over from the older (more complicated) implementation of this function. They should all be unused, but one wasn't by mistake. Delete them all, and replace the one that was still in use as intended.
2023-01-19Add tests for error traces, and fixesGuillaume Maudoux
2023-01-19Revert "Revert "Merge pull request #6204 from layus/coerce-string""Guillaume Maudoux
This reverts commit 9b33ef3879a764bed4cc2404a08344c3a697a646.
2023-01-18Restore support for channel: URLs in fetchTarballEelco Dolstra
Fixes #7625.
2023-01-18Revert "Merge pull request #6204 from layus/coerce-string"Robert Hensing
This reverts commit a75b7ba30f1e4f8b15e810fd18e63ee9552e0815, reversing changes made to 9af16c5f742300e831a2cc400e43df1e22f87f31.
2023-01-14Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson