aboutsummaryrefslogtreecommitdiff
path: root/src/libexpr/primops.cc
AgeCommit message (Collapse)Author
2024-08-22Revert "libexpr: Replace regex engine with boost::regex"jade
This reverts commit 447212fa65a80180150b265411924cc638a2c52c. Reason for revert: Regression in eval behaviour bug-compatibility. Expected behaviour (Nix 2.18.5, macOS and Linux [libstdc++/libc++]): ``` nix-repl> builtins.match "\\.*(.*)" ".keep" [ "keep" ] nix-repl> builtins.match "(\\.*)(.*)" ".keep" [ "." "keep" ] ``` Actual behaviour (boost::regex): ``` nix-repl> builtins.match "\\.*(.*)" ".keep" [ ".keep" ] nix-repl> builtins.match "(\\.*)(.*)" ".keep" [ "." "keep" ] ``` Bug: https://git.lix.systems/lix-project/lix/issues/483 Change-Id: Id462eb8586dcd54856cf095f09b3e3a216955b60
2024-08-22libexpr: Replace regex engine with boost::regexsugar
This avoids C++'s standard library regexes, which aren't the same across platforms, and have many other issues, like using stack so much that they stack overflow when processing a lot of data. To avoid backwards and forward compatibility issues, regexes are processed using a function converting libstdc++ regexes into Boost regexes, escaping characters that Boost needs to have escaped, and rejecting features that Boost has and libstdc++ doesn't. Related context: - Original failed attempt to use `boost::regex` in CppNix, failed due to boost icu dependency being large (disabling ICU is no longer necessary because linking ICU requires using a different header file, `boost/regex/icu.hpp`): https://github.com/NixOS/nix/pull/3826 - An attempt to use PCRE, rejected due to providing less backwards compatibility with `std::regex` than `boost::regex`: https://github.com/NixOS/nix/pull/7336 - Second attempt to use `boost::regex`, failed due to `}` regex failing to compile (dealt with by writing a wrapper that parses a regular expression and escapes `}` characters): https://github.com/NixOS/nix/pull/7762 Closes #34. Closes #476. Change-Id: Ieb0eb9e270a93e4c7eed412ba4f9f96cb00a5fa4
2024-08-08tree-wide: fix a pile of lintsJade Lovelace
This: - Converts a bunch of C style casts into C++ casts. - Removes some very silly pointer subtraction code (which is no more or less busted on i686 than it began) - Fixes some "technically UB" that never had to be UB in the first place. - Makes finally follow the noexcept status of the inner function. Maybe in the future we should ban the function from not being noexcept, but that is not today. - Makes various locally-used exceptions inherit from std::exception. Change-Id: I22e66972602604989b5e494fd940b93e0e6e9297
2024-08-08refactor: make HashType and Base enum classes for type safetyJade Lovelace
Change-Id: I9fbd55a9d50464a56fe11cb42a06a206914150d8
2024-08-03fix: warn and document when advanced attributes will have no impact due to ↵Tom Bereknyei
__structuredAttrs Backport of https://github.com/NixOS/nix/pull/10884. Change-Id: I82cc2794730ae9f4a9b7df0185ed0aea83efb65a
2024-07-20libexpr: refactor gc-agnostic helpers into one placeQyriad
Change-Id: Icc4b367e4f670d47256f62a3a002cd248a5c2d3b
2024-07-18expr: fix a compiler warning about different signs in comparisonJade Lovelace
We know that variable is >=0, so we can just cast it to unsigned. Change-Id: I3792eeb3ca43e6a507cc44c1a70584d42b2acd7b
2024-07-13language: cleanly ban integer overflowsJade Lovelace
This also bans various sneaking of negative numbers from the language into unsuspecting builtins as was exposed while auditing the consequences of changing the Nix language integer type to a newtype. It's unlikely that this change comprehensively ensures correctness when passing integers out of the Nix language and we should probably add a checked-narrowing function or something similar, but that's out of scope for the immediate change. During the development of this I found a few fun facts about the language: - You could overflow integers by converting from unsigned JSON values. - You could overflow unsigned integers by converting negative numbers into them when going into Nix config, into fetchTree, and into flake inputs. The flake inputs and Nix config cannot actually be tested properly since they both ban thunks, however, we put in checks anyway because it's possible these could somehow be used to do such shenanigans some other way. Note that Lix has banned Nix language integer overflows since the very first public beta, but threw a SIGILL about them because we run with -fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in production builds. Since the Nix language uses signed integers, overflow was simply undefined behaviour, and since we defined that to trap, it did. Trapping on it was a bad UX, but we didn't even entirely notice that we had done this at all until it was reported as a bug a couple of months later (which is, to be fair, that flag working as intended), and it's got enough production time that, aside from code that is IMHO buggy (and which is, in any case, not in nixpkgs) such as https://git.lix.systems/lix-project/lix/issues/445, we don't think anyone doing anything reasonable actually depends on wrapping overflow. Even for weird use cases such as doing funny bit crimes, it doesn't make sense IMO to have wrapping behaviour, since two's complement arithmetic overflow behaviour is so *aggressively* not what you want for *any* kind of mathematics/algorithms. The Nix language exists for package management, a domain where bit crimes are already only dubiously in scope to begin with, and it makes a lot more sense for that domain for the integers to never lose precision, either by throwing errors if they would, or by being arbitrary-precision. This change will be ported to CppNix as well, to maintain language consistency. Fixes: https://git.lix.systems/lix-project/lix/issues/423 Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
2024-07-12Use std::strong_ordering for version comparisonJade Lovelace
The actual motive here is the avoidance of integer overflow if we were to make these use checked NixInts and retain the subtraction. However, the actual *intent* of this code is a three-way comparison, which can be done with operator<=>, so we should just do *that* instead. Change-Id: I7f9a7da1f3176424b528af6d1b4f1591e4ab26bf
2024-07-04mildly cleanup libexpr/eval.hhQyriad
Change-Id: I40d01a8f8b7fb101279c6f88ebdf1f0969d9d7f0
2024-06-18libexpr: fix accessing uninitialized values and fix pure-eval docsJade Lovelace
We got UBSan working on Lix, so we of course immediately found a bug and some definitely nonsense behaviour. Accessing `pureEval` or `restrictEval` from a default setting value is nonsense, since they would never be actually set by the time that value is set so they are not going to do anything. The configuration is not applied in an initializer (and even if it were, it's not going to be in the right order). After looking into *that*, we hunted down what actually was applying these, since clearly this code did not do anything. The EvalState constructor should have a "search path added and removed here :)" sign on it, because that's where it is done. We added an explicit initialization of the optional in there because it was otherwise unclear why pureEval also has the search path to allowed paths setup code run. We then realized that the `pureEval` documentation was *also* bogus, and we rewrote it. In so doing, we realized that we forgot to file a bug to make `builtins.storePath` work in pure eval mode, so we filed one of those: https://git.lix.systems/lix-project/lix/issues/402 Yaks have been thoroughly shorn. UBSan report: ../src/libexpr/eval-settings.cc:66:10: runtime error: member call on address 0x752fa9a13060 which does not point to an object of type 'nix::BaseSetting<b ool>' 0x752fa9a13060: note: object has invalid vptr 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ^~~~~~~~~~~~~~~~~~~~~~~ invalid vptr 0 0x752fa95106a6 in nix::EvalSettings::getDefaultNixPath[abi:cxx11]() /home/jade/lix/lix2/build/src/libexpr/eval-settings.cc:66:10 1 0x752fa950e420 in nix::EvalSettings::EvalSettings() /home/jade/lix/lix2/build/src/libexpr/eval-settings.hh:36:15 2 0x752fa9469f1f in __cxx_global_var_init.50 /home/jade/lix/lix2/build/src/libexpr/eval-settings.cc:98:14 3 0x752fa9469f1f in _GLOBAL__sub_I_eval_settings.cc /home/jade/lix/lix2/build/src/libexpr/eval-settings.cc 4 0x752fabbd308d in call_init (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2+0x508d) (BuildId: a5b8228edc9f16078ac3c894af964eeb990ecb4c) 5 0x752fabbd317b in _dl_init (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2+0x517b) (BuildId: a5b8228edc9f16078ac3c894af964eeb990ecb4c) 6 0x752fabbe9c2f in _dl_start_user (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2+0x1bc2f) (BuildId: a5b8228edc9f16078ac3c894af964eeb990ecb4c) Change-Id: I5d8ffb7bfbe24b6584020ac74eed93d9f2e6d111
2024-06-17libexpr: add expr memory managementeldritch horrors
with the prepatory work done this mostly means turning plain pointers into unique_ptrs, with all the associated churn that necessitates. we might want to change some of these to box_ptrs at some point as well, but that would be a semantic change that isn't fully appropriate yet. Change-Id: I0c238c118617420650432f4ed45569baa3e3f413
2024-06-17libexpr: pass Exprs as references, not pointerseldritch horrors
almost all places where Exprs are passed as pointers expect the pointers to be non-null. pass them as references to encode this constraint in the type system as well (and also communicate that Exprs must not be freed). Change-Id: Ia98f166fec3c23151f906e13acb4a0954a5980a2
2024-06-16Change error messages about 'invalid paths' to 'path does not exist'.julia
Fixes #270. Change-Id: I07d2da41498cfdf324a03af40533044d58c97c7e
2024-05-29Remove 100s of CPU time (10%) from build times (1465s -> 1302s)Jade Lovelace
I saw that boost/lexical_cast was costing about 100s in CPU time on our compiles. We can fix this trivially by doing explicit template instantiation in exactly one place and eliminating all other includes of it, which is a code improvement anyway by hiding the boost. Before: ``` lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin Analyzing build trace from 'buildtimeold.bin'... **** Time summary: Compilation (551 times): Parsing (frontend): 1465.3 s Codegen & opts (backend): 1110.9 s <snip> **** Expensive headers: 178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via: 40x: command.hh 5x: command-installable-value.hh 3x: installable-flake.hh 2x: <direct include> 2x: installable-attr-path.hh 176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via: 36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh 12x: globals.hh config.hh experimental-features.hh 11x: file-system.hh file-descriptor.hh 6x: serialise.hh strings.hh 6x: <direct include> 6x: archive.hh serialise.hh strings.hh ... 173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via: 55x: <direct include> 39x: command.hh installable-value.hh installables.hh 7x: libexpr.hh 4x: local-store.hh 4x: command-installable-value.hh installable-value.hh installables.hh 3x: binary-cache-store.hh ... 170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via: 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh 14x: store-api.hh nar-info.hh hash.hh 11x: <direct include> 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh 7x: libexpr.hh value.hh source-path.hh archive.hh 6x: fetchers.hh hash.hh ... 169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via: 40x: command.hh installable-value.hh 5x: command-installable-value.hh installable-value.hh 3x: installable-flake.hh installable-value.hh 2x: <direct include> 1x: installable-derived-path.hh 1x: installable-value.hh ... 159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via: 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh 19x: <direct include> 14x: store-api.hh nar-info.hh hash.hh serialise.hh 11x: serialise.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh 7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh ... 156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via: 42x: <direct include> 7x: command-installable-value.hh 2x: installable-attr-path.hh 150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh 11x: file-system.hh 10x: globals.hh 6x: fetchers.hh 6x: serialise.hh strings.hh error.hh 5x: archive.hh ... 133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via : 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh 19x: file-system.hh 11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh 7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh 6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh ... 132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh 16x: gtest.h 11x: file-system.hh types.hh ref.hh 10x: globals.hh types.hh ref.hh 10x: json.hpp 6x: serialise.hh ... done in 0.6s. ``` After: ``` lix/lix2 » maintainers/buildtime_report.sh build Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'... done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it. Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'... **** Time summary: Compilation (551 times): Parsing (frontend): 1302.1 s Codegen & opts (backend): 956.3 s <snip> **** Expensive headers: 178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via: 36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh 12x: globals.hh config.hh experimental-features.hh 11x: file-system.hh file-descriptor.hh 6x: <direct include> 6x: serialise.hh strings.hh 6x: fetchers.hh hash.hh serialise.hh strings.hh ... 154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via: 40x: command.hh 5x: command-installable-value.hh 3x: installable-flake.hh 2x: <direct include> 2x: installable-attr-path.hh 153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via: 55x: <direct include> 39x: command.hh installable-value.hh installables.hh 7x: libexpr.hh 4x: local-store.hh 4x: command-installable-value.hh installable-value.hh installables.hh 3x: binary-cache-store.hh ... 149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh 11x: file-system.hh 10x: globals.hh 6x: fetchers.hh 6x: serialise.hh strings.hh error.hh 5x: archive.hh ... 144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via: 40x: command.hh installable-value.hh 5x: command-installable-value.hh installable-value.hh 3x: installable-flake.hh installable-value.hh 2x: <direct include> 1x: installable-value.hh 1x: installable-derived-path.hh ... 136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via: 42x: <direct include> 7x: command-installable-value.hh 2x: installable-attr-path.hh 133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh 16x: gtest.h 11x: file-system.hh types.hh ref.hh 10x: globals.hh types.hh ref.hh 10x: json.hpp 6x: serialise.hh ... 89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via: 37x: command.hh installable-value.hh installables.hh 25x: store-api.hh realisation.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh 6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh 6x: libexpr.hh value.hh context.hh 6x: shared.hh ... 87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via: 35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h 12x: regex sstream istream 10x: file-system.hh types.hh ref.hh memory unique_ptr.h 10x: gtest.h memory unique_ptr.h 10x: globals.hh types.hh ref.hh memory unique_ptr.h 6x: fetchers.hh types.hh ref.hh memory unique_ptr.h ... 85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via: 37x: command.hh installable-value.hh installables.hh derived-path.hh 20x: globals.hh 20x: logging.hh 16x: store-api.hh logging.hh 6x: <direct include> 6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh ... done in 0.5s. ``` Change-Id: I27f0a2d566db17832cd9be935f12efe7f95b92d0
2024-05-29util.{hh,cc}: Split out processes.{hh,cc}Tom Hubrecht
Change-Id: I39280dc40ca3f7f9007bc6c898ffcf760e2238b7
2024-05-24change "evaluating file" logs to debugQyriad
I can't imagine wanting this unless you are debugging something (in which case it's very useful) Change-Id: I90c6f182c18486e9f6b15a59379bbb8e88fb8e7f
2024-05-17Merge "primops: change to std::function, allowing the passing of user data" ↵Puck Meerburg
into main
2024-05-16primops: change to std::function, allowing the passing of user dataYorick van Pelt
(cherry picked from commit 48aa57549d514432d6621c1e29f051951eca2d7f) Change-Id: Ib7d5c6514031ceb6c42ac44588be6b0c1c3c225b
2024-05-15builtins: fix builtins.langVersion docs to state it's deprecatedJade Lovelace
This builtin was always a problem and nixpkgs uses it in exactly one place, to give up if the Nix version is absurdly old. It has no other use cases, and doesn't work in a multi-implementation world anyway. Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
2024-04-06avoid markdown which the repl's :doc cannot handlestuebinm
code blocks, if not surrounded by empty lines, have the language tags (in these cases, always `nix`) show up in the output of :doc. for example: nix-repl> :doc builtins.parseFlakeRef Synopsis: builtins.parseFlakeRef flake-ref Parse a flake reference, and return its exploded form. For example: nix builtins.parseFlakeRef "github:NixOS/nixpkgs/23.05?dir=lib" evaluates to: nix { dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; } is now instead: nix-repl> :doc builtins.parseFlakeRef Synopsis: builtins.parseFlakeRef flake-ref Parse a flake reference, and return its exploded form. For example: | builtins.parseFlakeRef "github:NixOS/nixpkgs/23.05?dir=lib" evaluates to: | { dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; } (closes #225) Change-Id: I0741aeb1006a5376bb2f663d202c7a4da7e38cce
2024-03-18use byte indexed locations for PosIdxeldritch horrors
we now keep not a table of all positions, but a table of all origins and their sizes. position indices are now direct pointers into the virtual concatenation of all parsed contents. this slightly reduces memory usage and time spent in the parser, at the cost of not being able to report positions if the total input size exceeds 4GiB. this limit is not unique to nix though, rustc and clang also limit their input to 4GiB (although at least clang refuses to process inputs that are larger, we will not). this new 4GiB limit probably will not cause any problems for quite a while, all of nixpkgs together is less than 100MiB in size and already needs over 700MiB of memory and multiple seconds just to parse. 4GiB worth of input will easily take multiple minutes and over 30GiB of memory without even evaluating anything. if problems *do* arise we can probably recover the old table-based system by adding some tracking to Pos::Origin (or increasing the size of PosIdx outright), but for time being this looks like more complexity than it's worth. since we now need to read the entire input again to determine the line/column of a position we'll make unsafeGetAttrPos slightly lazy: mostly the set it returns is only used to determine the file of origin of an attribute, not its exact location. the thunks do not add measurable runtime overhead. notably this change is necessary to allow changing the parser since apparently nothing supports nix's very idiosyncratic line ending choice of "anything goes", making it very hard to calculate line/column positions in the parser (while byte offsets are very easy). (cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda) Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
2024-03-17builtins.nixVersion: return fixed fake versionJade Lovelace
This builtin is only going to cause us problems because we are not Nix, so let's just falsify being in the 2.18 series, since that is the closest target that has any meaning. In future we might want to have a better feature detection mechanism, for when we actually add stuff to some builtin's attr set argument. But builtins.nixVersion is just going to be hopelessly broken and it should be stubbed out. Fixes https://git.lix.systems/lix-project/lix/issues/144 Change-Id: Id7390b32a29c6147f2977737d81846320de5d67e
2024-03-09Merge pull request #10066 from 9999years/print-all-frameseldritch horrors
Do not skip any stack frames when `--show-trace` is given (cherry picked from commit 0b47783d0a879875d558f0b56e49584f25ceb2d0) Change-Id: Ia0f18266dbcf97543110110c655c219c7a3e3270
2024-03-09Merge pull request #9914 from 9999years/debugger-on-traceeldritch horrors
Enter debugger on `builtins.trace` with an option (cherry picked from commit 774e7ca5847ebc392eac2a124a8f12b24da4f65a) Change-Id: If01e2110b3a128e639b05143227e365227d149f1
2024-03-09`:quit` in the debugger should quit the whole programeldritch horrors
(cherry picked from commit 2a8fe9a93837733e9dd9ed5c078734a35b203e14) Change-Id: I71dadfef6b24d9272b206e9e2c408040559d8a1c
2024-03-09Merge pull request #9925 from 9999years/fmt-cleanupeldritch horrors
Cleanup `fmt.hh` (cherry picked from commit 47a1dbb4b8e7913cbb9b4d604728b912e76e4ca0) Change-Id: Id076a45cb39652f437fe3f8bda10c310a9894777
2024-03-09Attach positions to errors in `derivationStrict`eldritch horrors
(cherry picked from commit 87dc4bc7d139a7eccb257e71558314a0d99e8d6a) Change-Id: Ib7509cbb1d246ca5aa3607ff860420fe7a754f6a
2024-03-09libexpr: Support structured error classeseldritch horrors
While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`. (cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732) Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
2024-03-09Merge pull request #9753 from 9999years/print-value-on-type-erroreldritch horrors
Print the value in `value is X while a Y is expected` error (cherry picked from commit 5f72a97092da6af28a7d2b2a50d74e9d34fae7e1) Change-Id: Idb4bc903ae59a0f5b6fb3b1da4d47970fe0a6efe
2024-03-07Merge pull request #9890 from obsidiansystems/remove-unneeded-toRealPatheldritch horrors
Change an `allowPath` call to take a store path again (cherry picked from commit 7c9ed1b1a325fe64a186e9d454607eaa0a7e8951) Change-Id: Ia5ec924315a1f2640a0438cfb4b1ee0689cd3558
2024-03-07Merge pull request #9985 from alois31/symlink-resolutioneldritch horrors
Restore `builtins.pathExists` behavior on broken symlinks (cherry picked from commit d53c8901ef7f2033855dd99063522e3d56a19dab) === note that this variant differs markedly from the source commit because we haven't endured quite as much lazy trees. Change-Id: I0facf282f21fe0db4134be5c65a8368c1b3a06fc
2024-03-05Merge pull request #9634 from 9999years/combine-abstract-pos-and-poseldritch horrors
Combine `AbstractPos`, `PosAdapter`, and `Pos` (cherry picked from commit 113499d16fc87d53b73fb62fe6242154909756ed) === this is a bit cursed because originally it was based on InputAccessor code that we don't have and moved/patched features we likewise don't have (fetchToStore caching, all the individual accessors, ContentAddressMethod). the commit is adjusted accordingly to match (remove caching, ignore accessors, use FileIngestionMethod). note that `state.rootPath . CanonPath == abs` and computeStorePathForPath works relative to cwd, so the slight rewrite in the moved fetchToStore is legal. Change-Id: I05fd340c273f0bcc8ffabfebdc4a88b98083bce5
2024-03-04Merge pull request #9747 from awakesecurity/mz/fix-quadratic-splitStringeldritch horrors
Fix performance of builtins.substring for empty substrings (cherry picked from commit b2deff1947c2fe57fdbf1a472eb9003eb407f8d3) Change-Id: I4ddfc8d26a4781c9520fff9807849a073ee7bed8
2024-03-04Merge pull request #9658 from pennae/env-dieteldritch horrors
reduce the size of Env by one pointer (cherry picked from commit 83f5622545a2fc31eb7e7d5105f64ed6dd3058b3) Change-Id: I5636290526d0165cfc61aee1e7a5b94db4a26cef
2024-03-04Merge pull request #9645 from tweag/undeprecate-isNulleldritch horrors
Undeprecate isNull (cherry picked from commit 8e64cd59b0fe1a4e1d3ba8d359ea0f9af2936f94) Change-Id: If36b85a3fc01ee700bcaf4d5d83a5884a4b5de92
2024-03-04Merge pull request #4093 from matthewbauer/eval-systemeldritch horrors
Add eval-system option (cherry picked from commit 071dbbee33af9f27338c3e53e4ea067dbfa14010) Change-Id: Ia81358c8cfb60241da07a4d0e84b9ee62a18a53f
2024-03-04Merge pull request #9430 from hercules-ci/remove-vlaseldritch horrors
Fix stack overflow in `filter` (cherry picked from commit cb7f25869daa2491eeac5fee49ad8f31b2218c15) Change-Id: Ib90f97a9805bbb4d0e2741551d490f054fc0a675
2024-03-04Merge pull request #9399 from edolstra/revert-vlaseldritch horrors
Revert use of boost::container::small_vector in the evaluator (cherry picked from commit 6832d18ac734f4b855f97c07b158491dd01cefcd) Change-Id: I7f10af0c8b8a8beb4b1e36424120995f4ed82738
2024-03-04Merge pull request #9395 from nbraud/buitlinseldritch horrors
builtins.concatMap: Fix typo in error message (cherry picked from commit 4292d997568eb30503e287f98e24821ff0bc2816) Change-Id: Ia33d1b02e41f699ef0c8c2d6487c9f70b2cc8cf4
2024-03-04Merge pull request #7348 from thufschmitt/dont-use-vlaseldritch horrors
Remove the usage of VLAs in the code (cherry picked from commit ac4431e9d016e62fb5dc9ae36833bd0c6cdadeec) Change-Id: Ifbf5fbfc2e27122362a2aaea4b62c7cf3ca46b1a
2024-03-04Fix `boost::bad_format_string` exception in `builtins.addErrorContext` (#9291)eldritch horrors
* Fix boost::bad_format_string exception in builtins.addErrorContext The message passed to addTrace was incorrectly being used as a format string and this this would cause an exception when the string contained a '%', which can be hit in places where arbitrary file paths are interpolated. * add test (cherry picked from commit 61d6fe059e959455e156c1d57bb91155d363e983) Change-Id: Idd671127a9c1ccc8b94e58e727632fcc064f3cbe
2024-01-11Merge branch '2.18-maintenance' into ifd-buildStore-2.18Shea Levy
2024-01-11Build IFD in the build store when using eval-store.Shea Levy
Previously, IFDs would be built within the eval store, even though one is typically using `--eval-store` precisely to *avoid* local builds. Because the resulting Nix expression must be copied back to the eval store in order to be imported, this requires the eval store to trust the build store's signatures. (cherry picked from commit c3942ef85ffbd83391410fbf012f1de366d2463c)
2023-10-01pathExists: isDir when endswith /.Robert Hensing
(cherry picked from commit f8a3893e8d77ce4a6e23719a0b2d88464cb84b9c)
2023-09-07Allow dynamic derivation deps in `inputDrvs`John Ericson
We use the same nested map representation we used for goals, again in order to save space. We might someday want to combine with `inputDrvs`, by doing `V = bool` instead of `V = std::set<OutputName>`, but we are not doing that yet for sake of a smaller diff. The ATerm format for Derivations also needs to be extended, in addition to the in-memory format. To accomodate this, we added a new basic versioning scheme, so old versions of Nix will get nice errors. (And going forward, if the ATerm format changes again the errors will be even better.) `parsedStrings`, an internal function used as part of parsing derivations in A-Term format, used to consume the final `]` but expect the initial `[` to already be consumed. This made for what looked like unbalanced brackets at callsites, which was confusing. Now it consumes both which is hopefully less confusing. As part of testing, we also created a unit test for the A-Term format for regular non-experimental derivations too. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Apply suggestions from code review Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-01Merge pull request #8869 from hercules-ci/fix-issue-8838-pathExists-isDirEelco Dolstra
Fix #8838, pathExists: isDir when ends with `/ `
2023-08-25pathExists: isDir when endswith /Robert Hensing
Fixes https://github.com/NixOS/nix/issues/8838
2023-08-25Introduce `OutputName` and `OutputNameView` type aliasesJohn Ericson
Hopefully they make the code easier to understand!
2023-08-18Fixing #7479John Ericson
Types converted: - `NixStringContextElem` - `OutputsSpec` - `ExtendedOutputsSpec` - `DerivationOutput` - `DerivationType` Existing ones mostly conforming the pattern cleaned up: - `ContentAddressMethod` - `ContentAddressWithReferences` The `DerivationGoal::derivationType` field had a bogus initialization, now caught, so I made it `std::optional`. I think #8829 can make it non-optional again because it will ensure we always have the derivation when we construct a `DerivationGoal`. See that issue (#7479) for details on the general goal. `git grep 'Raw::Raw'` indicates the two types I didn't yet convert `DerivedPath` and `BuiltPath` (and their `Single` variants) . This is because @roberth and I (can't find issue right now...) plan on reworking them somewhat, so I didn't want to churn them more just yet. Co-authored-by: Eelco Dolstra <edolstra@gmail.com>