aboutsummaryrefslogtreecommitdiff
path: root/src/libexpr/parser-state.hh
AgeCommit message (Collapse)Author
2024-03-18use byte indexed locations for PosIdxeldritch horrors
we now keep not a table of all positions, but a table of all origins and their sizes. position indices are now direct pointers into the virtual concatenation of all parsed contents. this slightly reduces memory usage and time spent in the parser, at the cost of not being able to report positions if the total input size exceeds 4GiB. this limit is not unique to nix though, rustc and clang also limit their input to 4GiB (although at least clang refuses to process inputs that are larger, we will not). this new 4GiB limit probably will not cause any problems for quite a while, all of nixpkgs together is less than 100MiB in size and already needs over 700MiB of memory and multiple seconds just to parse. 4GiB worth of input will easily take multiple minutes and over 30GiB of memory without even evaluating anything. if problems *do* arise we can probably recover the old table-based system by adding some tracking to Pos::Origin (or increasing the size of PosIdx outright), but for time being this looks like more complexity than it's worth. since we now need to read the entire input again to determine the line/column of a position we'll make unsafeGetAttrPos slightly lazy: mostly the set it returns is only used to determine the file of origin of an attribute, not its exact location. the thunks do not add measurable runtime overhead. notably this change is necessary to allow changing the parser since apparently nothing supports nix's very idiosyncratic line ending choice of "anything goes", making it very hard to calculate line/column positions in the parser (while byte offsets are very easy). (cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda) Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
2024-03-16diagnose duplicated attrs at correct patheldritch horrors
diagnose attr duplication at the path the duplication was detected, not at the path the current attribute wanted to place. doing the latter is only correct if a leaf attribute was duplicated, not if an attrpath was set to a non-attrset in one binding and a (potentially implied) attrset in another binding. fixes #124 Change-Id: Ic4aa9cc12a9874d4e7897c6f64408f10aa36fc82
2024-03-10remove ExprAttrs::AttrDef::inheritedeldritch horrors
it's no longer widely used and has a rather confusing meaning now that inherit-from is handled very differently. (cherry picked from commit 1cd87b7042d14aae1fafa47b1c28db4c5bd20de7) Change-Id: I90bbebddf06762960d8ca4f621cf042ce8ae83f9
2024-03-10evaluate inherit (from) exprs only once per directiveeldritch horrors
desugaring inherit-from to syntactic duplication of the source expr also duplicates side effects of the source expr (such as trace calls) and expensive computations (such as derivationStrict). (cherry picked from commit cefd0302b55b3360dbca59cfcb4bf6a750d6cdcf) Change-Id: Iff519f991adef2e51683ba2c552d37a3df7a179e
2024-03-10preserve information about whether/how an attribute was inheritedeldritch horrors
(cherry picked from commit c66ee57edc6cac3571bfbf77d0c0ea4d25b4e805) Change-Id: Ie8606a8b2f5946c87dd4d16b7b46203e199a4cc1
2024-03-09Merge pull request #9925 from 9999years/fmt-cleanupeldritch horrors
Cleanup `fmt.hh` (cherry picked from commit 47a1dbb4b8e7913cbb9b4d604728b912e76e4ca0) Change-Id: Id076a45cb39652f437fe3f8bda10c310a9894777
2024-03-09libexpr: Support structured error classeseldritch horrors
While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`. (cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732) Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
2024-03-09Minor formatting tweakseldritch horrors
(cherry picked from commit 365b831e6f290c733da6879dae871dada343a1eb) Change-Id: Ife3d269d2f87d6e3fe8a348995019dfc08ac75eb
2024-03-09don't repeatedly look up ast internal symbolseldritch horrors
these symbols are used a *lot*, so it makes sense to cache them. this mostly increases clarity of the code (however clear one may wish to call the parser desugaring here), but it also provides a small performance benefit. (cherry picked from commit 09a1128d9e2ff0ae6176784938047350d6f8a782) Change-Id: I73d9f66be4555168e048cb2d542277251580c2d1
2024-03-09decouple parser and EvalStateeldritch horrors
there's no reason the parser itself should be doing semantic analysis like bindVars. split this bit apart (retaining the previous name in EvalState) and have the parser really do *only* parsing, decoupled from EvalState. (cherry picked from commit b596cc9e7960b9256bcd557334d81e9d555be5a2) Change-Id: I481a7623afc783e9d28a6eb4627552cf8a780986
2024-03-09rename ParserState::{makeCurPos -> at}eldritch horrors
most instances of this being used do not refer to the "current" position, sometimes not even to one reasonably close by. it could also be called `makePos` instead, but `at` seems clear in context. (cherry picked from commit 835a6c7bcfd0b22acc16f31de5fc7bb650d52017) Change-Id: I17cab8a6cc14cac5b64624431957bfcf04140809
2024-03-09move ParseData to own header, rename to ParserStateeldritch horrors
ParserState better describes what this struct really is. the parser really does modify its state (most notably position and symbol tables), so calling it that rather than obliquely "data" (which implies being input only) makes sense. (cherry picked from commit 007605616477f4f0d8a0064c375b1d3cf6188ac5) Change-Id: I92feaec796530e1d4d0f7d4fba924229591cea95