| Age | Commit message (Collapse) | Author |
|---|
|
We got confused what formals did and had to briefly figure it out. We
should just have docs, so these are some.
Change-Id: If3e794a401e69d022785cbfa0b0c2e2284f41f58
|
|
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).
this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.
since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.
notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).
(cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda)
Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
|
|
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit f24e445bc024cfd3c26be5f061280af549321c22)
Change-Id: I7acda5d5c34c0914a78adc2385d32782c4c275cd
|
|
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.
(cherry picked from commit 1cd87b7042d14aae1fafa47b1c28db4c5bd20de7)
Change-Id: I90bbebddf06762960d8ca4f621cf042ce8ae83f9
|
|
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
(cherry picked from commit cefd0302b55b3360dbca59cfcb4bf6a750d6cdcf)
Change-Id: Iff519f991adef2e51683ba2c552d37a3df7a179e
|
|
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
(cherry picked from commit 6c08fba533ef31cad2bdc03ba72ecf58dc8ee5a0)
Change-Id: Ia887f629305645bb8a165fbbc0d32e620912595a
|
|
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.
(cherry picked from commit 1f542adb3e18e7078e6a589182a53a47d971748a)
Change-Id: If948c9d43e875de18f213a73a06a36f7c335b536
|
|
(cherry picked from commit c66ee57edc6cac3571bfbf77d0c0ea4d25b4e805)
Change-Id: Ie8606a8b2f5946c87dd4d16b7b46203e199a4cc1
|
|
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of
EvalError("expected 'boolean' but found '%1%'", showType(v))
we could write
TypeError(v, "boolean")
or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.
This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).
The core design is in `eval-error.hh`. Generally, errors like this:
state.error("'%s' is not a string", getAttrPathStr())
.debugThrow<TypeError>()
are transformed like this:
state.error<TypeError>("'%s' is not a string", getAttrPathStr())
.debugThrow()
The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
(cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732)
Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
|
|
(cherry picked from commit c62c21e29af20f1c14a59ab37d7a25dd0b70f69e)
Change-Id: Id4ea2fc33b0874b2f1f2a32cabcbeb0afa26808f
|
|
these symbols are used a *lot*, so it makes sense to cache them. this
mostly increases clarity of the code (however clear one may wish to call
the parser desugaring here), but it also provides a small performance
benefit.
(cherry picked from commit 09a1128d9e2ff0ae6176784938047350d6f8a782)
Change-Id: I73d9f66be4555168e048cb2d542277251580c2d1
|
|
Combine `AbstractPos`, `PosAdapter`, and `Pos`
(cherry picked from commit 113499d16fc87d53b73fb62fe6242154909756ed)
===
this is a bit cursed because originally it was based on InputAccessor
code that we don't have and moved/patched features we likewise don't
have (fetchToStore caching, all the individual accessors,
ContentAddressMethod). the commit is adjusted accordingly to
match (remove caching, ignore accessors, use FileIngestionMethod).
note that `state.rootPath . CanonPath == abs` and
computeStorePathForPath works relative to cwd, so the slight rewrite in
the moved fetchToStore is legal.
Change-Id: I05fd340c273f0bcc8ffabfebdc4a88b98083bce5
|
|
Optimize empty list constants
(cherry picked from commit 315aade89d00c692715e5953c36a1b7d6528b703)
Change-Id: I0f28ef8a27ccedc45acf44243eec9dc35b733300
|
|
reduce the size of Env by one pointer
(cherry picked from commit 83f5622545a2fc31eb7e7d5105f64ed6dd3058b3)
Change-Id: I5636290526d0165cfc61aee1e7a5b94db4a26cef
|
|
a packet of small optimizations
(cherry picked from commit ee439734e924eb337a869ff2e48aff8b989198bc)
Change-Id: I125d870710750a32a0dece48f39a3e9132b0d023
|
|
Pass positions when evaluating
(cherry picked from commit c8458bd731eb1c74159bebe459ea00165e056b65)
Change-Id: I1b4a5d58973be6264ffdb23b4492da200fdb71be
|
|
libexpr: const rvalue reference -> value for nix::Expr nodes
(cherry picked from commit de99647b9c66219b2e7bc698a62377cc4f6148f6)
Change-Id: I96baf96b0a17e401548a926a7015f85e11d7c459
|
|
|
|
* Finish converting existing comments for internal API docs
99% of this was just reformatting existing comments. Only two exceptions:
- Expanded upon `BuildResult::status` compat note
- Split up file-level `symbol-table.hh` doc comments to get
per-definition docs
Also fixed a few whitespace goofs, turning leading tabs to spaces and
removing trailing spaces.
Picking up from #8133
* Fix two things from comments
* Use triple-backtick not indent for `dumpPath`
* Convert GNU-style `\`..'` quotes to markdown style in API docs
This will render correctly.
|
|
|
|
`///@file` makes them show up in the internal API dos. A tiny few were
missing `#pragma once`.
|
|
We are looking for *$ because it indicate that it was constructed with a new but
not release. De-referencing shallow copy so deleting as whole might create
dangling pointer that's why we move it so we delete a empty containers + the
nice perf boost.
|
|
|
|
This reverts commit 9b33ef3879a764bed4cc2404a08344c3a697a646.
|
|
This reverts commit a75b7ba30f1e4f8b15e810fd18e63ee9552e0815, reversing
changes made to 9af16c5f742300e831a2cc400e43df1e22f87f31.
|
|
|
|
|
|
This makes the position object used in exceptions abstract, with a
method getSource() to get the source code of the file in which the
error originated. This is needed for lazy trees because source files
don't necessarily exist in the filesystem, and we don't want to make
libutil depend on the InputAccessor type in libfetcher.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
In particular, use std::make_shared and enumerate(). Also renamed some
fields to fit naming conventions.
|
|
|
|
|
|
|
|
Since Symbol is just an integer, passing it by const reference is
never advantageous.
|
|
after #6218 `Symbol` no longer confers a uniqueness invariant on the
string it wraps, it is now possible to create multiple symbols that
compare equal but whose string contents have different addresses. this
guarantee is now only provided by `SymbolIdx`, leaving `Symbol` only as
a string wrapper that knows about the intricacies of how symbols need to
be formatted for output.
this change renames `SymbolIdx` to `Symbol` to restore the previous
semantics of `Symbol` to that name. we also keep the wrapper type and
rename it to `SymbolStr` instead of returning plain strings from lookups
into the symbol table because symbols are formatted for output in many
places. theoretically we do not need `SymbolStr`, only a function that
formats a string for output as a symbol, but having to wrap every symbol
that appears in a message into eg `formatSymbol()` is error-prone and
inconvient.
|
|
|
|
this slightly increases the amount of memory used for any given symbol, but this
increase is more than made up for if the symbol is referenced more than once in
the EvalState that holds it. on average every symbol should be referenced at
least twice (once to introduce a binding, once to use it), so we expect no
increase in memory on average.
symbol tables are limited to 2³² entries like position tables, and similar
arguments apply to why overflow is not likely: 2³² symbols would require as many
string instances (at 24 bytes each) and map entries (at 24 bytes or more each,
assuming that the map holds on average at most one item per bucket as the docs
say). a full symbol table would require at least 192GB of memory just for
symbols, which is well out of reach. (an ofborg eval of nixpks today creates
less than a million symbols!)
|
|
PosTable deduplicates origin information, so using symbols for paths is no
longer necessary. moving away from path Symbols also reduces the usage of
symbols for things that are not keys in attribute sets, which will become
important in the future when we turn symbols into indices as well.
|
|
Pos objects are somewhat wasteful as they duplicate the origin file name and
input type for each object. on files that produce more than one Pos when parsed
this a sizeable waste of memory (one pointer per Pos). the same goes for
ptr<Pos> on 64 bit machines: parsing enough source to require 8 bytes to locate
a position would need at least 8GB of input and 64GB of expression memory. it's
not likely that we'll hit that any time soon, so we can use a uint32_t index to
locate positions instead.
|
|
|
|
|
|
|
