aboutsummaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2024-08-10nix: remove explosions if you have a window size less than fourJade Lovelace
Turns out strings do not like being resized to -4. This was discovered while messing with the tests to remove unbuffer and trying stdbuf instead. Turns out that was not the right approach. This basically rewrites the handling of this case to be much more correct, and fixes a bug where with small window sizes where it would ALSO truncate the attr names in addition to the optional descriptions. Change-Id: Ifd1beeaffdb47cbb5f4a462b183fcb6c0ff6c524
2024-08-10build: remove expect as a dependencyJade Lovelace
I was packaging Lix 2.91 for nixpkgs and was annoyed at the expect dependency. Turns out that you can replace unbuffer with a pretty-short Python script. It became less short after I found out that Linux was converting \n to \r\n in the terminal subsystem, which was not very funny, but is at least solved by twiddling termios bits. Change-Id: I8a2700abcbbf6a9902e01b05b40fa9340c0ab90c
2024-08-08Merge changes I526cceed,Ia4e2f1fa,I22e66972,I9fbd55a9,Ifca22e44 into mainjade
* changes: sqlite: add a Use::fromStrNullable util: implement charptr_cast tree-wide: fix a pile of lints refactor: make HashType and Base enum classes for type safety build: integrate clang-tidy into CI
2024-08-08tree-wide: fix a pile of lintsJade Lovelace
This: - Converts a bunch of C style casts into C++ casts. - Removes some very silly pointer subtraction code (which is no more or less busted on i686 than it began) - Fixes some "technically UB" that never had to be UB in the first place. - Makes finally follow the noexcept status of the inner function. Maybe in the future we should ban the function from not being noexcept, but that is not today. - Makes various locally-used exceptions inherit from std::exception. Change-Id: I22e66972602604989b5e494fd940b93e0e6e9297
2024-08-08refactor: make HashType and Base enum classes for type safetyJade Lovelace
Change-Id: I9fbd55a9d50464a56fe11cb42a06a206914150d8
2024-08-08libexpr: Add experimental pipe operatorpiegames
The |> operator is a reverse function operator with low binding strength to replace lib.pipe. Implements RFC 148, see the RFC text for more details. Closes #438. Change-Id: I21df66e8014e0d4dd9753dd038560a2b0b7fd805
2024-08-07Merge "src/libcmd/repl.cc: allow :log /path/to/store.drv" into mainMax Siling
2024-08-07libexpr/parser: Test experimental featurespiegames
Currently, the parser relies on the global experimental feature flags. In order to properly test conditional language features, we instead need to pass it around in the parser::State. This means that the parser cannot cache the result of isEnabled anymore, which wouldn't necessarily hurt performance if the function didn't perform a linear search on the list of enabled features on every single call. While we could simply evaluate once at the start of parsing and cache the result in the parser state, the more sustainable solution would be to fix `isEnabled` such that all callers may profit from the performance improvement. Change-Id: Ic9b9c5d882b6270e1114988b63e6064d36c25cf2
2024-08-07src/libcmd/repl.cc: allow :log /path/to/store.drvGoldstein
This adds a second form to the `:log` command: it now can accept a derivation path in addition to a derivation expression. As derivation store paths start with `/nix/store`, this is not ambiguous. Resolves: https://git.lix.systems/lix-project/lix/issues/51 Change-Id: Iebc7b011537e7012fae8faed4024ea1b8fdc81c3
2024-08-07Merge "fix: warn and document when advanced attributes will have no impact ↵Maximilian Bosch
due to __structuredAttrs" into main
2024-08-04tree-wide: fix various lint warningsJade Lovelace
Change-Id: I0fc80718eb7e02d84cc4b5d5deec4c0f41116134
2024-08-04nix flake metadata: print modified dates for input flakesJade Lovelace
This was always in the lock file and we can simply actually print it. The test for this is a little bit silly but it should correctly control for my daring to exercise timezone code *and* locale code in a test, which I strongly suspect nobody dared do before. Sample (abridged): ``` Path: /nix/store/gaxb42z68bcr8lch467shvmnhjjzgd8b-source Last modified: 1970-01-01 00:16:40 Inputs: ├───flake-compat: github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33 │ Last modified: 2023-10-04 13:37:54 ├───flake-utils: github:numtide/flake-utils/b1d9ab70662946ef0850d488da1c9019f3a9752a │ Last modified: 2024-03-11 08:33:50 │ └───systems: github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e │ Last modified: 2023-04-09 08:27:08 ``` Change-Id: I355f82cb4b633974295375ebad646fb6e2107f9b
2024-08-04cli: eat terminal codes from stdout alsoJade Lovelace
This *should* be sound, plus or minus the amount that the terminal code eating code is messed up already. This is useful for testing CLI output because it will strip the escapes enough to just shove the expected output in a file. Change-Id: I8a9b58fafb918466ac76e9ab585fc32fb9294819
2024-08-03fix: warn and document when advanced attributes will have no impact due to ↵Tom Bereknyei
__structuredAttrs Backport of https://github.com/NixOS/nix/pull/10884. Change-Id: I82cc2794730ae9f4a9b7df0185ed0aea83efb65a
2024-08-02Merge "nix flake show: add the description if it exists" into mainIsabel
2024-08-02Merge "Reapply "libfetchers: make attribute / URL query handling ↵Maximilian Bosch
consistent"" into main
2024-08-01Reapply "libfetchers: make attribute / URL query handling consistent"Maximilian Bosch
The original attempt at this introduced a regression; this commit reverts the revert and fixes the regression. This reverts commit 3e151d4d77b5296b9da8c3ad209932d1dfa44c68. Fix to the regression: flakeref: fix handling of `?dir=` param for flakes in subdirs As reported in #419[1], accessing a flake in a subdir of a Git repository fails with the previous commit[2] applied with the error error: unsupported Git input attribute 'dir' The problem is that the `dir`-param is inserted into the parsed URL if a flake is fetched from the subdir of a Git repository. However, for the fetching part this isn't even needed. The fix is to just pass `subdir` as second argument to `FlakeRef` (which needs a `basedir` that can be empty) and leave the parsedURL as-is. Added a regression test to make sure we don't run into this again. [1] https://git.lix.systems/lix-project/lix/issues/419 [2] e22172aaf6b6a366cecd3c025590e68fa2b91bcc, originally 3e151d4d77b5296b9da8c3ad209932d1dfa44c68 Change-Id: I2c72d5a32e406a7ca308e271730bd0af01c5d18b
2024-07-31tree-wide: add support for asan!Jade Lovelace
What if you could find memory bugs in Lix without really trying very hard? I've had variously scuffed patches to do this, but this is blocked on boost coroutines removal at this point tbh. Change-Id: Id762af076aa06ad51e77a6c17ed10275929ed578
2024-07-30src/libcmd/repl.cc: avoid unneeded reload after :eGoldstein
If `:edit`ing a store path, don't reload repl afterwards to avoid losing local variables: store is immutable, so "editing" a store path is always just viewing it. Resolves: https://git.lix.systems/lix-project/lix/issues/341 Change-Id: I3747f75ce26e0595e953069c39ddc3ee80699718
2024-07-25libstore/build: block io_uringAlois Wohlschlager
Unfortunately, io_uring is totally opaque to seccomp, and while currently there are no dangerous operations implemented, there is no guarantee that it remains this way. This means that io_uring should be blocked entirely to ensure that the sandbox is future-proof. This has not been observed to cause issues in practice. Change-Id: I45d3895f95abe1bc103a63969f444c334dbbf50d
2024-07-25libstore/build: use an allowlist approach to syscall filteringAlois Wohlschlager
Previously, system call filtering (to prevent builders from storing files with setuid/setgid permission bits or extended attributes) was performed using a blocklist. While this looks simple at first, it actually carries significant security and maintainability risks: after all, the kernel may add new syscalls to achieve the same functionality one is trying to block, and it can even be hard to actually add the syscall to the blocklist when building against a C library that doesn't know about it yet. For a recent demonstration of this happening in practice to Nix, see the introduction of fchmodat2 [0] [1]. The allowlist approach does not share the same drawback. While it does require a rather large list of harmless syscalls to be maintained in the codebase, failing to update this list (and roll out the update to all users) in time has rather benign effects; at worst, very recent programs that already rely on new syscalls will fail with an error the same way they would on a slightly older kernel that doesn't support them yet. Most importantly, no unintended new ways of performing dangerous operations will be silently allowed. Another possible drawback is reduced system call performance due to the larger filter created by the allowlist requiring more computation [2]. However, this issue has not convincingly been demonstrated yet in practice, for example in systemd or various browsers. To the contrary, it has been measured that the the actual filter constructed here has approximately the same overhead as a very simple filter blocking only one system call. This commit tries to keep the behavior as close to unchanged as possible. The system call list is in line with libseccomp 2.5.5 and glibc 2.39, which are the latest versions at the point of writing. Since libseccomp 2.5.5 is already a requirement and the distributions shipping this together with older versions of glibc are mostly not a thing any more, this should not lead to more build failures any more. [0] https://github.com/NixOS/nixpkgs/issues/300635 [1] https://github.com/NixOS/nix/issues/10424 [2] https://github.com/flatpak/flatpak/pull/4462#issuecomment-1061690607 Change-Id: I541be3ea9b249bcceddfed6a5a13ac10b11e16ad
2024-07-23nix flake show: add the description if it existsisabel
(cherry picked from commit 8cd1d02f90eb9915e640c5d370d919fad9833c65) nix flake show: Only print up to the first new line if it exists. (cherry picked from commit 5281a44927bdb51bfe6e5de12262d815c98f6fe7) add tests (cherry picked from commit 74ae0fbdc70a5079a527fe143c4832d1357011f7) Handle long strings, embedded new lines and empty descriptions (cherry picked from commit 2ca7b3afdbbd983173a17fa0a822cf7623601367) Account for total length of 80 (cherry picked from commit 1cc808c18cbaaf26aaae42bb1d7f7223f25dd364) docs: add nix flake show description release note fix: remove white space nix flake show: trim length based on terminal size test: account for terminal size docs(flake-description): before and after commands; add myself to credits Upstream-PR: https://github.com/NixOS/nix/pull/10980 Change-Id: Ie1c667dc816b3dd81e65a1f5395e57ea48ee0362
2024-07-19gc: refactor the gc server thread out into a class without changing itJade Lovelace
This removes a *whole load* of variables from scope and enforces thread boundaries with the type system. There is not much change of significance in here, so the things to watch out for while reviewing it are primarily that the destructor ordering may have changed inadvertently, I think. Change-Id: I3cd87e6d5a08dfcf368637407251db22a8906316
2024-07-18libexpr/print: do not show elided nested items when there are noneAlois Wohlschlager
When the configured maximum depth has been reached, attribute sets and lists are printed with ellipsis to indicate the elision of nested items. Previously, this happened even in case the structure being printed is empty, so that such items do not in fact exist. This is confusing, so stop doing it. Change-Id: I0016970dad3e42625e085dc896e6f476b21226c9
2024-07-18libexpr/print: never show empty attrsets or derivations as «repeated»Alois Wohlschlager
The repeated value detection logic exists so that the occurrence of large common substructures does not fill up the screen or the computer's memory. However, empty attribute sets and derivations (when their detection is enabled) are always cheap to print, and in practice I have observed them to make up a significant majority of the cases where I was annoyed by the repeated value detection kicking in. Furthermore, `nix-instantiate --eval` already disables this logic for empty attribute sets, and empty lists are already exempted everywhere. For these reasons, always print empty attribute sets and derivations as what they are. Change-Id: I5dac8e7739f9d726b76fd0521ec46f38af94463f
2024-07-18libexpr/print: pretty-print idempotentlyAlois Wohlschlager
When pretty-printing is enabled, previously an unforced thunk would trigger indentation, even when it subsequently does not evaluate to a nested structure. The resulting output looked inconsistent, and furthermore pretty-printing was not idempotent (since pretty-printing the same value again, which is now fully evaluated, will not trigger indentation). When strict evaluation is enabled, force the item before inspecting its type, so that it is properly known whether it contains a nested structure. Furthermore, there is no need to cause indentation for unforced thunks, since the very next operation will be printing them as `«thunk»`. This is mostly a port of https://github.com/NixOS/nix/pull/11100 , but we only force the item when it's going to be forced anyway due to strict pretty-printing, and a new test was written since the REPL testing framework in Lix is different. Co-Authored-By: Robert Hensing <robert@roberthensing.nl> Change-Id: Ib7560fe531d09e05ca6b2037a523fe21a26d9d58
2024-07-17Merge "tests/functional/repl.sh: actually fail test on wrong stdout" into mainMax Siling
2024-07-17tests/functional/repl.sh: actually fail test on wrong stdoutGoldstein
Previous test implementation assumed that grep supports newlines in patterns. It doesn't, so tests spuriously passed, even though some tests outputs were broken. This patches output (and expected output) before grepping, so there're no newlines in pattern. Change-Id: Ie6561f9f2e18b83d976f162269d20136e2595141
2024-07-16libstore: remove WriteConn::sink fieldseldritch horrors
we no longer need these since we're no longer using sinks to serialize things. Change-Id: Iffb1a3eab33c83f611c88fa4e8beaa8d5ffa079b
2024-07-16libstore: generatorize protocol serializerseldritch horrors
this is cursed. deeply and profoundly cursed. under NO CIRCUMSTANCES must protocol serializer helpers be applied to temporaries! doing so will inevitably cause dangling references and cause the entire thing to crash. we need to do this even so to get rid of boost coroutines, and likewise to encapsulate the serializers we suffer today at least a little bit to allow a gradual migration to an actual IPC protocol. (this isn't a problem that's unique to generators. c++ coroutines in general cannot safely take references to arbitrary temporaries since c++ does not have a lifetime system that can make this safe. -sigh-) Change-Id: I2921ba451e04d86798752d140885d3c5cc08e146
2024-07-15nix3-upgrade-nix: always use the /new/ nix-env to perform the installationQyriad
Fixes #411. Change-Id: I8d87c0e9295deea26ff33234e15ee33cc68ab303
2024-07-13language: cleanly ban integer overflowsJade Lovelace
This also bans various sneaking of negative numbers from the language into unsuspecting builtins as was exposed while auditing the consequences of changing the Nix language integer type to a newtype. It's unlikely that this change comprehensively ensures correctness when passing integers out of the Nix language and we should probably add a checked-narrowing function or something similar, but that's out of scope for the immediate change. During the development of this I found a few fun facts about the language: - You could overflow integers by converting from unsigned JSON values. - You could overflow unsigned integers by converting negative numbers into them when going into Nix config, into fetchTree, and into flake inputs. The flake inputs and Nix config cannot actually be tested properly since they both ban thunks, however, we put in checks anyway because it's possible these could somehow be used to do such shenanigans some other way. Note that Lix has banned Nix language integer overflows since the very first public beta, but threw a SIGILL about them because we run with -fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in production builds. Since the Nix language uses signed integers, overflow was simply undefined behaviour, and since we defined that to trap, it did. Trapping on it was a bad UX, but we didn't even entirely notice that we had done this at all until it was reported as a bug a couple of months later (which is, to be fair, that flag working as intended), and it's got enough production time that, aside from code that is IMHO buggy (and which is, in any case, not in nixpkgs) such as https://git.lix.systems/lix-project/lix/issues/445, we don't think anyone doing anything reasonable actually depends on wrapping overflow. Even for weird use cases such as doing funny bit crimes, it doesn't make sense IMO to have wrapping behaviour, since two's complement arithmetic overflow behaviour is so *aggressively* not what you want for *any* kind of mathematics/algorithms. The Nix language exists for package management, a domain where bit crimes are already only dubiously in scope to begin with, and it makes a lot more sense for that domain for the integers to never lose precision, either by throwing errors if they would, or by being arbitrary-precision. This change will be ported to CppNix as well, to maintain language consistency. Fixes: https://git.lix.systems/lix-project/lix/issues/423 Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
2024-07-13libutil: add checked arithmetic toolsJade Lovelace
This is in preparation for adding checked arithmetic to the evaluator. Change-Id: I6e115ce8f5411feda1706624977a4dcd5efd4d13
2024-07-11libutil: rewrite RewritingSink as sourceeldritch horrors
the rewriting sink was just broken. when given a rewrite set that contained a key that is also a proper infix of another key it was possible to produce an incorrectly rewritten result if the writer used the wrong block size. fixing this duplicates rewriteStrings, to avoid this we'll rewrite rewriteStrings to use RewritingSource in a new mode that'll allow rewrites we had previously forbidden. Change-Id: I57fa0a9a994e654e11d07172b8e31d15f0b7e8c0
2024-07-09Fix dry-run flag for nix-collect-garbageQuantum Jump
`nix-collect-garbage --dry-run` previously elided the entire garbage collection check, meaning that it would just exit the script without printing anything. This change makes the dry run flag instead set the GC action to `gcReturnDead` rather than `gcDeleteDead`, and then continue with the script. So if you set `--dry-run`, it will print the paths it *would* have garbage collected, but not actually delete them. I filed a bug for this: https://git.lix.systems/lix-project/lix/issues/432 but then realised I could give fixing it a go myself. Change-Id: I062dbf1a80bbab192b5fd0b3a453a0b555ad16f2
2024-07-05libutil: allow construction of sources from generatorseldritch horrors
Change-Id: I78ff8d0720f06bce731e26d5e1c53b1382bbd589
2024-07-04distinguish between throws & errors during throwQyriad
Turns errors like this: let throwMsg = a: throw (a + " invalid bar"); in throwMsg "bullshit" error: … from call site at «string»:3:4: 2| throwMsg = a: throw (a + " invalid bar"); 3| in throwMsg "bullshit" | ^ … while calling 'throwMsg' at «string»:2:14: 1| let 2| throwMsg = a: throw (a + " invalid bar"); | ^ 3| in throwMsg "bullshit" … while calling the 'throw' builtin at «string»:2:17: 1| let 2| throwMsg = a: throw (a + " invalid bar"); | ^ 3| in throwMsg "bullshit" error: bullshit invalid bar into errors like this: let throwMsg = a: throw (a + " invalid bar"); in throwMsg "bullshit" error: … from call site at «string»:3:4: 2| throwMsg = a: throw (a + " invalid bar"); 3| in throwMsg "bullshit" | ^ … while calling 'throwMsg' at «string»:2:14: 1| let 2| throwMsg = a: throw (a + " invalid bar"); | ^ 3| in throwMsg "bullshit" … caused by explicit throw at «string»:2:17: 1| let 2| throwMsg = a: throw (a + " invalid bar"); | ^ 3| in throwMsg "bullshit" error: bullshit invalid bar Change-Id: I593688928ece20f97999d1bf03b2b46d9ac338cb
2024-07-04trace when the `foo` part of `foo.bar.baz` errorsQyriad
Turns errors like: let errpkg = throw "invalid foobar"; in errpkg.meta error: … while calling the 'throw' builtin at «string»:2:12: 1| let 2| errpkg = throw "invalid foobar"; | ^ 3| in errpkg.meta error: invalid foobar into errors like: let errpkg = throw "invalid foobar"; in errpkg.meta error: … while evaluating 'errpkg' to select 'meta' on it at «string»:3:4: 2| errpkg = throw "invalid foobar"; 3| in errpkg.meta | ^ … while calling the 'throw' builtin at «string»:2:12: 1| let 2| errpkg = throw "invalid foobar"; | ^ 3| in errpkg.meta error: invalid foobar For the low price of one try/catch, you too can have the incorrect line of code actually show up in the trace! Change-Id: If8d6200ec1567706669d405c34adcd7e2d2cd29d
2024-07-04add an impl of Expr::show for ExprInheritFrom that doesn't crashQyriad
ExprVar::show() assumes it has a name. dynamic inherits do not necessarily (ever?) have a name. Change-Id: If10893188e307431da17f0c1bd0787adc74f7141
2024-07-03libutil: begin porting serialization to generatorseldritch horrors
generators are a better basis for serializers than streaming into sinks as we do currently for many reasons, such as being usable as sources if one wishes to (without requiring an intermediate sink to serialize full data sets into memory, or boost coroutines to turn sinks into sources), composing more naturally (as one can just yield a sub-generator instead of being forced to wrap entire substreams into clunky functions or even more clunky custom types to implement operator<< on), allowing wrappers to transform data with clear ownership semantics (removing the need for explicit memory allocations and Source wrappers), and many other things Change-Id: I361d89ff556354f6930d9204f55117565f2f7f20
2024-07-03libutil: generator type with on-yield value mappingeldritch horrors
this will be the basis of non-boost coroutines in lix. anything that is a boost coroutine *should* be representable with a Generator coroutine, and many things that are not currently boost coroutines but behave much like one (such as, notably, serializers) should be as well. this allows us to greatly simplify many things that look like iteration but aren't. Change-Id: I2cebcefa0148b631fb30df4c8cfa92167a407e34
2024-07-02Merge "tree-wide: unify progress bar inactive and paused states" into mainalois31
2024-07-02Merge "Reject fully-qualified URLs in 'from' argument of `nix registry add`" ↵Delan Azabani
into main
2024-07-01tree-wide: unify progress bar inactive and paused statesAlois Wohlschlager
Previously, the progress bar had two subtly different states in which the bar would not actually render, both with their own shortcomings: inactive (which was irreversible) and paused (reversible, but swallowing logs). Furthermore, there was no way of resetting the statistics, so a very bad solution was implemented (243c0f18dae2a08ea0e46f7ff33277c63f7506d7) that would create a new logger for each line of the repl, leaking the previous one and discarding the value of printBuildLogs. Finally, if stderr was not attached to a TTY, the update thread was started even though the logger was not active, violating the invariant required by the destructor (which is not observed because the logger is leaked). In this commit, the two aforementioned states are unified into a single one, which can be exited again, correctly upholds the invariant that the update thread is only running while the progress bar is active, and does not swallow logs. The latter change in behavior is not expected to be a problems in the rare cases where the paused state was used before, since other loggers (like the simple one) don't exhibit it anyway. The startProgressBar/stopProgressBar API is removed due to being a footgun, and a new method for properly resetting the progress is added. Co-Authored-By: Qyriad <qyriad@qyriad.me> Change-Id: I2b7c3eb17d439cd0c16f7b896cfb61239ac7ff3a
2024-07-01Merge changes Ie29a8a89,I873eedcf into mainjade
* changes: store: delete obsolete lsof-disabling code store: guess the URL of failing fixed-output derivations
2024-06-30libexpr/flake: allow automatic rejection of configuration options from flakesAlois Wohlschlager
The `allow-flake-configuration` option allows the user to control whether to accept configuration options supplied by flakes. Unfortunately, setting this to false really meant "ask each time" (with an option to remember the choice for each specific option encountered). Let no mean no, and introduce (and default to) a separate value for the "ask each time" behaviour. Co-Authored-By: Jade Lovelace <lix@jade.fyi> Change-Id: I7ccd67a95bfc92cffc1ebdc972d243f5191cc1b4
2024-06-29Reject fully-qualified URLs in 'from' argument of `nix registry add`Delan Azabani
We previously allowed you to map any flake URL to any other flake URL, including shorthand flakerefs, indirect flake URLs like `flake:nixpkgs`, direct flake URLs like `github:NixOS/nixpkgs`, or local paths. But flake registry entries mapping from direct flake URLs often come from swapping the 'from' and 'to' arguments by accident, and even when created intentionally, they may not actually work correctly. This patch rejects those URLs (and fully-qualified flake: URLs), making it harder to swap the arguments by accident. Fixes #181. Change-Id: I24713643a534166c052719b8770a4edfcfdb8cf3
2024-06-27store: guess the URL of failing fixed-output derivationsJade Lovelace
This is a shameless layering violation in favour of UX. It falls back trivially to "unknown", so it's purely a UX feature. Diagnostic sample: ``` error: hash mismatch in fixed-output derivation '/nix/store/sjfw324j4533lwnpmr5z4icpb85r63ai-x1.drv': likely URL: https://meow.puppy.forge/puppy.tar.gz specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= got: sha256-a1Qvp3FOOkWpL9kFHgugU1ok5UtRPSu+NwCZKbbaEro= ``` Change-Id: I873eedcf7984ab23f57a6754be00232b5cb5b02c
2024-06-25Merge "change shebangs of all .sh scripts to bash" into mainjade
2024-06-25libexpr: rewrite the parser with pegtl instead of flex/bisoneldritch horrors
this gives about 20% performance improvements on pure parsing. obviously it will be less on full eval, but depending on how much parsing is to be done (e.g. including hackage-packages.nix or not) it's more like 4%-10%. this has been tested (with thousands of core hours of fuzzing) to ensure that the ASTs produced by the new parser are exactly the same as the old one would have produced. error messages will change (sometimes by a lot) and are not yet perfect, but we would rather leave this as is for later. test results for running only the parser (excluding the variable binding code) in a tight loop with inputs and parameters as given are promising: - 40% faster on lix's package.nix at 10000 iterations - 1.3% faster on nixpkgs all-packages.nix at 1000 iterations - equivalent on all of nixpkgs concatenated at 100 iterations (excluding invalid files, each file surrounded with parens) more realistic benchmarks are somewhere in between the extremes, parsing once again getting the largest uplift. other realistic workloads improve by a few percentage points as well, notably system builds are 4% faster. Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json) old/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix mean: 0.408s ± 0.025s user: 0.355s | system: 0.033s median: 0.389s range: 0.388s ... 0.442s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix mean: 0.332s ± 0.024s user: 0.279s | system: 0.033s median: 0.314s range: 0.313s ... 0.361s relative: 0.814 --- old/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 6.133s ± 0.022s user: 5.395s | system: 0.437s median: 6.128s range: 6.099s ... 6.183s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 5.925s ± 0.025s user: 5.176s | system: 0.456s median: 5.934s range: 5.861s ... 5.943s relative: 0.966 --- GC_INITIAL_HEAP_SIZE=10g old/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 4.503s ± 0.027s user: 3.731s | system: 0.547s median: 4.499s range: 4.478s ... 4.541s relative: 1 GC_INITIAL_HEAP_SIZE=10g new/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 4.285s ± 0.031s user: 3.504s | system: 0.571s median: 4.281s range: 4.221s ... 4.328s relative: 0.951 --- old/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello mean: 16.475s ± 0.07s user: 14.088s | system: 1.572s median: 16.495s range: 16.351s ... 16.536s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello mean: 15.973s ± 0.013s user: 13.558s | system: 1.615s median: 15.973s range: 15.946s ... 15.99s relative: 0.97 --- Change-Id: Ie66ec2d045dec964632c6541e25f8f0797319ee2