aboutsummaryrefslogtreecommitdiff
path: root/src/libexpr
AgeCommit message (Collapse)Author
2022-02-06Merge pull request #6042 from pennae/fix-repl-aEelco Dolstra
fix nix repl not overriding existing bindings in :a
2022-02-04fix nix repl not overriding existing bindings in :apennae
previously :a would override old bindings of a name with new values if the added set contained names that were already bound. in nix 2.6 this doesn't happen any more, which is potentially confusing. fixes #6041
2022-02-04Make most calls to determinePos() lazyEelco Dolstra
2022-02-04Merge branch 'issue-3505' of https://github.com/kamadorueda/nixEelco Dolstra
2022-02-02Merge pull request #6036 from tweag/balsoft/and-yet-another-follows-bugfixEelco Dolstra
Flake follows: resolve all follows to absolute
2022-02-02Flake follows: resolve all follows to absoluteAlexander Bantyev
It's not possible in general to know in computeLocks, relative to which path the follows was intended to be. So, we always resolve follows to their absolute states when we encounter them (which can either be in parseFlakeInput or computeLocks' fake input population). Fixes https://github.com/NixOS/nix/issues/6013 Fixes https://github.com/NixOS/nix/issues/5609 Fixes https://github.com/NixOS/nix/issues/5697 (again)
2022-02-02Merge branch 'parser-improvements' of https://github.com/pennae/nixEelco Dolstra
2022-02-02Merge branch 'more-stringviews' of https://github.com/pennae/nixEelco Dolstra
2022-01-30fix spelling mistakes reported by Debian's lintian toolThomas Koch
2022-01-28Remove a repeated std::move in a for loopEelco Dolstra
2022-01-27optionally return string_view from coerceToStringpennae
we'll retain the old coerceToString interface that returns a string, but callers that don't need the returned value to outlive the Value it came from can save copies by using the new interface instead. for values that weren't stringy we'll pass a new buffer argument that'll be used for storage and shouldn't be inspected.
2022-01-27return string_views from forceString*pennae
once a string has been forced we already have dynamic storage allocated for it, so we can easily reuse that storage instead of copying.
2022-01-27convert a for more utilities to string_viewpennae
2022-01-27Don’t require `NIX_PATH` entries to be valid pathsregnat
It’s totally valid to have entries in `NIX_PATH` that aren’t valid paths (they can even be arbitrary urls or `channel:<channel-name>`). Fix #5998 and #5980
2022-01-25Fix parsing of variable names that are a suffix of '__curPos'regnat
Follow-up from #5969 Fix #5982
2022-01-24Allow builtins.{readFile,path} on invalid pathsEelco Dolstra
Stop-gap measure to fix #5975.
2022-01-24Fix parsing of variable names that are a prefix of '__curPos'Eelco Dolstra
Fixes $ nix-instantiate --parse -E 'x: with x; _' (x: (with x; __curPos))
2022-01-24ExprConcatStrings::show(): Print values instead of pointersEelco Dolstra
2022-01-22allocate a GC root value for the Value cache pointerpennae
keeping it as a simple data member means it won't be scanned by the GC, so eventually the GC will collect a cache that is still referenced (resulting in use-after-free of cache elements). fixes #5962
2022-01-21determinePos: remove from critical pathKevin Amado
2022-01-21forceAttrs: make pos mandatoryKevin Amado
2022-01-21avoid unnecesary callsKevin Amado
2022-01-21forceValue: make pos mandatoryKevin Amado
- Make passing the position to `forceValue` mandatory, this way we remember people that the position is important for better error messages - Add pos to all `forceValue` calls
2022-01-21Fix exception handling around realisePath()Eelco Dolstra
This no longer worked correctly because 'path' is uninitialised when an exception occurs, leading to errors like … while importing '' at /nix/store/rrzz5b1pshvzh1437ac9nkl06br81lkv-source/flake.nix:352:13: So move the adding of the error context into realisePath().
2022-01-19defer formals duplicate check for incresed efficiency all roundpennae
if we defer the duplicate argument check for lambda formals we can use more efficient data structures for the formals set, and we can get rid of the duplication of formals names to boot. instead of a list of formals we've seen and a set of names we'll keep a vector instead and run a sort+dupcheck step before moving the parsed formals into a newly created lambda. this improves performance on search and rebuild by ~1%, pure parsing gains more (about 4%). this does reorder lambda arguments in the xml output, but the output is still stable. this shouldn't be a problem since argument order is not semantically important anyway. before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.550 s ± 0.060 s [User: 6.470 s, System: 1.664 s] Range (min … max): 8.435 s … 8.666 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 346.7 ms ± 2.1 ms [User: 312.4 ms, System: 34.2 ms] Range (min … max): 343.8 ms … 353.4 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.720 s ± 0.031 s [User: 2.415 s, System: 0.231 s] Range (min … max): 2.662 s … 2.780 s 20 runs after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.462 s ± 0.063 s [User: 6.398 s, System: 1.661 s] Range (min … max): 8.339 s … 8.542 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 329.1 ms ± 1.4 ms [User: 296.8 ms, System: 32.3 ms] Range (min … max): 326.1 ms … 330.8 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.687 s ± 0.035 s [User: 2.392 s, System: 0.228 s] Range (min … max): 2.626 s … 2.754 s 20 runs
2022-01-19don't use Symbols for stringspennae
string expressions by and large do not need the benefits a Symbol gives us, instead they pollute the symbol table and cause unnecessary overhead for almost all strings. the one place we can think of that benefits from them (attrpaths with expressions) extracts the benefit in the parser, which we'll have to touch anyway when changing ExprString to hold strings. this gives a sizeable improvement on of 3-5% on all benchmarks we've run. before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.844 s ± 0.045 s [User: 6.750 s, System: 1.663 s] Range (min … max): 8.758 s … 8.922 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 367.4 ms ± 3.3 ms [User: 332.3 ms, System: 35.2 ms] Range (min … max): 364.0 ms … 375.2 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.810 s ± 0.030 s [User: 2.517 s, System: 0.225 s] Range (min … max): 2.742 s … 2.854 s 20 runs after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.533 s ± 0.068 s [User: 6.485 s, System: 1.642 s] Range (min … max): 8.404 s … 8.657 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 347.6 ms ± 3.1 ms [User: 313.1 ms, System: 34.5 ms] Range (min … max): 343.3 ms … 354.6 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.709 s ± 0.032 s [User: 2.414 s, System: 0.232 s] Range (min … max): 2.655 s … 2.788 s 20 runs
2022-01-19remove ExprIndStrpennae
it can be replaced with StringToken if we add another bit if information to StringToken, namely whether this string should take part in indentation scanning or not. since all escaping terminates indentation scanning we need to set this bit only for the non-escaped IND_STRING rule. this improves performance by about 1%. before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.880 s ± 0.048 s [User: 6.809 s, System: 1.643 s] Range (min … max): 8.781 s … 8.993 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 375.0 ms ± 2.2 ms [User: 339.8 ms, System: 35.2 ms] Range (min … max): 371.5 ms … 379.3 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.831 s ± 0.040 s [User: 2.536 s, System: 0.225 s] Range (min … max): 2.769 s … 2.912 s 20 runs after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.832 s ± 0.048 s [User: 6.757 s, System: 1.657 s] Range (min … max): 8.743 s … 8.921 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 367.4 ms ± 3.2 ms [User: 332.7 ms, System: 34.7 ms] Range (min … max): 364.6 ms … 374.6 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.810 s ± 0.030 s [User: 2.517 s, System: 0.225 s] Range (min … max): 2.742 s … 2.854 s 20 runs
2022-01-18allow modifying lockfile commit msg with nix config optionlincoln auster [they/them]
This allows setting the commit-lockfile-summary option to a non-empty string to override the commit summary while leaving the body unchanged.
2022-01-18Merge pull request #5906 from pennae/primops-optimizationEelco Dolstra
optimize primops and utils by caching more and copying less
2022-01-18Merge pull request #5720 from tomberek/flake_searchThéophane Hufschmitt
flakes: search up to git or filesystem boundary
2022-01-17Merge pull request #5812 from pennae/small-perf-improvementsEelco Dolstra
improve parser performance a bit
2022-01-14Add more tests for flake upward searchingAlexander Bantyev
2022-01-14Apply suggestions from code reviewtomberek
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-14also cache split regexes, not just match regexespennae
gives about 1% improvement on system eval, a bit less on nix search. # before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 7.419 s ± 0.045 s [User: 6.362 s, System: 0.794 s] Range (min … max): 7.335 s … 7.517 s 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.921 s ± 0.023 s [User: 2.626 s, System: 0.210 s] Range (min … max): 2.883 s … 2.957 s 20 runs # after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 7.370 s ± 0.059 s [User: 6.333 s, System: 0.791 s] Range (min … max): 7.286 s … 7.541 s 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.891 s ± 0.033 s [User: 2.606 s, System: 0.210 s] Range (min … max): 2.823 s … 2.958 s 20 runs
2022-01-14optimize removeAttrs builtinpennae
use a sorted array of symbols to be removed instead of a set. this saves a lot of memory allocations and slightly speeds up removal.
2022-01-13optimize unescapeStrpennae
mainly to avoid an allocation and a copy of a string that can be modified in place (ever since EvalState holds on to the buffer, not the generated parser itself). # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 570.4 ms ± 2.8 ms [User: 561.3 ms, System: 8.6 ms] Range (min … max): 564.6 ms … 578.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 375.4 ms ± 1.3 ms [User: 343.2 ms, System: 31.7 ms] Range (min … max): 373.4 ms … 378.2 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.925 s ± 0.006 s [User: 2.704 s, System: 0.219 s] Range (min … max): 2.910 s … 2.942 s 50 runs
2022-01-13avoid copies of parser input datapennae
when given a string yacc will copy the entire input to a newly allocated location so that it can add a second terminating NUL byte. since the parser is a very internal thing to EvalState we can ensure that having two terminating NUL bytes is always possible without copying, and have the parser itself merely check that the expected NULs are present. # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs
2022-01-13avoid allocations in SymbolTable::createpennae
speeds up parsing by ~3%, system builds by a bit more than 1% # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 574.7 ms ± 2.8 ms [User: 566.3 ms, System: 8.0 ms] Range (min … max): 569.2 ms … 580.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 394.4 ms ± 0.8 ms [User: 361.8 ms, System: 32.3 ms] Range (min … max): 392.7 ms … 395.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.976 s ± 0.005 s [User: 2.757 s, System: 0.218 s] Range (min … max): 2.966 s … 2.990 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs
2022-01-13don't strdup tokens in the lexerpennae
every stringy token the lexer returns is turned into a Symbol and not used further, so we don't have to strdup. using a string_view is sufficient, but due to limitations of the current parser we have to use a POD type that holds the same information. gives ~2% on system build, 6% on search, 8% on parsing alone # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 610.6 ms ± 2.4 ms [User: 602.5 ms, System: 7.8 ms] Range (min … max): 606.6 ms … 617.3 ms 50 runs Benchmark 2: nix eval -f hackage-packages.nix Time (mean ± σ): 430.1 ms ± 1.4 ms [User: 393.1 ms, System: 36.7 ms] Range (min … max): 428.2 ms … 434.2 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 3.032 s ± 0.005 s [User: 2.808 s, System: 0.223 s] Range (min … max): 3.023 s … 3.041 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 574.7 ms ± 2.8 ms [User: 566.3 ms, System: 8.0 ms] Range (min … max): 569.2 ms … 580.7 ms 50 runs Benchmark 2: nix eval -f hackage-packages.nix Time (mean ± σ): 394.4 ms ± 0.8 ms [User: 361.8 ms, System: 32.3 ms] Range (min … max): 392.7 ms … 395.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.976 s ± 0.005 s [User: 2.757 s, System: 0.218 s] Range (min … max): 2.966 s … 2.990 s 50 runs
2022-01-13move strings in derivationStrictpennae
the temporary will be discarded anyway, so we can move out of it and save many small allocations and copies.
2022-01-13avoid string copies in attrNames sort comparisonpennae
symbols can also be cast to string_view, which compares the same but doesn't require a copy of both symbol names on every comparison.
2022-01-13cache more often-used symbols for primopspennae
there's a few symbols in primops we can create once and pick them out of EvalState afterwards instead of creating them every time we need them. this gives almost 1% speedup to an uncached nix search.
2022-01-12Merge pull request #5887 from pennae/avoid-streamsThéophane Hufschmitt
avoid std::?stream overhead when it's not helpful
2022-01-12optimize ExprConcatStrings::evalpennae
constructing an ostringstream for non-string concats (like integer addition) is a small constant cost that we can avoid. for string concats we can keep all the string temporaries we get from coerceToString and concatenate them in one go, which saves a lot of intermediate temporaries and copies in ostringstream. we can also avoid copying the concatenated string again by directly allocating it in GC memory and moving ownership of the concatenated string into the target value. saves about 2% on system eval. before: Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.837 s ± 0.031 s [User: 2.562 s, System: 0.191 s] Range (min … max): 2.796 s … 2.892 s 20 runs after: Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.790 s ± 0.035 s [User: 2.532 s, System: 0.187 s] Range (min … max): 2.722 s … 2.836 s 20 runs
2022-01-10Merge branch 'readfile' of https://github.com/abbradar/nixEelco Dolstra
2022-01-09builtins.readFile: Propagate path contextNikolay Amiantov
Co-authored-by: Shea Levy <shea@shealevy.com>
2022-01-06Merge pull request #5546 from NixOS/allow-paths-in-flake-local-optionsEelco Dolstra
allow paths in flake local options
2022-01-04Remove EvalState::mkAttrs()Eelco Dolstra
2022-01-04Move empty attrset optimisationEelco Dolstra
2022-01-04Turn mkString(Symbol) into a methodEelco Dolstra
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 16:34:25 +0000