aboutsummaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2022-01-19Replace withBuffer by boost small_vectorRobert Hensing
Although this will leave gaps in the stack, the performance impact of those should be insignificant and we get a simpler solution this way.
2022-01-19withBuffer: Make sure to hit the stack protectorRobert Hensing
2022-01-19withBuffer: avoid allocating a std::functionRobert Hensing
2022-01-19Add withBufferRobert Hensing
... to avoid non-standard, unidiomatic alloca.
2022-01-19Fix segfault or stack overflow caused by large derivation fieldsRobert Hensing
This removes a dynamic stack allocation, making the derivation unparsing logic robust against overflows when large strings are added to a derivation. Overflow behavior depends on the platform and stack configuration. For instance, x86_64-linux/glibc behaves as (somewhat) expected: $ (ulimit -s 20000; nix-instantiate tests/lang/eval-okay-big-derivation-attr.nix) error: stack overflow (possible infinite recursion) $ (ulimit -s 40000; nix-instantiate tests/lang/eval-okay-big-derivation-attr.nix) error: expression does not evaluate to a derivation (or a set or list of those) However, on aarch64-darwin: $ nix-instantiate big-attr.nix ~ zsh: segmentation fault nix-instantiate big-attr.nix This indicates a slight flaw in the single stack protection page approach that is not encountered with normal stack frames.
2022-01-19don't use Symbols for stringspennae
string expressions by and large do not need the benefits a Symbol gives us, instead they pollute the symbol table and cause unnecessary overhead for almost all strings. the one place we can think of that benefits from them (attrpaths with expressions) extracts the benefit in the parser, which we'll have to touch anyway when changing ExprString to hold strings. this gives a sizeable improvement on of 3-5% on all benchmarks we've run. before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.844 s ± 0.045 s [User: 6.750 s, System: 1.663 s] Range (min … max): 8.758 s … 8.922 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 367.4 ms ± 3.3 ms [User: 332.3 ms, System: 35.2 ms] Range (min … max): 364.0 ms … 375.2 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.810 s ± 0.030 s [User: 2.517 s, System: 0.225 s] Range (min … max): 2.742 s … 2.854 s 20 runs after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.533 s ± 0.068 s [User: 6.485 s, System: 1.642 s] Range (min … max): 8.404 s … 8.657 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 347.6 ms ± 3.1 ms [User: 313.1 ms, System: 34.5 ms] Range (min … max): 343.3 ms … 354.6 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.709 s ± 0.032 s [User: 2.414 s, System: 0.232 s] Range (min … max): 2.655 s … 2.788 s 20 runs
2022-01-19Fix the build with nlohmann/json 3.10.4+regnat
2022-01-19Make `nix why-depends` quieter by defaultregnat
Unless `--precise` is passed, make `nix why-depends` only show the dependencies between the store paths, without introspecting them to find the actual references. This also makes it ~3x faster
2022-01-19remove ExprIndStrpennae
it can be replaced with StringToken if we add another bit if information to StringToken, namely whether this string should take part in indentation scanning or not. since all escaping terminates indentation scanning we need to set this bit only for the non-escaped IND_STRING rule. this improves performance by about 1%. before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.880 s ± 0.048 s [User: 6.809 s, System: 1.643 s] Range (min … max): 8.781 s … 8.993 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 375.0 ms ± 2.2 ms [User: 339.8 ms, System: 35.2 ms] Range (min … max): 371.5 ms … 379.3 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.831 s ± 0.040 s [User: 2.536 s, System: 0.225 s] Range (min … max): 2.769 s … 2.912 s 20 runs after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 8.832 s ± 0.048 s [User: 6.757 s, System: 1.657 s] Range (min … max): 8.743 s … 8.921 s 20 runs nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 367.4 ms ± 3.2 ms [User: 332.7 ms, System: 34.7 ms] Range (min … max): 364.6 ms … 374.6 ms 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.810 s ± 0.030 s [User: 2.517 s, System: 0.225 s] Range (min … max): 2.742 s … 2.854 s 20 runs
2022-01-18allow modifying lockfile commit msg with nix config optionlincoln auster [they/them]
This allows setting the commit-lockfile-summary option to a non-empty string to override the commit summary while leaving the body unchanged.
2022-01-18Merge pull request #5906 from pennae/primops-optimizationEelco Dolstra
optimize primops and utils by caching more and copying less
2022-01-18Merge pull request #5918 from tweag/tty-logsEelco Dolstra
Print build logs when not connected to a TTY
2022-01-18Add examplesEelco Dolstra
2022-01-18Add Installable::toDrvPaths()Eelco Dolstra
This is needed to get the path of a derivation that might not exist (e.g. for 'nix store copy-log'). InstallableStorePath::toDerivedPaths() cannot be used for this because it calls readDerivation(), so it fails if the store doesn't have the derivation.
2022-01-18BinaryCacheStore: Implement addBuildLog()Eelco Dolstra
2022-01-18Add command 'nix store copy-log'Eelco Dolstra
Fixes #5222.
2022-01-18Factor out --from / --to logicEelco Dolstra
2022-01-18DohEelco Dolstra
2022-01-18DohEelco Dolstra
2022-01-18Merge pull request #5932 from edolstra/remove-shared-stringsEelco Dolstra
Remove shared strings
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2022-01-18Merge pull request #5720 from tomberek/flake_searchThéophane Hufschmitt
flakes: search up to git or filesystem boundary
2022-01-18Fix the rendering of the example directory treeregnat
2022-01-17StringSource: Use std::string_viewEelco Dolstra
2022-01-17serialise.hh: Use std::string_viewEelco Dolstra
2022-01-17Merge pull request #5812 from pennae/small-perf-improvementsEelco Dolstra
improve parser performance a bit
2022-01-17Update documentation for paths on command lineAlexander Bantyev
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-17Show build and substitution information when not connected to a TTYAlexander Bantyev
When stderr is not connected to a tty, show "building" and "substituting" messages, a-la nix-build et al. Closes https://github.com/NixOS/nix/issues/4402 Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-17Merge pull request #5914 from NixOS/5911-fix-nix-why-depends-pagerEelco Dolstra
Start the pager early-enough in `nix why-depends`
2022-01-15Make queryJSON not bail immediately on an assertion or errorFarid Zakaria
2022-01-14Document searching upwards and fix documentation for installablesAlexander Bantyev
2022-01-14Add more tests for flake upward searchingAlexander Bantyev
2022-01-14Apply suggestions from code reviewtomberek
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-14also cache split regexes, not just match regexespennae
gives about 1% improvement on system eval, a bit less on nix search. # before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 7.419 s ± 0.045 s [User: 6.362 s, System: 0.794 s] Range (min … max): 7.335 s … 7.517 s 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.921 s ± 0.023 s [User: 2.626 s, System: 0.210 s] Range (min … max): 2.883 s … 2.957 s 20 runs # after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 7.370 s ± 0.059 s [User: 6.333 s, System: 0.791 s] Range (min … max): 7.286 s … 7.541 s 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.891 s ± 0.033 s [User: 2.606 s, System: 0.210 s] Range (min … max): 2.823 s … 2.958 s 20 runs
2022-01-14optimize removeAttrs builtinpennae
use a sorted array of symbols to be removed instead of a set. this saves a lot of memory allocations and slightly speeds up removal.
2022-01-14Start the pager early-enough in `nix why-depends`regnat
`nix why-depends` is piping its output into a pager by default. However the pager was only started after the first path is printed, causing it to be excluded from the pager output. (Actually the pager was started *inside* the recursive function that was printing the dependency chain, so a new instance was started at each level. It’s a little miracle that it worked at all). Fix #5911
2022-01-13optimize unescapeStrpennae
mainly to avoid an allocation and a copy of a string that can be modified in place (ever since EvalState holds on to the buffer, not the generated parser itself). # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 570.4 ms ± 2.8 ms [User: 561.3 ms, System: 8.6 ms] Range (min … max): 564.6 ms … 578.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 375.4 ms ± 1.3 ms [User: 343.2 ms, System: 31.7 ms] Range (min … max): 373.4 ms … 378.2 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.925 s ± 0.006 s [User: 2.704 s, System: 0.219 s] Range (min … max): 2.910 s … 2.942 s 50 runs
2022-01-13avoid copies of parser input datapennae
when given a string yacc will copy the entire input to a newly allocated location so that it can add a second terminating NUL byte. since the parser is a very internal thing to EvalState we can ensure that having two terminating NUL bytes is always possible without copying, and have the parser itself merely check that the expected NULs are present. # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs
2022-01-13avoid allocations in SymbolTable::createpennae
speeds up parsing by ~3%, system builds by a bit more than 1% # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 574.7 ms ± 2.8 ms [User: 566.3 ms, System: 8.0 ms] Range (min … max): 569.2 ms … 580.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 394.4 ms ± 0.8 ms [User: 361.8 ms, System: 32.3 ms] Range (min … max): 392.7 ms … 395.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.976 s ± 0.005 s [User: 2.757 s, System: 0.218 s] Range (min … max): 2.966 s … 2.990 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs
2022-01-13don't strdup tokens in the lexerpennae
every stringy token the lexer returns is turned into a Symbol and not used further, so we don't have to strdup. using a string_view is sufficient, but due to limitations of the current parser we have to use a POD type that holds the same information. gives ~2% on system build, 6% on search, 8% on parsing alone # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 610.6 ms ± 2.4 ms [User: 602.5 ms, System: 7.8 ms] Range (min … max): 606.6 ms … 617.3 ms 50 runs Benchmark 2: nix eval -f hackage-packages.nix Time (mean ± σ): 430.1 ms ± 1.4 ms [User: 393.1 ms, System: 36.7 ms] Range (min … max): 428.2 ms … 434.2 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 3.032 s ± 0.005 s [User: 2.808 s, System: 0.223 s] Range (min … max): 3.023 s … 3.041 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 574.7 ms ± 2.8 ms [User: 566.3 ms, System: 8.0 ms] Range (min … max): 569.2 ms … 580.7 ms 50 runs Benchmark 2: nix eval -f hackage-packages.nix Time (mean ± σ): 394.4 ms ± 0.8 ms [User: 361.8 ms, System: 32.3 ms] Range (min … max): 392.7 ms … 395.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.976 s ± 0.005 s [User: 2.757 s, System: 0.218 s] Range (min … max): 2.966 s … 2.990 s 50 runs
2022-01-13move strings in derivationStrictpennae
the temporary will be discarded anyway, so we can move out of it and save many small allocations and copies.
2022-01-13avoid string copies in attrNames sort comparisonpennae
symbols can also be cast to string_view, which compares the same but doesn't require a copy of both symbol names on every comparison.
2022-01-13cache more often-used symbols for primopspennae
there's a few symbols in primops we can create once and pick them out of EvalState afterwards instead of creating them every time we need them. this gives almost 1% speedup to an uncached nix search.
2022-01-13use more string_view in utilspennae
there's a couple places that can be easily converted from using strings to using string_views instead. gives a slight (~1%) boost to system eval. # before nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.946 s ± 0.026 s [User: 2.655 s, System: 0.209 s] Range (min … max): 2.905 s … 2.995 s 20 runs # after Time (mean ± σ): 2.928 s ± 0.024 s [User: 2.638 s, System: 0.211 s] Range (min … max): 2.893 s … 2.970 s 20 runs
2022-01-12Merge pull request #5887 from pennae/avoid-streamsThéophane Hufschmitt
avoid std::?stream overhead when it's not helpful
2022-01-12use boost::lexical_cast for string2*pennae
this avoids one copy from `s` into `str`, and possibly another copy needed to construct `s` at the call site. lexical_cast is also more efficient in general.
2022-01-12optimize ExprConcatStrings::evalpennae
constructing an ostringstream for non-string concats (like integer addition) is a small constant cost that we can avoid. for string concats we can keep all the string temporaries we get from coerceToString and concatenate them in one go, which saves a lot of intermediate temporaries and copies in ostringstream. we can also avoid copying the concatenated string again by directly allocating it in GC memory and moving ownership of the concatenated string into the target value. saves about 2% on system eval. before: Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.837 s ± 0.031 s [User: 2.562 s, System: 0.191 s] Range (min … max): 2.796 s … 2.892 s 20 runs after: Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.790 s ± 0.035 s [User: 2.532 s, System: 0.187 s] Range (min … max): 2.722 s … 2.836 s 20 runs
2022-01-11Merge pull request #5883 from fzakaria/faridzakaria/check-metaThéophane Hufschmitt
Check for `--meta` before emitting it for `--json`
2022-01-11Merge pull request #5898 from layus/repair-path-linksEelco Dolstra
Make --repair-path also repair corrupt optimised links
2022-01-11Prefer RepairFlag over bool when applicableGuillaume Maudoux
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 15:40:05 +0000