aboutsummaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2022-01-17Merge pull request #5914 from NixOS/5911-fix-nix-why-depends-pagerEelco Dolstra
Start the pager early-enough in `nix why-depends`
2022-01-15Make queryJSON not bail immediately on an assertion or errorFarid Zakaria
2022-01-14Document searching upwards and fix documentation for installablesAlexander Bantyev
2022-01-14Add more tests for flake upward searchingAlexander Bantyev
2022-01-14Apply suggestions from code reviewtomberek
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-14also cache split regexes, not just match regexespennae
gives about 1% improvement on system eval, a bit less on nix search. # before nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 7.419 s ± 0.045 s [User: 6.362 s, System: 0.794 s] Range (min … max): 7.335 s … 7.517 s 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.921 s ± 0.023 s [User: 2.626 s, System: 0.210 s] Range (min … max): 2.883 s … 2.957 s 20 runs # after nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 7.370 s ± 0.059 s [User: 6.333 s, System: 0.791 s] Range (min … max): 7.286 s … 7.541 s 20 runs nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.891 s ± 0.033 s [User: 2.606 s, System: 0.210 s] Range (min … max): 2.823 s … 2.958 s 20 runs
2022-01-14optimize removeAttrs builtinpennae
use a sorted array of symbols to be removed instead of a set. this saves a lot of memory allocations and slightly speeds up removal.
2022-01-14Start the pager early-enough in `nix why-depends`regnat
`nix why-depends` is piping its output into a pager by default. However the pager was only started after the first path is printed, causing it to be excluded from the pager output. (Actually the pager was started *inside* the recursive function that was printing the dependency chain, so a new instance was started at each level. It’s a little miracle that it worked at all). Fix #5911
2022-01-13optimize unescapeStrpennae
mainly to avoid an allocation and a copy of a string that can be modified in place (ever since EvalState holds on to the buffer, not the generated parser itself). # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 570.4 ms ± 2.8 ms [User: 561.3 ms, System: 8.6 ms] Range (min … max): 564.6 ms … 578.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 375.4 ms ± 1.3 ms [User: 343.2 ms, System: 31.7 ms] Range (min … max): 373.4 ms … 378.2 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.925 s ± 0.006 s [User: 2.704 s, System: 0.219 s] Range (min … max): 2.910 s … 2.942 s 50 runs
2022-01-13avoid copies of parser input datapennae
when given a string yacc will copy the entire input to a newly allocated location so that it can add a second terminating NUL byte. since the parser is a very internal thing to EvalState we can ensure that having two terminating NUL bytes is always possible without copying, and have the parser itself merely check that the expected NULs are present. # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs
2022-01-13avoid allocations in SymbolTable::createpennae
speeds up parsing by ~3%, system builds by a bit more than 1% # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 574.7 ms ± 2.8 ms [User: 566.3 ms, System: 8.0 ms] Range (min … max): 569.2 ms … 580.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 394.4 ms ± 0.8 ms [User: 361.8 ms, System: 32.3 ms] Range (min … max): 392.7 ms … 395.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.976 s ± 0.005 s [User: 2.757 s, System: 0.218 s] Range (min … max): 2.966 s … 2.990 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs
2022-01-13don't strdup tokens in the lexerpennae
every stringy token the lexer returns is turned into a Symbol and not used further, so we don't have to strdup. using a string_view is sufficient, but due to limitations of the current parser we have to use a POD type that holds the same information. gives ~2% on system build, 6% on search, 8% on parsing alone # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 610.6 ms ± 2.4 ms [User: 602.5 ms, System: 7.8 ms] Range (min … max): 606.6 ms … 617.3 ms 50 runs Benchmark 2: nix eval -f hackage-packages.nix Time (mean ± σ): 430.1 ms ± 1.4 ms [User: 393.1 ms, System: 36.7 ms] Range (min … max): 428.2 ms … 434.2 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 3.032 s ± 0.005 s [User: 2.808 s, System: 0.223 s] Range (min … max): 3.023 s … 3.041 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 574.7 ms ± 2.8 ms [User: 566.3 ms, System: 8.0 ms] Range (min … max): 569.2 ms … 580.7 ms 50 runs Benchmark 2: nix eval -f hackage-packages.nix Time (mean ± σ): 394.4 ms ± 0.8 ms [User: 361.8 ms, System: 32.3 ms] Range (min … max): 392.7 ms … 395.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.976 s ± 0.005 s [User: 2.757 s, System: 0.218 s] Range (min … max): 2.966 s … 2.990 s 50 runs
2022-01-13move strings in derivationStrictpennae
the temporary will be discarded anyway, so we can move out of it and save many small allocations and copies.
2022-01-13avoid string copies in attrNames sort comparisonpennae
symbols can also be cast to string_view, which compares the same but doesn't require a copy of both symbol names on every comparison.
2022-01-13cache more often-used symbols for primopspennae
there's a few symbols in primops we can create once and pick them out of EvalState afterwards instead of creating them every time we need them. this gives almost 1% speedup to an uncached nix search.
2022-01-13use more string_view in utilspennae
there's a couple places that can be easily converted from using strings to using string_views instead. gives a slight (~1%) boost to system eval. # before nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.946 s ± 0.026 s [User: 2.655 s, System: 0.209 s] Range (min … max): 2.905 s … 2.995 s 20 runs # after Time (mean ± σ): 2.928 s ± 0.024 s [User: 2.638 s, System: 0.211 s] Range (min … max): 2.893 s … 2.970 s 20 runs
2022-01-12Merge pull request #5887 from pennae/avoid-streamsThéophane Hufschmitt
avoid std::?stream overhead when it's not helpful
2022-01-12use boost::lexical_cast for string2*pennae
this avoids one copy from `s` into `str`, and possibly another copy needed to construct `s` at the call site. lexical_cast is also more efficient in general.
2022-01-12optimize ExprConcatStrings::evalpennae
constructing an ostringstream for non-string concats (like integer addition) is a small constant cost that we can avoid. for string concats we can keep all the string temporaries we get from coerceToString and concatenate them in one go, which saves a lot of intermediate temporaries and copies in ostringstream. we can also avoid copying the concatenated string again by directly allocating it in GC memory and moving ownership of the concatenated string into the target value. saves about 2% on system eval. before: Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.837 s ± 0.031 s [User: 2.562 s, System: 0.191 s] Range (min … max): 2.796 s … 2.892 s 20 runs after: Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.790 s ± 0.035 s [User: 2.532 s, System: 0.187 s] Range (min … max): 2.722 s … 2.836 s 20 runs
2022-01-11Merge pull request #5883 from fzakaria/faridzakaria/check-metaThéophane Hufschmitt
Check for `--meta` before emitting it for `--json`
2022-01-11Merge pull request #5898 from layus/repair-path-linksEelco Dolstra
Make --repair-path also repair corrupt optimised links
2022-01-11Prefer RepairFlag over bool when applicableGuillaume Maudoux
2022-01-11Make --repair-path also repair corrupt optimised linksGuillaume Maudoux
There already existed a smoke test for the link content length, but it appears that there exists some corruptions pernicious enough to replace the file content with zeros, and keeping the same length. --repair-path now goes as far as checking the content of the link, making it true to its name and actually repairing the path for such coruption cases.
2022-01-10Merge branch 'readfile' of https://github.com/abbradar/nixEelco Dolstra
2022-01-10'target' points to 'source'Eelco Dolstra
2022-01-10Merge branch 'recursive-nix-fix' of https://github.com/L-as/nixEelco Dolstra
2022-01-09builtins.readFile: Propagate path contextNikolay Amiantov
Co-authored-by: Shea Levy <shea@shealevy.com>
2022-01-07dump(): Use emplace()Eelco Dolstra
2022-01-07isInDir() / isDirOrInDir(): Use std::string_viewEelco Dolstra
2022-01-07DerivationGoal::loadDerivation(): Don't use derivationFromPath()Eelco Dolstra
This causes a recursive call to ensurePath(), which is not a good idea.
2022-01-07avoid ostream sentries per json string characterpennae
we don't have to create an ostream sentry object for every character of a JSON string we write. format a bunch of characters and flush them to the stream all at once instead. this doesn't affect small numbers of string characters, but larger numbers of total JSON string characters written gain a lot. at 1MB of total string written we gain almost 30%, at 16MB it's almost a factor of 3x. large numbers of JSON string characters do occur naturally in a nixos system evaluation to generate documentation (though this is now somewhat mitigated by caching the largest part of nixos option docs). benchmarked with hyperfine 'nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) {e})"' --warmup 1 -L e 1,4,256,4096,65536 before: Benchmark 1: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 1)" Time (mean ± σ): 12.5 ms ± 0.2 ms [User: 9.2 ms, System: 4.0 ms] Range (min … max): 11.9 ms … 13.1 ms 223 runs Benchmark 2: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4)" Time (mean ± σ): 12.5 ms ± 0.2 ms [User: 9.3 ms, System: 3.8 ms] Range (min … max): 11.9 ms … 13.2 ms 220 runs Benchmark 3: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 256)" Time (mean ± σ): 13.2 ms ± 0.3 ms [User: 9.8 ms, System: 4.0 ms] Range (min … max): 12.6 ms … 14.3 ms 205 runs Benchmark 4: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4096)" Time (mean ± σ): 24.0 ms ± 0.4 ms [User: 19.4 ms, System: 5.2 ms] Range (min … max): 22.7 ms … 25.8 ms 119 runs Benchmark 5: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 65536)" Time (mean ± σ): 196.0 ms ± 3.7 ms [User: 171.2 ms, System: 25.8 ms] Range (min … max): 190.6 ms … 201.5 ms 14 runs after: Benchmark 1: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 1)" Time (mean ± σ): 12.4 ms ± 0.3 ms [User: 9.1 ms, System: 4.0 ms] Range (min … max): 11.7 ms … 13.3 ms 204 runs Benchmark 2: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4)" Time (mean ± σ): 12.4 ms ± 0.2 ms [User: 9.2 ms, System: 3.9 ms] Range (min … max): 11.8 ms … 13.0 ms 214 runs Benchmark 3: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 256)" Time (mean ± σ): 12.6 ms ± 0.2 ms [User: 9.5 ms, System: 3.8 ms] Range (min … max): 12.1 ms … 13.3 ms 209 runs Benchmark 4: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4096)" Time (mean ± σ): 15.9 ms ± 0.2 ms [User: 11.4 ms, System: 5.1 ms] Range (min … max): 15.2 ms … 16.4 ms 171 runs Benchmark 5: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 65536)" Time (mean ± σ): 69.0 ms ± 0.9 ms [User: 44.3 ms, System: 25.3 ms] Range (min … max): 67.2 ms … 70.9 ms 42 runs
2022-01-06Check for `--meta` before emitting it for `--json`Farid Zakaria
Check that the meta flag is present when emitting JSON query information for `nix-env`. fixes #5882
2022-01-06Merge branch 'install-nlohmann-json-headers' of ↵Eelco Dolstra
https://github.com/hercules-ci/nix
2022-01-06Merge pull request #5878 from fzakaria/faridzakaria/json-outputsEelco Dolstra
Add outputs to JSON query
2022-01-06Merge pull request #5546 from NixOS/allow-paths-in-flake-local-optionsEelco Dolstra
allow paths in flake local options
2022-01-06Add outputs to JSON queryFarid Zakaria
Emit output information when printing JSON information and `--out-paths` is requested. fixes #5877
2022-01-06Merge pull request #5749 from GavinRay97/patch-1Domen Kožar
Better diagnostics if no valid signature found
2022-01-04Remove EvalState::mkAttrs()Eelco Dolstra
2022-01-04Move empty attrset optimisationEelco Dolstra
2022-01-04Turn mkString(Symbol) into a methodEelco Dolstra
2022-01-04Remove non-method mkPath()Eelco Dolstra
2022-01-04Remove non-method mk<X> functionsEelco Dolstra
2022-01-04Remove non-method mkString()Eelco Dolstra
2022-01-04Ensure that attrsets are sortedEelco Dolstra
Previously you had to remember to call value->attrs->sort() after populating value->attrs. Now there is a BindingsBuilder helper that wraps Bindings and ensures that sort() is called before you can use it.
2022-01-03add zipAttrsWith primoppennae
nixpkgs can save a good bit of eval memory with this primop. zipAttrsWith is used quite a bit around nixpkgs (eg in the form of recursiveUpdate), but the most costly application for this primop is in the module system. it improves the implementation of zipAttrsWith from nixpkgs by not checking an attribute multiple times if it occurs more than once in the input list, allocates less values and set elements, and just avoids many a temporary object in general. nixpkgs has a more generic version of this operation, zipAttrsWithNames, but this version is only used once so isn't suitable for being the base of a new primop. if it were to be used more we should add a second primop instead.
2022-01-03Merge pull request #5839 from tweag/balsoft/yet-another-follows-bugfixEelco Dolstra
flake.cc: computeLocks: Only verify overrides when they could change
2022-01-03Merge pull request #5840 from tweag/balsoft/nix-repl-show-traceEelco Dolstra
nix repl: fix --show-trace and add the ability to set trace display
2021-12-29Fix IFD with chroot storeSebastian Ullrich
2021-12-29Add ability to toggle show-trace from within the replAlexander Bantyev
2021-12-28Sort attrs from tables in fromTOMLLily Foster
This was dropped in 10a8b5d for the migration from cpptoml to toml11 but seems to be necessary for the attrsets to work correctly. Fixes #5833
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 22:19:21 +0000