aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-04-29Test fetchGit with non-'master' remote repoKjetil Orbekk
2022-04-29Resolve reference for remote repositoryKjetil Orbekk
Resolves the HEAD reference from the remote repository instead of assuming "master".
2022-04-28Merge pull request #6461 from edolstra/libcxxEelco Dolstra
Fix libcxx build
2022-04-28Fix libcxx buildEelco Dolstra
Fixes #6458.
2022-04-28Add libcxxStdenv devshellEelco Dolstra
2022-04-28Shut up clang warningEelco Dolstra
2022-04-28Merge pull request #6459 from edolstra/fix-out-pathsEelco Dolstra
Fix passing $OUT_PATHS to the post-build hook
2022-04-28Fix passing $OUT_PATHS to the post-build hookEelco Dolstra
Fixes #6446.
2022-04-26Merge pull request #6444 from midchildan/feat/zsh-runhelpEelco Dolstra
feat: add integration with zsh's run-help
2022-04-27refactor: don't hardcode nix subcommands in run-help-nixmidchildan
2022-04-26Merge pull request #6447 from edolstra/eval-cache-symbolsEelco Dolstra
EvalCache: Use Symbol in more places
2022-04-26Merge pull request #6440 from Ma27/fix-nix-with-latest-gitThéophane Hufschmitt
libfetchers/git: hardcode `--git-dir`
2022-04-26EvalCache AttrKey: Use Symbol instead of std::stringEelco Dolstra
2022-04-26EvalCache: Revert to using symbols in getAttr()Eelco Dolstra
2022-04-26Don't pass Symbol by referenceEelco Dolstra
Since Symbol is just an integer, passing it by const reference is never advantageous.
2022-04-26Merge branch 'symbolidx-symbol' of https://github.com/pennae/nixEelco Dolstra
2022-04-25rename SymbolIdx -> Symbol, Symbol -> SymbolStrpennae
after #6218 `Symbol` no longer confers a uniqueness invariant on the string it wraps, it is now possible to create multiple symbols that compare equal but whose string contents have different addresses. this guarantee is now only provided by `SymbolIdx`, leaving `Symbol` only as a string wrapper that knows about the intricacies of how symbols need to be formatted for output. this change renames `SymbolIdx` to `Symbol` to restore the previous semantics of `Symbol` to that name. we also keep the wrapper type and rename it to `SymbolStr` instead of returning plain strings from lookups into the symbol table because symbols are formatted for output in many places. theoretically we do not need `SymbolStr`, only a function that formats a string for output as a symbol, but having to wrap every symbol that appears in a message into eg `formatSymbol()` is error-prone and inconvient.
2022-04-25fix: typo make-content-addressable -> make-content-addressedmidchildan
2022-04-25feat: add integration with zsh's run-helpmidchildan
2022-04-24libfetchers/git: fix for nixos-rebuildMaximilian Bosch
The `--git-dir=` must be `.` in some cases (for cached repos that are "bare" repos in `~/.cache/nix/gitv3`). With this fix we can add `--git-dir` to each `git`-invokation needed for `nixos-rebuild`.
2022-04-23libfetchers/git: hardcode `--git-dir`Maximilian Bosch
To demonstrate the problem: * You need a `git` at 2.33.3 in your $PATH * An expression like this in a git repository: ``` nix { outputs = { self, nixpkgs }: { packages.foo.x86_64-linux = with nixpkgs.legacyPackages.x86_64-linux; runCommand "snens" { } '' echo ${(builtins.fetchGit ./.).lastModifiedDate} > $out ''; }; } ``` Now, when instantiating the package via `builtins.getFlake`, it fails on Nix 2.7 like this: $ nix-instantiate -E '(builtins.getFlake "'"$(pwd)"'").packages.foo.x86_64-linux' fatal: unsafe repository ('/nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source' is owned by someone else) To add an exception for this directory, call: git config --global --add safe.directory /nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source error: program 'git' failed with exit code 128 (use '--show-trace' to show detailed location information) This breaks e.g. `nixops`-deployments using flakes with similar expressions as shown above. The cause for this is that `git(1)` tries to find the highest `.git`-directory in the directory tree and if it finds a such a directory, but with another owning user (root vs. the user who evaluates the expression), it fails as above. This was changed recently to fix CVE-2022-24765[1]. By explicitly specifying `--git-dir`, Git assumes to be in the top-level directory and doesn't attempt to look for a `.git`-directory in the parent directories and thus the code-path leading to said error is never reached. [1] https://lore.kernel.org/git/xmqqv8veb5i6.fsf@gitster.g/
2022-04-22Merge pull request #6259 from Artturin/nixbuildaddprintstorepathsThéophane Hufschmitt
nix build: add --print-out-paths flag
2022-04-22Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepathsThéophane Hufschmitt
2022-04-22Merge pull request #6437 from NixOS/fix-darwin-buildThéophane Hufschmitt
Fix the darwin build
2022-04-22Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepathsThéophane Hufschmitt
2022-04-22Fix the darwin buildThéophane Hufschmitt
Looks like the auto-merge is indeed quite broken and merges even when the CI fails
2022-04-22Merge pull request #6218 from pennae/pos-symbol-tablesThéophane Hufschmitt
reduce the size of Attr from 3 pointers to 2 on 64 bit machines
2022-04-22Add some tests for ChunkedVectorThéophane Hufschmitt
2022-04-22Move ChunkedVector to its own headerThéophane Hufschmitt
2022-04-22Merge pull request #6436 from flox/tofile_allowThéophane Hufschmitt
fix: builtins.toFile adds path to allowedPaths
2022-04-21fix: builtins.toFile adds path to allowedPathsTom Bereknyei
The produced path is then allowed be imported or utilized elsewhere: ``` assert (43 == import (builtins.toFile "source" "43")); "good" ``` This will still fail on write-only stores.
2022-04-21remove pos<T>pennae
it's no longer needed now that positions aren't really pointers any more.
2022-04-21shrink Attr by 8 bytes on 64bit machinespennae
with position and symbol tables in place we can now shrink Attr by a full pointer with some simple field reordering. since Attr is a very hot struct this has substantial impact on memory use, decreasing GC allocations and heap size by 10-15% each. we also get a ~15% performance improvement due to reduced GC loading. pure parsing has taken a hit over the branch base because positions are now slightly more expensive to create, but overall we get a noticeable improvement. before (on memory-friendliness): Benchmark 1: nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 6.960 s ± 0.028 s [User: 5.832 s, System: 0.897 s] Range (min … max): 6.886 s … 7.005 s 20 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 328.1 ms ± 1.7 ms [User: 295.8 ms, System: 32.2 ms] Range (min … max): 324.9 ms … 331.2 ms 20 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.688 s ± 0.029 s [User: 2.365 s, System: 0.238 s] Range (min … max): 2.642 s … 2.742 s 20 runs after: Benchmark 1: nix search --no-eval-cache --offline ../nixpkgs hello Time (mean ± σ): 6.902 s ± 0.039 s [User: 5.844 s, System: 0.783 s] Range (min … max): 6.820 s … 6.956 s 20 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 330.7 ms ± 2.2 ms [User: 300.6 ms, System: 30.0 ms] Range (min … max): 327.5 ms … 334.5 ms 20 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.330 s ± 0.027 s [User: 2.040 s, System: 0.234 s] Range (min … max): 2.272 s … 2.383 s 20 runs
2022-04-21store Symbols in a table as well, like positionspennae
this slightly increases the amount of memory used for any given symbol, but this increase is more than made up for if the symbol is referenced more than once in the EvalState that holds it. on average every symbol should be referenced at least twice (once to introduce a binding, once to use it), so we expect no increase in memory on average. symbol tables are limited to 2³² entries like position tables, and similar arguments apply to why overflow is not likely: 2³² symbols would require as many string instances (at 24 bytes each) and map entries (at 24 bytes or more each, assuming that the map holds on average at most one item per bucket as the docs say). a full symbol table would require at least 192GB of memory just for symbols, which is well out of reach. (an ofborg eval of nixpks today creates less than a million symbols!)
2022-04-21don't use Symbol in Pos to represent a pathpennae
PosTable deduplicates origin information, so using symbols for paths is no longer necessary. moving away from path Symbols also reduces the usage of symbols for things that are not keys in attribute sets, which will become important in the future when we turn symbols into indices as well.
2022-04-21replace most Pos objects/ptrs with indexes into a position tablepennae
Pos objects are somewhat wasteful as they duplicate the origin file name and input type for each object. on files that produce more than one Pos when parsed this a sizeable waste of memory (one pointer per Pos). the same goes for ptr<Pos> on 64 bit machines: parsing enough source to require 8 bytes to locate a position would need at least 8GB of input and 64GB of expression memory. it's not likely that we'll hit that any time soon, so we can use a uint32_t index to locate positions instead.
2022-04-21make throw*Error member functions of EvalStatepennae
when we introduce position and symbol tables we'll need to do lookups to turn indices into those tables into actual positions/symbols. having the error functions as members of EvalState will avoid a lot of churn for adding lookups into the tables for each caller.
2022-04-21don't use full Pos for findPackageFilename/editorForpennae
only file and line of the returned position were ever used, it wasn't actually used a position. as such we may as well use a path+int pair for only those two values and remove a use of Pos that would not work well with a position table.
2022-04-21remove Bindings::needpennae
a future commit will remove the ability to convert the symbol type used in bindings to strings. since we only have two users we can inline the error check.
2022-04-21remove Symbol::emptypennae
the only use of this function is to determine whether a lambda has a non-set formal, but this use is arguably better served by Symbol::set and using a non-Symbol instead of an empty symbol in the parser when no such formal is present.
2022-04-21turn primop names into stringspennae
we don't *need* symbols here. the only advantage they have over strings is making call-counting slightly faster, but that's a diagnostic feature and thus needn't be optimized. this also fixes a move bug that previously didn't show up: PrimOp structs were accessed after being moved from, which technically invalidates them. previously the names remained valid because Symbol copies on move, but strings are invalidated. we now copy the entire primop struct instead of moving since primop registration happen once and are not performance-sensitive.
2022-04-21Rename fmt test -> hilteEelco Dolstra
2022-04-21Fix fmt testEelco Dolstra
2022-04-21Merge pull request #6433 from edolstra/hiliteEelco Dolstra
Move hiliteMatches into a separate header
2022-04-21Move hiliteMatches into a separate headerEelco Dolstra
This is mostly so that we don't #include <regex> everywhere (which adds quite a bit of compilation time).
2022-04-21Merge pull request #6416 from moduon/feat-opensshEelco Dolstra
feat: include openssh in docker image
2022-04-21Merge pull request #6431 from NixOS/unbreak-my-buildEelco Dolstra
Make the default SQLiteError constructor public
2022-04-21Merge pull request #5479 from NixOS/selfref-caThéophane Hufschmitt
Fix the removal of ca-induced self-references
2022-04-21Merge pull request #6324 from trofi/selfref-ca-indexThéophane Hufschmitt
ca: add sqlite index on `RealisationsRefs(realisationReference)`
2022-04-21Make sure to delete all the realisation refsregnat
Deleting just one will only work in the test cases where I didn’t bother creating too many of them :p