aboutsummaryrefslogtreecommitdiff
path: root/src/libutil
AgeCommit message (Collapse)Author
2024-03-25Merge pull request #8817 from iFreilicht/flake-update-lock-overhaulThéophane Hufschmitt
Overhaul `nix flake update` and `nix flake lock` UX (cherry picked from commit 12a0ae73dbb37becefa5a442eb4532ff0de9ce65) Change-Id: Iff3b4f4235ebb1948ec612036b39ab29e4ca22b2
2024-03-25Overhaul completions, redo #6693 (#8131)John Ericson
As I complained in https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a comment on the wrong PR, sorry again!), #6693 introduced a second completions mechanism to fix a bug. Having two completion mechanisms isn't so nice. As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef` to `std::string` when collecting flake refs. Now it is `FlakeRefs` again. The underlying issue that sought to work around was that completion of arguments not at the end can still benefit from the information from latter arguments. To fix this better, we rip out that change and simply defer all completion processing until after all the (regular, already-complete) arguments have been passed. In addition, I noticed the original completion logic used some global variables. I do not like global variables, because even if they save lines of code, they also obfuscate the architecture of the code. I got rid of them moved them to a new `RootArgs` class, which now has `parseCmdline` instead of `Args`. The idea is that we have many argument parsers from subcommands and what-not, but only one root args that owns the other per actual parsing invocation. The state that was global is now part of the root args instead. This did, admittedly, add a bunch of new code. And I do feel bad about that. So I went and added a lot of API docs to try to at least make the current state of things clear to the next person. -- This is needed for RFC 134 (tracking issue #7868). It was very hard to modularize `Installable` parsing when there were two completion arguments. I wouldn't go as far as to say it is *easy* now, but at least it is less hard (and the completions test finally passed). Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Change-Id: If18cd5be78da4a70635e3fdcac6326dbfeea71a5 (cherry picked from commit 67eb37c1d0de28160cd25376e51d1ec1b1c8305b)
2024-03-23libutil: don't memset 64k in drainFDeldritch horrors
this is not needed and introduces a bunch of memset calls, making up for 3% of valgrind cycle estimation *alone*. real-world impact is a lot lower on our test machine, but we suspect that less powerful machines would see an impact from dropping this. Change-Id: Iad10e9d556e64fdeb0bee0059a4e52520058d11e
2024-03-22build: optionally build and install with mesonQyriad
This commit adds several meson.build, which successfully build and install Lix executables, libraries, and headers. Meson does not yet build docs, Perl bindings, or run tests, which will be added in following commits. As such, this commit does not remove the existing build system, or make it the default, and also as such, this commit has several FIXMEs and TODOs as notes for what should be done before the existing autoconf + make buildsystem can be removed and Meson made the default. This commit does not modify any source files. A Meson-enabled build is also added as a Hydra job, and to `nix flake check`. Change-Id: I667c8685b13b7bab91e281053f807a11616ae3d4
2024-03-19libutil: remove exception handling workingness checkeldritch horrors
within lix itself this problem is caught by the test suite. outside of lix itself three cases can be had: either the problem is fully inside lix libs, fully inside user code, or it exists at the boundary. the first is caught by the test suite, the second isn't caught at all, and the third is something lix should not be responsible for. Change-Id: I95aa35d8cb6f0ef5816a2941c467bc0c15916063
2024-03-18Merge changes I72c945ca,I2138bb4d,Ib96749f3 into mainjade
* changes: Release notes for builtins.nixVersion change un-nixes ur lix, a little issue importer: list issues that are *not* closed when finding existing issues
2024-03-18un-nixes ur lix, a littleJade Lovelace
I didn't really go attack the docs because we need to pull a bunch of PRs. I went looking for strings in the code that called lix nix. Change-Id: I2138bb4dd239096bc530946b281db7f875195b39
2024-03-18libutil: make AutoCloseFD a better resourceeldritch horrors
add a reset() method to close the wrapped fd instead of assigning magic constants. also make the from-fd constructor explicit so you can't accidentally assign the *wrong* magic constant, or even an unrelated integer that also just happens to be an fd by pure chance. Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-18Merge "Delete hasPrefix and hasSuffix from the codebase" into mainjade
2024-03-18match line endings used by parser and error reportseldritch horrors
the parser treats a plain \r as a newline, error reports do not. this can lead to interesting divergences if anything makes use of this feature, with error reports pointing to wrong locations in the input (or even outside the input altogether). (cherry picked from commit 2be6b143289e5479cc4a2667bb84e879116c2447) Change-Id: Ieb7f7655bac8cb0cf5734c60bd41723388f2973c
2024-03-18libutil: remove vforkeldritch horrors
vfork confers a large performance advantage over fork, measured locally at 16µs per vfork agains 90µs per fork. however nix *almost always* follows a vfork up with an execve-family call, melting the performance advantage from 6x to only 15%. in most of those cases it's doing things that are undefined behavior (like manipulating the heap, or even throwing exceptions and trashing the parent process stack). most notably the one place that could benefit from the vfork performance improvement is linux derivation sandbox setup—which doesn't use vfork. Change-Id: I2037b7384d5a4ca24da219a569e1b1f39531410e
2024-03-17Delete hasPrefix and hasSuffix from the codebaseJade Lovelace
These now have equivalents in the standard lib in C++20. This change was performed with a custom clang-tidy check which I will submit later. Executed like so: ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
2024-03-11add automated usage mode to the replJade Lovelace
This is definitely not a stable thing, but it does feel slightly crimes to put it as an experimental feature. Shrug, up for bikeshedding. Change-Id: I6ef176e3dee6fb1cac9c0a7a60d553a2c63ea728
2024-03-11finally.hh: delete copy constructor which is a bad ideaJade Lovelace
Change-Id: I6d0b5736893c44bddc6f5789b452b434f8671b9b
2024-03-11Add box_ptr: nonnull unique_ptr with value semanticsJade Lovelace
This solves the problem of collections of boxed subclasses with virtual dispatch, which should still be treated as values, since the indirection is only there due to the virtual dispatch. Change-Id: I368daedd3f31298e99c6e56a15606337a55494c6
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-09Merge pull request #10066 from 9999years/print-all-frameseldritch horrors
Do not skip any stack frames when `--show-trace` is given (cherry picked from commit 0b47783d0a879875d558f0b56e49584f25ceb2d0) Change-Id: Ia0f18266dbcf97543110110c655c219c7a3e3270
2024-03-09`:quit` in the debugger should quit the whole programeldritch horrors
(cherry picked from commit 2a8fe9a93837733e9dd9ed5c078734a35b203e14) Change-Id: I71dadfef6b24d9272b206e9e2c408040559d8a1c
2024-03-09Merge pull request #9925 from 9999years/fmt-cleanupeldritch horrors
Cleanup `fmt.hh` (cherry picked from commit 47a1dbb4b8e7913cbb9b4d604728b912e76e4ca0) Change-Id: Id076a45cb39652f437fe3f8bda10c310a9894777
2024-03-09Remove `EXCEPTION_NEEDS_THROW_SPEC`eldritch horrors
We're on C++ 20 now, we don't need this (cherry picked from commit faaccecbc82d98288582bdc8ca96991796561371) Change-Id: I172fa336107fd18b1aac2262309682e0d7065d07
2024-03-09libexpr: Support structured error classeseldritch horrors
While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`. (cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732) Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
2024-03-09Merge pull request #9753 from 9999years/print-value-on-type-erroreldritch horrors
Print the value in `value is X while a Y is expected` error (cherry picked from commit 5f72a97092da6af28a7d2b2a50d74e9d34fae7e1) Change-Id: Idb4bc903ae59a0f5b6fb3b1da4d47970fe0a6efe
2024-03-09Unify and refactor value printingeldritch horrors
Previously, there were two mostly-identical value printers -- one in `libexpr/eval.cc` (which didn't force values) and one in `libcmd/repl.cc` (which did force values and also printed ANSI color codes). This PR unifies both of these printers into `print.cc` and provides a `PrintOptions` struct for controlling the output, which allows for toggling whether values are forced, whether repeated values are tracked, and whether ANSI color codes are displayed. Additionally, `PrintOptions` allows tuning the maximum number of attributes, list items, and bytes in a string that will be displayed; this makes it ideal for contexts where printing too much output (e.g. all of Nixpkgs) is distracting. (As requested by @roberth in https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735) Please read the tests for example output. Future work: - It would be nice to provide this function as a builtin, perhaps `builtins.toStringDebug` -- a printing function that never fails would be useful when debugging Nix code. - It would be nice to support customizing `PrintOptions` members on the command line, e.g. `--option to-string-max-attrs 1000`. (cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, ) === Restore ambiguous value printer for `nix-instantiate` The Nix team has requested that this output format remain unchanged. I've added a warning to the man page explaining that `nix-instantiate --eval` output will not parse correctly in many situations. (cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79) Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
2024-03-07Merge pull request #9787 from obsidiansystems/bind-proc-syserroreldritch horrors
`bind`: give same treatment as `connect` in #8544, dedup (cherry picked from commit 28674247ec792a981741198abc190a71bb254b82) Change-Id: I1ac5fc43fa10ec5f37a226730c3d84033fdbfd52
2024-03-07Merge pull request #8544 from edolstra/handle-missing-gc-socketeldritch horrors
LocalStore: :addTempRoot(): Handle ENOENT (cherry picked from commit 7115edc85af060ef235ac0270245ab46cc828f7c) Change-Id: Ie6b1596049c3fde09b98f2f0727899f98e48e6b1
2024-03-07Merge pull request #9985 from alois31/symlink-resolutioneldritch horrors
Restore `builtins.pathExists` behavior on broken symlinks (cherry picked from commit d53c8901ef7f2033855dd99063522e3d56a19dab) === note that this variant differs markedly from the source commit because we haven't endured quite as much lazy trees. Change-Id: I0facf282f21fe0db4134be5c65a8368c1b3a06fc
2024-03-06Merge "fix: bounds check result in getMaxCPU" into mainjade
2024-03-06fix: bounds check result in getMaxCPUJade Lovelace
Upstream-Bug: https://github.com/NixOS/nix/issues/9725 Upstream-PR: https://github.com/NixOS/nix/pull/10172 Change-Id: I2b8e8b15ee28951be98e5cbe3ccea5b9c8de5994
2024-03-07Merge pull request #9934 from nmeum/absPath-out-of-boundseldritch horrors
absPath: Explicitly check if path is empty before accessing it (cherry picked from commit 6ec08b85f607852eb6f976c1392c4917d0a53787) Change-Id: Ieeb53fb65d0e334e6017ceb3a48b3b6ae1047843
2024-03-07Copy the output of fixed-output derivations before registering themPuck Meerburg
It is possible to exfiltrate a file descriptor out of the build sandbox of FODs, and use it to modify the store path after it has been registered. To avoid that issue, don't register the output of the build, but a copy of it (that will be free of any leaked file descriptor). Test that we can't leverage abstract unix domain sockets to leak file descriptors out of the sandbox and modify the path after it has been registered. (cherry picked from commit 2dadfeb690e7f4b8f97298e29791d202fdba5ca6) (tests cherry picked from commit c854ae5b3078ac5d99fa75fe148005044809e18c) Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Theophane Hufschmitt <theophane.hufschmitt@tweag.io> Co-authored-by: Tom Bereknyei <tomberek@gmail.com> Change-Id: I87cd58f1c0a4f7b7a610d354206b33301e47b1a4
2024-03-05Merge pull request #9634 from 9999years/combine-abstract-pos-and-poseldritch horrors
Combine `AbstractPos`, `PosAdapter`, and `Pos` (cherry picked from commit 113499d16fc87d53b73fb62fe6242154909756ed) === this is a bit cursed because originally it was based on InputAccessor code that we don't have and moved/patched features we likewise don't have (fetchToStore caching, all the individual accessors, ContentAddressMethod). the commit is adjusted accordingly to match (remove caching, ignore accessors, use FileIngestionMethod). note that `state.rootPath . CanonPath == abs` and computeStorePathForPath works relative to cwd, so the slight rewrite in the moved fetchToStore is legal. Change-Id: I05fd340c273f0bcc8ffabfebdc4a88b98083bce5
2024-03-05Merge pull request #9860 from 9999years/set-stack-darwineldritch horrors
Increase stack size on macOS as well as Linux (cherry picked from commit efb91d5979a625d5c50558aeabfd24e802ed9173, 4a2444b3f32a2f5d42c4d65302793b987d1ac667) Change-Id: Ieb72283c61bb9e360683f531d6635697b293c313
2024-03-04Ban building Nix with NDEBUGeldritch horrors
When reviewing old PRs, I found that #9997 adds some code to ensure one particular assert is always present. But, removing asserts isn't something we do in our own release builds either in the flake here or in nixpkgs, and is plainly a bad idea that increases support burden, especially if other distros make bad choices of build flags in their Nix packaging. For context, the assert macro in the C standard is defined to do nothing if NDEBUG is set. There is no way in our build system to set -DNDEBUG without manually adding it to CFLAGS, so this is simply a configuration we do not use. Let's ban it at compile time. I put this preprocessor directive in src/libutil.cc because it is not obvious where else to put it, and it seems like the most logical file since you are not getting a usable nix without it. Upstream-PR: https://github.com/NixOS/nix/pull/10126 Original-Change-Id: I513cceaac1371decb3d96231e6ef9181c910c218 Change-Id: I531a51f6348a746e8e41d88203b08f614898356c
2024-03-04Merge pull request #9924 from 9999years/rename-yellowtxteldritch horrors
Rename `yellowtxt` -> `magentatxt` (cherry picked from commit fb78a99e04206e7b1df84a362bb87d3300b41855) Change-Id: I9ade553d9f499e6713aeff3463c9a653a880a051
2024-03-04Merge pull request #9798 from edolstra/remote-store-eofeldritch horrors
Print a more helpful message if the daemon crashes (cherry picked from commit 32706b14a7531c2c21b9f96da083a540a0031ec4) Change-Id: Ief7c465bca7666e2b7e7c9d1dd0c01c5f9014146
2024-03-04Merge pull request #9687 from edolstra/withFramedSink-ctrl-c-hangeldritch horrors
withFramedSink(): Receive interrupts on the stderr thread (cherry picked from commit 965cfe96886c988c3aa94bfc7fefdd37325f4536) Change-Id: I8320a96957c01ec0e3450d1b3ae38a3baff78d49
2024-03-04Merge pull request #9673 from pennae/drv-parse-optseldritch horrors
optimize derivation parsing (cherry picked from commit 3511430902941f0f26dc71313a54bb5096f57305) Change-Id: I00f76dcd464a5811944613731501af504b6e8c29
2024-03-04Merge pull request #9617 from 9999years/stack-overflow-segfaulteldritch horrors
Fix segfault on infinite recursion in some cases (cherry picked from commit bf1b294bd81ca76c5ec9fe3ecd52196bf52a8300) Change-Id: Id137541426ec8536567835953fccf986a3aebf16
2024-03-04Merge pull request #9582 from pennae/misc-optseldritch horrors
a packet of small optimizations (cherry picked from commit ee439734e924eb337a869ff2e48aff8b989198bc) Change-Id: I125d870710750a32a0dece48f39a3e9132b0d023
2024-03-04Merge pull request #9621 from blaggacao/fix/too-restrictive-branch-regex-mastereldritch horrors
fix: valid branch name (cherry picked from commit d2016c6b59f4a5223a18c92c817d61c448ce39ab) Change-Id: Ic346739b91bf1b10515c9ff3b3e180a64afe6343
2024-03-04Merge pull request #9619 from 9999years/remove-blank-lines-in-errorseldritch horrors
Remove some blank lines from stack traces (cherry picked from commit cea83544362bd6426e9bc0b7dd0d611c19b0e3fd) Change-Id: Ic1af8b09e9994d3c69fd3b37ae47a4bb786a15a1
2024-03-04Merge pull request #9557 from bryanhonof/bryanhonof.fix-apple-double-shenaniganseldritch horrors
Add option to libarchive so it behaves correctly (cherry picked from commit c3827ff6348a4d5199eaddf8dbc2ca2e2ef46ec5) Change-Id: Ib0f928851093f4c644bac071d1c8f8aeec803198
2024-03-04Merge pull request #9233 from bouk/bouk/apply-config-innereldritch horrors
config: add included files into parsedContents before applying (cherry picked from commit 82359eba6b692691ef08a71196ef25a61bc4d3d3) Change-Id: Idde3177010fec7b8bafe6088c3c23d5caf491845
2024-03-04fix: `nlohmann::adl_serializer` for `std::optional` (#9147)eldritch horrors
This allows templates such as `NLOHMANN_DEFINE_TYPE_*` templates and other generators with things like `std::vector<std::optional<T>>`. Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems> (cherry picked from commit 02bd821f2e71372d31bbe6700bd68086cc2ee70a) Change-Id: I8b0ebcf2af4226610dadd565962f2d2327415a03
2024-03-04Merge pull request #9299 from tfc/config-improvementseldritch horrors
Improvements in src/libutil/config.* (cherry picked from commit dea63bb81078d7290410544285c6df1a8e002952) Change-Id: I8090eb2ad393dd9964cf2d5254ec8c796b1bd367
2024-03-04Merge pull request #9191 from tfc/libutil-implementationeldritch horrors
libutil: Small improvements (cherry picked from commit edc07588ecd5053da57c2ab6c2543abf49730ce3) Change-Id: I01a481d872d859b372a4bade9bebd3dab2fb4efb
2024-03-04Merge pull request #9170 from tfc/fix-broken-moveeldritch horrors
Fix broken move (cherry picked from commit aa4f41d796630004e90790bffe2239420f77f696) Change-Id: I39d3f9f4f27328500de005c9205d905c55d6f9ea
2024-03-04Merge pull request #9167 from obsidiansystems/pre-overhaul-completionseldritch horrors
Improve tests and docs prior to refactoring completions (cherry picked from commit 5442d9b47298389918d1f38d20f768a80ffc2369) Change-Id: Ief99ac2cd9c92981a9a522d15b9c3daf99182c9d
2024-03-04Merge pull request #8905 from hercules-ci/no-unknown-locationeldritch horrors
Don't print unknown locations unless requested for dev purposes (cherry picked from commit 3dd4475826dff8052fef42d4600b61e6b91950ac) Change-Id: I04a91277d1d9d09f5c1bf4a28fc99f0702b161e5
2024-03-04Merge pull request #9056 from aakropotkin/patch-1eldritch horrors
Respect `NOCOLOR` (cherry picked from commit 4b78a66bc5066fab46b4c84a5b85c3839e9d8aab) Change-Id: I005a13dea6cb33133f7bc59926cbaf5cf51c561a