aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-09-26Merge "libutil: add async collection mechanism" into maineldritch horrors
2024-09-26Merge "fmt: fail hard on bad format strings going into nix::fmt too" into mainjade
2024-09-26Merge "main: log stack traces for std::terminate" into mainjade
2024-09-26libutil: add async collection mechanismeldritch horrors
like kj::joinPromisesFailFast this allows waiting for the results of multiple promises at once, but unlike it not all input promises must be complete (or any of them failed) for results to become available. Change-Id: I0e4a37e7bd90651d56b33d0bc5afbadc56cde70c
2024-09-26libutil: add an async semaphore implementationeldritch horrors
like a normal semaphore, but with awaitable acquire actions. this is primarily intended as an intermediate concurrency limiting device in the Worker code, but it may find other uses over time. we do not use std::counting_semaphore as a base because the counter of that is not inspectable as will be needed for Worker. we also do not need atomic operations for cross-thread consistency since we don't have multiple threads (thanks to kj event loops being confined to a single thread) Change-Id: Ie2bcb107f3a2c0185138330f7cbba4cec6cbdd95
2024-09-26fix internal-api-docs buildeldritch horrors
this one is also run from a gcc stdenv. Change-Id: I91ff6915c6689ece15224f348f54367cff5d2b5a
2024-09-26Fix passing custom CA files into the builtin:fetchurl sandboxPuck Meerburg
Without this, verifying TLS certificates would fail on macOS, as well as any system that doesn't have a certificate file at /etc/ssl/certs/ca-certificates.crt, which includes e.g. Fedora. Change-Id: Iaa2e0e9db3747645b5482c82e3e0e4e8f229f5f9
2024-09-26Merge "flake: use clangStdenv for `overlays.default`" into mainMaximilian Bosch
2024-09-25[security] builtin:fetchurl: Enable TLS verificationEelco Dolstra
This is better for privacy and to avoid leaking netrc credentials in a MITM attack, but also the assumption that we check the hash no longer holds in some cases (in particular for impure derivations). Partially reverts https://github.com/NixOS/nix/commit/5db358d4d78aea7204a8f22c5bf2a309267ee038. (cherry picked from commit c04bc17a5a0fdcb725a11ef6541f94730112e7b6) (cherry picked from commit f2f47fa725fc87bfb536de171a2ea81f2789c9fb) (cherry picked from commit 7b39cd631e0d3c3d238015c6f450c59bbc9cbc5b) Upstream-PR: https://github.com/NixOS/nix/pull/11585 Change-Id: Ia973420f6098113da05a594d48394ce1fe41fbb9
2024-09-25fmt: fail hard on bad format strings going into nix::fmt tooJade Lovelace
Previously we would only crash the program for bad HintFmt calls. nix::fmt should also crash. Change-Id: I4ba0abeb8557b208bd9c0be624c022a60446ef7e
2024-09-25main: log stack traces for std::terminateJade Lovelace
These stack traces kind of suck for the reasons mentioned on the CppTrace page here (no symbols for inline functions is a major one): https://github.com/jeremy-rifkin/cpptrace I would consider using CppTrace if it were packaged, but to be honest, I think that the more reasonable option is actually to move entirely to out-of-process crash handling and symbolization. The reason for this is that if you want to generate anything of substance on SIGSEGV or really any deadly signal, you are stuck in async-signal-safe land, which is not a place to be trying to run a symbolizer. LLVM does it anyway, probably carefully, and chromium *can* do it on debug builds but in general uses crashpad: https://source.chromium.org/chromium/chromium/src/+/main:base/debug/stack_trace_posix.cc;l=974;drc=82dff63dbf9db05e9274e11d9128af7b9f51ceaa;bpv=1;bpt=1 However, some stack traces are better than *no* stack traces when we get mystery exceptions falling out the bottom of the program. I've also promoted the path for "mystery exceptions falling out the bottom of the program" to hard crash and generate a core dump because although there's been some months since the last one of these, these are nonetheless always *atrociously* diagnosed. We can't improve the crash handling further until either we use Crashpad (which involves more C++ deps, no thanks) or we put in the ostensibly work in progress Rust minidump infrastructure, in which case we need to finish full support for Rust in libutil first. Sample report: Lix crashed. This is a bug. We would appreciate if you report it at https://git.lix.systems/lix-project/lix/issues with the following information included: Exception: std::runtime_error: lol Stack trace: 0# nix::printStackTrace() in /home/jade/lix/lix3/build/src/nix/../libutil/liblixutil.so 1# 0x000073C9862331F2 in /home/jade/lix/lix3/build/src/nix/../libmain/liblixmain.so 2# 0x000073C985F2E21A in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6 3# 0x000073C985F2E285 in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6 4# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/jade/lix/lix3/build/src/nix/../libmain/liblixmain.so 5# 0x00005CF65B6B048B in /home/jade/lix/lix3/build/src/nix/nix 6# 0x000073C985C8810E in /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6 7# __libc_start_main in /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6 8# 0x00005CF65B610335 in /home/jade/lix/lix3/build/src/nix/nix Change-Id: I1a9f6d349b617fd7145a37159b78ecb9382cb4e9
2024-09-25Merge "package.nix: fix cross for editline" into mainjade
2024-09-25flake: use clangStdenv for `overlays.default`Maximilian Bosch
We don't support GCC anymore for building, so the overlay currently fails to evaluate with error: assertion '((stdenv).cc.isClang || lintInsteadOfBuild)' failed `clangStdenv` seems like a reasonable default now. Noticed while upgrading Lix for our Hydra fork. Change-Id: I948a7c03b3e5648fc7c596f96e1b8053a9e7f92f
2024-09-24internal-api-docs: allow Doxygen to build regardless of workdirOlivia Crain
Previously, Doxygen needed to be ran from the project's source root dir due to the relative paths in the config's `INPUT` tag. We now preprocess the relative paths by prefixing them with the absolute path of the project's source root dir. The HTML output remains unchanged. Fixes: https://git.lix.systems/lix-project/lix/issues/240 Change-Id: I85f099c22bfc5fdbf26be27c2db7dcbc8155c8b2
2024-09-21Merge "local-store: make extended attribute handling more robust" into mainalois31
2024-09-18Merge "util: fix brotli decompression of empty input" into mainjade
2024-09-18util: fix brotli decompression of empty inputJade Lovelace
This caused an infinite loop before since it would just keep asking the underlying source for more data. In practice this happened because an HTTP server served a response to a HEAD request (for which curl will not retrieve any body or call our write callback function) with Content-Encoding: br, leading to decompressing nothing at all and going into an infinite loop. This adds a test to make sure none of our compression methods do that again, as well as just patching the HTTP client to never feed empty data into a compression algorithm (since they absolutely have the right to throw CompressionError on unexpectedly-short streams!). Reported on Matrix: https://matrix.to/#/!lymvtcwDJ7ZA9Npq:lix.systems/$8BWQR_zKxCQDJ40C5NnDo4bQPId3pZ_aoDj2ANP7Itc?via=lix.systems&via=matrix.org&via=tchncs.de Change-Id: I027566e280f0f569fdb8df40e5ecbf46c211dad1
2024-09-18Merge "tests/compression: rewrite" into mainrebecca “wiggles” turner
2024-09-18path-info: wipe the progress bar before printingAlois Wohlschlager
The legitimate output of `nix path-info` may visually interfere with the progress bar, by appending to stale progress output before the latter has been erased. Conveniently, all expensive operations (evaluation or building) have already been performed before, so we can simply wipe the progress bar at this point to fix the issue. Fixes: https://git.lix.systems/lix-project/lix/issues/343 Change-Id: Id9a807a5c882295b3e6fbf841f9c15dc96f67f6e
2024-09-17package.nix: fix cross for editlineJade Lovelace
editline's dep on ncurses is a runtime one, so it should be in buildInputs, not nativeBuildInputs. CC: https://git.lix.systems/lix-project/lix/issues/527 Change-Id: I631c192a55677b0cc77faa7511986f1fa2205e91
2024-09-17tests/compression: rewriteJade Lovelace
This test suite was in desperate need of using the parameterization available with gtest, and was a bunch of useless duplicated code. At least now it's not duplicated code, though it still probably should be more full of property tests. Change-Id: Ia8ccee7ef4f02b2fa40417b79aa8c8f0626ea479
2024-09-17Merge "Remove readline support" into mainrebecca “wiggles” turner
2024-09-16Remove readline supportRebecca Turner
Lix cannot be built with GNU readline, and we would "rather not" be GPL. Change-Id: I0e86f0f10dab966ab1d1d467fb61fd2de50c00de
2024-09-15common-eval-args: raise warning if `--arg` isn't a valid Nix identifierMaximilian Bosch
See https://git.lix.systems/lix-project/lix/issues/496. The core idea is to be able to do e.g. nix-instantiate -A some-nonfree-thing --arg config.allowUnfree true which is currently not possible since `config.allowUnfree` is interpreted as attribute name with a dot in it. In order to change that (probably), Jade suggested to find out if there are any folks out there relying on this behavior. For such a use-case, it may still be possible to accept strings, i.e. `--arg '"config.allowUnfree"'. Change-Id: I986c73619fbd87a95b55e2f0ac03feaed3de2d2d
2024-09-14fix: docs issue template was bustedJade Lovelace
Apparently forgejo has a more creative interpretation of \(\) than I was hoping in their markdown parser and thought it was maths. I have no idea then how you put a link in parens next to another square-bracket link, but I am not going to worry about it. There were several more typos, which I also fixed. Fixes: https://git.lix.systems/lix-project/lix/issues/517 Change-Id: I6b144c6881f92ca60ba72a304ce7a0bcb9c6659a
2024-09-14Merge "store: add a hint on how to fix Lix installs broken by macOS Sequoia" ↵jade
into main
2024-09-14Merge changes Ia1481da4,Ifca1d74d into mainjade
* changes: archive: refactor bad mutable-state API in the NAR parse listener archive: rename ParseSink to NARParseVisitor
2024-09-14local-store: make extended attribute handling more robustAlois Wohlschlager
* Move the extended attribute deletion after the hardlink sanity check. We shouldn't be removing extended attributes on random files. * Make the entity owner-writable before attempting to remove extended attributes, since this operation usually requires write access on the file, and we shouldn't fail xattr deletion on a file that has been made unwritable by the builder or a previous canonicalisation pass. Fixes: https://git.lix.systems/lix-project/lix/issues/507 Change-Id: I7e6ccb71649185764cd5210f4a4794ee174afea6
2024-09-14store: add a hint on how to fix Lix installs broken by macOS SequoiaJade Lovelace
This is not a detailed diagnosis, and it's not worth writing one, tbh. This error basically never happens in normal operation, so diagnosing it by changing the error on macOS is good enough. Relevant: https://git.lix.systems/lix-project/lix-installer/issues/24 Relevant: https://git.lix.systems/lix-project/lix-installer/issues/18 Relevant: https://git.lix.systems/lix-project/lix/issues/521 Change-Id: I03701f917d116575c72a97502b8e1617679447f2
2024-09-13archive: refactor bad mutable-state API in the NAR parse listenerJade Lovelace
Remove the mutable state stuff that assumes that one file is being written a time. It's true that we don't write multiple files interleaved, but that mutable state is evil. Change-Id: Ia1481da48255d901e4b09a9b783e7af44fae8cff
2024-09-12Merge "fish-completion: leave the shell prompt intact" into mainalois31
2024-09-11fish-completion: leave the shell prompt intactAlois Wohlschlager
When generating shell completions, no logging output should be visible because it would destroy the shell prompt. Originally this was attempted to be done by simply disabling the progress bar (ca946860ce6ce5d4800b0d93d3f83c30d3c953c0), since the situation is particularly bad there (the screen clearing required for the rendering ends up erasing the shell prompt). Due to overlooking the implementation of this hack, it was accidentally undone during a later change (0dd1d8ca1cdccfc620644a7f690ed35bcd2d1e74). Since even with the hack correctly in place, it is still possible to mess up the prompt by logging output (for example warnings for disabled experimental features, or messages generated by `builtins.trace`), simply send it to the bit bucket where it belongs. This was already done for bash and zsh (9d840758a8d195e52e8b7d08cd9c15f6b8259724), and it seems that fish was simply missed at that time. The last trace of the no-longer-working and obsolete hack is deleted too. Fixes: https://git.lix.systems/lix-project/lix/issues/513 Change-Id: I59f1ebf90903034e2059298fa8d76bf970bc3315
2024-09-11editline: Vendor cl/1883 patch to recognize `Alt+Left`/`Alt+Right`Rebecca Turner
This vendors the patch added in cl/1883 to avoid GitHub garbage-collecting the commits we're referring to. As @emilazy pointed out on GitHub: > GitHub can garbage‐collect unmerged PR commits if they are later > force‐pushed, which means that code review in upstreams can cause > Nixpkgs builds to fail to reproduce in future. See: https://github.com/NixOS/nixpkgs/pull/341131#discussion_r1753046220 See: https://github.com/troglobit/editline/pull/70 See: https://gerrit.lix.systems/c/lix/+/1883 Change-Id: Ifff522f7f23310d6dbe9efc72fd40be5500ae872
2024-09-11archive: rename ParseSink to NARParseVisitorJade Lovelace
- Rename the listener to not be called a "sink". If it were a "sink" it would be eating bytes and conform with any of the Nix sink stuff (maybe FileHandle should be a Sink itself! but that's a later CL's problem). This is a parser listener. - Move the RetrieveRegularNARSink thing into store-api.cc, which is its only usage, and fix it to actually do what it is stated to do: crash if its invariants are violated. It's, of course, used to erm, unpack single-file NAR files, generated via a horrible contraption of sources and sinks that looks like a plumbing blueprint. Refactoring that is a future task. - Add a description of the invariants of NARParseVisitor in preparation of refactoring it. Change-Id: Ifca1d74d2947204a1f66349772e54dad0743e944
2024-09-11Merge "repl: Patch editline to recognize Meta-Left & Meta-Right" into mainrebecca “wiggles” turner
2024-09-10Merge "repl-overlays: Provide an elaborate example" into mainrebecca “wiggles” turner
2024-09-10Merge "Add `getCwd`" into mainrebecca “wiggles” turner
2024-09-09repl: Patch editline to recognize Meta-Left & Meta-RightRebecca Turner
This applies https://github.com/troglobit/editline/pull/70 to our build of editline, which translates `meta-left` and `meta-right` into `fd_word` and `bk_word`. This makes `nix repl` soooo much nicer to use! Note: My terminal renders `meta-left` as `\e\e[C` and `meta-right` as `\e\e[D`. Closes https://git.lix.systems/lix-project/lix/issues/501 Change-Id: I048b10cf17231bbf4e6bf38e1d1d8572cedaa194
2024-09-09Merge changes If8ec210f,I6e2851b2 into mainalois31
* changes: libfetchers: serialise accept-flake-config properly libstore: declare SandboxMode JSON serialisation in the header
2024-09-09forbid gcc for compilation, only allow clangeldritch horrors
while gcc 12 and older miscompile our generators, gcc 13 and older outright crash on kj coroutines. (newer gcc versions may fix this) Change-Id: I19f12c8c147239680eb0fa5a84ef5c7de38c9263
2024-09-08Merge "libmain/progress-bar: erase all lines of the multi-line format" into mainalois31
2024-09-08libstore: turn Worker in a kj event loop usereldritch horrors
using a proper event loop basis we no longer have to worry about most of the intricacies of poll(), or platform-dependent replacements for it. we may even be able to use the event loop and its promise system for all of our scheduling in the future. we don't do any real async processing yet, this is just preparation to separate the first such change from the huge api design difference with the async framework we chose (kj from capnp): kj::Promise, unlike std::future, doesn't return exceptions unmangled. it instead wraps any non-kj exception into a kj exception, erasing all type information and preserving mostly the what() string in the process. this makes sense in the capnp rpc use case where unrestricted exception types can't be transferred, and since it moves error handling styles closer to a world we'd actually like there's no harm in doing it only here for now Change-Id: I20f888de74d525fb2db36ca30ebba4bcfe9cc838
2024-09-08libutil: add a result type using boost outcomeeldritch horrors
we're using boost::outcome rather than leaf or stl types because stl types are not available everywhere and leaf does not provide its own storage for error values, relying on thread-locals and the stack. if we want to use promises we won't have a stack and would have to wrap everything into leaf-specific allocating wrappers, so outcome it is. Change-Id: I35111a1f9ed517e7f12a839e2162b1ba6a993f8f
2024-09-07libmain/progress-bar: erase all lines of the multi-line formatAlois Wohlschlager
When the multi-line log format is enabled, the progress bar usually occupies multiple lines on the screen. When stopping the progress bar, only the last line was wiped, leaving all others visible on the screen. Erase all lines belonging to the progress bar to prevent these leftovers. Asking the user for input is theoretically affected by a similar issue, but this is not observed in practice since the only place where the user is asked (whether configuration options coming from flakes should be accepted) does not actually have multiple lines on the progress bar. However, there is no real reason to not fix this either, so let's do it anyway. Change-Id: Iaa5a701874fca32e6f06d85912835d86b8fa7a16
2024-09-06Merge "Stop the logger in legacy commands again" into mainalois31
2024-09-03Merge "Test including relative paths in configuration" into mainrebecca “wiggles” turner
2024-09-03Merge "Expand comment on `std::string operator+`" into mainrebecca “wiggles” turner
2024-09-02libfetchers: serialise accept-flake-config properlyAlois Wohlschlager
The AcceptFlakeConfig type used was missing its JSON serialisation definition, so it was incorrectly serialised as an integer, ending up that way for example in the nix.conf manual page. Declare a proper serialisation. Change-Id: If8ec210f9d4dd42fe480c4e97d0a4920eb66a01e
2024-09-02libstore: declare SandboxMode JSON serialisation in the headerAlois Wohlschlager
The JSON serialisation should be declared in the header so that all translation units can see it when needed, even though it seems that it has not been used anywhere else so far. Unfortunately, this means we cannot use the NLOHMANN_JSON_SERIALIZE_ENUM convenience macro, since it uses a slightly different signature, but the code is not too bad either. Change-Id: I6e2851b250e0b53114d2fecb8011ff1ea9379d0f
2024-09-01Test including relative paths in configurationRebecca Turner
Change-Id: If6c69a5e16d1ccd223fba392890f08f0032fb754