aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-08-25libstore: remove unused Goal ctor parametereldritch horrors
Change-Id: I9345fe272d6df5bd592621ce2da369fc1cd36d6d
2024-08-25Merge "fix: good errors for failures caused by allowSubstitutes" into mainjade
2024-08-25Merge changes Ief8e8ebc,Id3135db0,If1e76169 into mainjade
* changes: libutil: delete unused boost context cruft build: remove approximately 400 seconds of CPU time (30%) fix: use http proxy for s3 access
2024-08-25Add `ApplyConfigOptions`Rebecca Turner
Change-Id: Ic876bcabd0b68e579bbd30ca1755919df43d4813
2024-08-25libstore: diagnose local build failure in goaleldritch horrors
this should be done where we're actually trying to build something, not in the main worker loop that shouldn't have to be aware of such details Change-Id: I07276740c0e2e5591a8ce4828a4bfc705396527e
2024-08-24devShell: can we have debuggers?eldritch horrors
macos: we have debuggers at home (the debuggers at home: entitled little brats) Change-Id: Iefd4b5880da97846a81d601db05d2b46530a2b58
2024-08-23fix: good errors for failures caused by allowSubstitutesJade Lovelace
This caused an absolute saga which I would not like anyone else to have to experience. Let's put in a laser targeted error message that diagnoses this exact problem. Fixes: https://git.lix.systems/lix-project/lix/issues/484 Change-Id: I2a79f04aeb4a1b67c10115e5e39501d958836298
2024-08-23Fix comment in `getHome`Rebecca Turner
The logic in the comment is the opposite of the truth. Change-Id: I64add84539209782ffa46431f3db1fb306d90b3f
2024-08-23Merge "libutil/config: unify path setting types" into mainrebecca “wiggles” turner
2024-08-23libutil: delete unused boost context cruftJade Lovelace
This was from before we got rid of the boost coroutines. Now we don't need any of this code. Change-Id: Ief8e8ebc184f02f48e30cb253a66b540faa56329
2024-08-23build: remove approximately 400 seconds of CPU time (30%)Jade Lovelace
This took parsing time from 1421s or so to 1060s or so. The reason is entirely nlohmann. All of the stuff below is just Obliterated because it's built in the PCH instead: **** Templates that took longest to instantiate: 219051 ms: nlohmann::basic_json<>::parse<const char *> (276 times, avg 793 ms) 169675 ms: nlohmann::basic_json<>::basic_json (1127 times, avg 150 ms) 129416 ms: nlohmann::detail::parser<nlohmann::basic_json<>, nlohmann::detail::i... (276 times, avg 468 ms) 98155 ms: nlohmann::detail::parser<nlohmann::basic_json<>, nlohmann::detail::i... (276 times, avg 355 ms) 81322 ms: nlohmann::basic_json<>::json_value::json_value (1405 times, avg 57 ms) 53531 ms: nlohmann::detail::json_sax_dom_callback_parser<nlohmann::basic_json<... (276 times, avg 193 ms) clang-only. This brings the clang build time to not far from *half* of the gcc build time. Also, clang does not enjoy so much to miscompile coroutines. Maybe we should just be clang-only. Change-Id: Id3135db0094e4560830674090e32e6da2c22fcc6
2024-08-23fix: use http proxy for s3 accessJade Lovelace
I don't know why the AWS sdk disabled it by default. It would be nice to have test coverage of the s3 store or proxies, but neither currently exist. Fixes: https://git.lix.systems/lix-project/lix/issues/433 Change-Id: If1e76169a3d66dbec2e926af0d0d0eccf983b97b
2024-08-23lix-clang-tidy: Require Clang >= 16Rebecca Turner
Nixpkgs Clang on macOS is 16, not 17. nix-repl> packages.aarch64-darwin.nix-clangStdenv.stdenv.cc «derivation /nix/store/ycych9qpim4r42hjkznl8f6zmj0jns45-clang-wrapper-16.0.6.drv» nix-repl> packages.x86_64-linux.nix-clangStdenv.stdenv.cc «derivation /nix/store/y48dhgidb2vs230r9ayim14q61xwcdg9-clang-wrapper-17.0.6.drv» Change-Id: Ib267b8882f80eef4db665fb9df50ae285ea68b2b
2024-08-22Merge "Revert "libexpr: Replace regex engine with boost::regex"" into mainjade
2024-08-22Revert "libexpr: Replace regex engine with boost::regex"jade
This reverts commit 447212fa65a80180150b265411924cc638a2c52c. Reason for revert: Regression in eval behaviour bug-compatibility. Expected behaviour (Nix 2.18.5, macOS and Linux [libstdc++/libc++]): ``` nix-repl> builtins.match "\\.*(.*)" ".keep" [ "keep" ] nix-repl> builtins.match "(\\.*)(.*)" ".keep" [ "." "keep" ] ``` Actual behaviour (boost::regex): ``` nix-repl> builtins.match "\\.*(.*)" ".keep" [ ".keep" ] nix-repl> builtins.match "(\\.*)(.*)" ".keep" [ "." "keep" ] ``` Bug: https://git.lix.systems/lix-project/lix/issues/483 Change-Id: Id462eb8586dcd54856cf095f09b3e3a216955b60
2024-08-22Merge "libexpr: Replace regex engine with boost::regex" into mainsugar🍬🍭🏳️‍⚧️
2024-08-22libexpr: Replace regex engine with boost::regexsugar
This avoids C++'s standard library regexes, which aren't the same across platforms, and have many other issues, like using stack so much that they stack overflow when processing a lot of data. To avoid backwards and forward compatibility issues, regexes are processed using a function converting libstdc++ regexes into Boost regexes, escaping characters that Boost needs to have escaped, and rejecting features that Boost has and libstdc++ doesn't. Related context: - Original failed attempt to use `boost::regex` in CppNix, failed due to boost icu dependency being large (disabling ICU is no longer necessary because linking ICU requires using a different header file, `boost/regex/icu.hpp`): https://github.com/NixOS/nix/pull/3826 - An attempt to use PCRE, rejected due to providing less backwards compatibility with `std::regex` than `boost::regex`: https://github.com/NixOS/nix/pull/7336 - Second attempt to use `boost::regex`, failed due to `}` regex failing to compile (dealt with by writing a wrapper that parses a regular expression and escapes `}` characters): https://github.com/NixOS/nix/pull/7762 Closes #34. Closes #476. Change-Id: Ieb0eb9e270a93e4c7eed412ba4f9f96cb00a5fa4
2024-08-21fix: build with meson 1.5 alsoJade Lovelace
nixpkgs delivered us the untimely gift of a meson 1.5 upgrade, which *does* make our lives easier by allowing us to delete wrap generation code, but it does so at the cost of renaming all rust crates in such a way that the wrap logic cannot tolerate the new names on the old meson version 😭. It also means that support burden for this is going to be atrocious until we either give in and vendor meson 1.5 or we make a CI target for it. Neither seems appealing, though the latter is not super absurd for ensuring we don't break nixpkgs unstable. This commit causes meson 1.5 to ignore the .wrap files in subprojects/ entirely (since they have the wrong names lol) and instead use Cargo.lock, so it now hard-depends on our workspace reshuffling improvement. It also deletes the hack that we were using to get the sources of Cargo deps into meson by using a feature that went unnoticed when this code was originally written: MESON_PACKAGE_CACHE_DIR: https://github.com/mesonbuild/meson/blob/8a202de6ec763284cbb7160b9d43d5e7e0703f19/mesonbuild/wrap/wrap.py#L490-L502 Change-Id: I7a28f12fc2812c6ed7537b60bc3025c141a05874
2024-08-21build: move to a Cargo workspaceJade Lovelace
This is purely to let Cargo's dependency resolver do stuff for us, we do not actually intend to build this stuff with Cargo to begin with. Change-Id: I4c08d55595c7c27b7096375022581e1e34308a87
2024-08-21libutil/config: unify path setting typesAlois Wohlschlager
There have been multiple setting types for paths that are supposed to be canonicalised, depending on whether zero or one, one, or any number of paths is to be specified. Naturally, they behaved in slightly different ways in the code. Simplify things by unifying them and removing special behaviour (mainly the "multiple paths type can coerce to boolean" thing). Change-Id: I7c1ce95e9c8e1829a866fb37d679e167811e9705
2024-08-21Stop the logger in legacy commands againAlois Wohlschlager
Commit 0dd1d8ca1cdccfc620644a7f690ed35bcd2d1e74 included an accidental revert of 1461e6cdda06f7f461114cce5b415f6d50381311 (actually slightly worse), leading to the progress bar not being stopped properly when a legacy command was invoked with `--log-format bar` (or similar options that show a progress bar). Move the progress bar stopping code to its proper place again to fix this regression. Change-Id: I676333da096d5990b717a387924bb988c9b73fab
2024-08-21Merge "libexpr: Soft-deprecate ancient let syntax" into mainpiegames
2024-08-21libexpr: Soft-deprecate ancient let syntaxpiegames
Change-Id: I6802b26f038578870ea1fa1ed298f0c4b1f29c4a
2024-08-21Merge "flake: fix compiler warning" into mainjade
2024-08-21treewide: Stop using ancient let syntaxpiegames
Shows for how long these tests have not been touched by anyone … Change-Id: I3d0c1209a86283ddb012db4e7d45073264fdd0eb
2024-08-21libexpr: Soft-deprecate __overridespiegames
Change-Id: I787e69e1dad6edc5ccdb747b74a9ccd6e8e13bb3
2024-08-20flake: fix compiler warningJade Lovelace
GCC was complaining, rightfully, about mixed-sign comparisons in there. I removed some extra sign mixing too. Change-Id: I949a618c7405c23d4dc3fd17440ea2d7b5c22c9d
2024-08-20Merge "tests/functional/restricted: Don't use a process substitution" into mainAudrey Dutcher
2024-08-20lix-doc: move under src/Jade Lovelace
This is required to make more meson stuff easier/possible, and honestly it *is* now Lix sources anyhow. Change-Id: Ia6c38fabce9aa5c53768745ee38c5cf344f5c226
2024-08-20build: build lix-doc with Meson! 🎉Qyriad
lix-doc is now built with Meson, with lix-doc's dependencies built as Meson subprojects, either fetched on demand with .wrap files, or fetched in advance by Nix with importCargoLock. It even builds statically. Fixes #256. Co-authored-by: Lunaphied <lunaphied@lunaphied.me> Co-authored-by: Jade Lovelace <lix@jade.fyi> Change-Id: I3a4731ff13278e7117e0316bc0d7169e85f5eb0c
2024-08-20Merge "libutil: fix conditional for close_range availability" into mainYureka
2024-08-20libutil: fix conditional for close_range availabilityYureka
This check is wrong and would cause the close_range() function being called even when it's not available Change-Id: Ide65b36830e705fe772196c37349873353622761
2024-08-19tests/functional/restricted: Don't use a process substitutionAudrey Dutcher
The <() process substitution syntax doesn't work for this one testcase in bash for FreeBSD. The exact reason for this is unknown, possibly to do with pipe vs file vs fifo EOF behavior. The prior behavior was this test hanging forever, with no children of the bash process. Change-Id: I71822a4b9dea6059b34300568256c5b7848109ac
2024-08-19flake: don't refetch unmodified inputs by recursive followsMaximilian Bosch
Closes #460 I managed to trigger the issue by having the following inputs (shortened): authentik-nix.url = "github:nix-community/authentik-nix"; authentik-nix.inputs.poetry2nix.inputs.nixpkgs.follows = "nixpkgs"; When evaluating this using nix-eval-jobs --flake .#hydraJobs I got the following error: error: cannot update unlocked flake input 'authentik-nix/poetry2nix' in pure mode The issue we have here is that `authentik-nix/poetry2nix` was written into the `overrideMap` which caused Nix to assume it's a new input and tried to refetch it (#460) or errored out in pure mode (nix-eval-jobs / Hydra). The testcase unfortunately only involves checking for the output log and makes sure that something *is* logged on the first fetch so that the test doesn't rot when the logging changes since I didn't manage to trigger the error above with the reproducer from #460. In fact, I only managed to trigger the `cannot update unlocked flake input` error in this context with `nix-eval-jobs`. Change-Id: Ifd00091eec9a0067ed4bb3e5765a15d027328807
2024-08-19libstore: un-enable_shared_from_this Goaleldritch horrors
it's no longer needed for anything, and not even a great idea. Change-Id: Ia7a59e1e3f9d8f4ad2ac3b054e38485157c210a6
2024-08-19libstore: make Worker::childStarted privateeldritch horrors
this can be a proper WorkResult now. childTerminated is unfortunately a lot more stubborn and won't be made private for quite a while yet. once we can get rid of the Worker poll loop that *should* be possible though Change-Id: I2218df202da5cb84e852f6a37e4c20367495b617
2024-08-19libstore: turn HookReply into a variant typeeldritch horrors
we'll need this once we want to pass extra information out of accepting replies, such as fd sets or possibly even async output reader promises. Change-Id: I5e2f18cdb80b0d2faf3067703cc18bd263329b3f
2024-08-19libstore: downsize hook pipeseldritch horrors
don't keep fds open we're not using. currently this does not cause any problems, but it does increase the size of our fd table needlessly and in the future, when we have proper async processing, having builderOut open in the daemon once the hook has been fully started is problematic Change-Id: I6e7fb773b280b042873103638d3e04272ca1e4fc
2024-08-19libstore: run childStarted as late as possibleeldritch horrors
otherwise we *technically* give away the output fds before we've read them. Change-Id: I6ad0d6a1bb553ecfcdd7708f50d34142a425374d
2024-08-19libstore: move respect-timeoutiness to goal methodeldritch horrors
this is useless to do on the face of it, but it'll make it easier to convert the entire output handling to use async io and promises soon Change-Id: I2d1eb62c4bbf8f57bd558b9599c08710a389b1a8
2024-08-19Merge "doc: fix broken meson deps for various manuals outputs" into mainjade
2024-08-19Merge "build: limit clang-tidy concurrency and respect NIX_BUILD_CORES" into ↵jade
main
2024-08-18libstore: clarify that build log fd and hook log fd are differenteldritch horrors
only DerivationGoal can set the hook to anything at all. it always sets buildOutFD to something that is not related to fromHook in any way, and mixing the two would have rather dire consequences for log consistency. Change-Id: Ida86727fd1cd5e1ecd78f07f3bde330a346658a8
2024-08-18build: limit clang-tidy concurrency and respect NIX_BUILD_CORESJade Lovelace
Apparently it was impolite to lint with 128 jobs on our CI machine with 128 threads. Let's fix it. Change-Id: I9ca7306294c6773c6f233690ba49d45a1da6bf7a
2024-08-18doc: fix broken meson deps for various manuals outputsJade Lovelace
This is incredibly haunted, but it can happen that you change libutil, breaking the generation of the .json files, which then does not rebuild the files. I don't expect they are slow to build, so it does not seem so bad to just rebuild them every time instead of extracting a list of all the possible deps. We want to delete this nonsense anyway and replace it with generated code. Change-Id: Ia576d1a3bdee48fbaefbb5ac194354428d179a84
2024-08-18libstore: remove DerivationGoal::isReadDesceldritch horrors
all derivation goals need a log fd of some description. let's save this single fd in a dedicated pointer field for all subclasses so that later we have just the one spot to change if we turn this into async promises Change-Id: If223adf90909247363fb823d751cae34d25d0c0b
2024-08-18libutil: Optimize feature checkspiegames
Instead of doing a linear search on an std::set, we use a bitset enum. Change-Id: Ide537f6cffdd16d06e59aaeb2e4ac0acb6493421
2024-08-18libstore: don't ignore max-build-log-size for ssh-ngeldritch horrors
Change-Id: Ieab14662bea6e6f5533325f0e945147be998f9a2
2024-08-18libstore: add explicit in-build-slot-ness to goalseldritch horrors
we don't need to expose information about how busy a Worker is if the worker can instead tell its work items whether they are in a slot. in the future we might use this to not start items waiting for a slot if no slots are currently available, but that requires more preparation. Change-Id: Ibe01ac536da7e6d6f80520164117c43e772f9bd9
2024-08-18libstore: remove method without definitioneldritch horrors
Change-Id: I676411752a4b1777045d7211ac1176693f1a3d7d