aboutsummaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-08-22Merge "Revert "libexpr: Replace regex engine with boost::regex"" into mainjade
2024-08-22Revert "libexpr: Replace regex engine with boost::regex"jade
This reverts commit 447212fa65a80180150b265411924cc638a2c52c. Reason for revert: Regression in eval behaviour bug-compatibility. Expected behaviour (Nix 2.18.5, macOS and Linux [libstdc++/libc++]): ``` nix-repl> builtins.match "\\.*(.*)" ".keep" [ "keep" ] nix-repl> builtins.match "(\\.*)(.*)" ".keep" [ "." "keep" ] ``` Actual behaviour (boost::regex): ``` nix-repl> builtins.match "\\.*(.*)" ".keep" [ ".keep" ] nix-repl> builtins.match "(\\.*)(.*)" ".keep" [ "." "keep" ] ``` Bug: https://git.lix.systems/lix-project/lix/issues/483 Change-Id: Id462eb8586dcd54856cf095f09b3e3a216955b60
2024-08-22Merge "libexpr: Replace regex engine with boost::regex" into mainsugar🍬🍭🏳️‍⚧️
2024-08-22libexpr: Replace regex engine with boost::regexsugar
This avoids C++'s standard library regexes, which aren't the same across platforms, and have many other issues, like using stack so much that they stack overflow when processing a lot of data. To avoid backwards and forward compatibility issues, regexes are processed using a function converting libstdc++ regexes into Boost regexes, escaping characters that Boost needs to have escaped, and rejecting features that Boost has and libstdc++ doesn't. Related context: - Original failed attempt to use `boost::regex` in CppNix, failed due to boost icu dependency being large (disabling ICU is no longer necessary because linking ICU requires using a different header file, `boost/regex/icu.hpp`): https://github.com/NixOS/nix/pull/3826 - An attempt to use PCRE, rejected due to providing less backwards compatibility with `std::regex` than `boost::regex`: https://github.com/NixOS/nix/pull/7336 - Second attempt to use `boost::regex`, failed due to `}` regex failing to compile (dealt with by writing a wrapper that parses a regular expression and escapes `}` characters): https://github.com/NixOS/nix/pull/7762 Closes #34. Closes #476. Change-Id: Ieb0eb9e270a93e4c7eed412ba4f9f96cb00a5fa4
2024-08-21fix: build with meson 1.5 alsoJade Lovelace
nixpkgs delivered us the untimely gift of a meson 1.5 upgrade, which *does* make our lives easier by allowing us to delete wrap generation code, but it does so at the cost of renaming all rust crates in such a way that the wrap logic cannot tolerate the new names on the old meson version 😭. It also means that support burden for this is going to be atrocious until we either give in and vendor meson 1.5 or we make a CI target for it. Neither seems appealing, though the latter is not super absurd for ensuring we don't break nixpkgs unstable. This commit causes meson 1.5 to ignore the .wrap files in subprojects/ entirely (since they have the wrong names lol) and instead use Cargo.lock, so it now hard-depends on our workspace reshuffling improvement. It also deletes the hack that we were using to get the sources of Cargo deps into meson by using a feature that went unnoticed when this code was originally written: MESON_PACKAGE_CACHE_DIR: https://github.com/mesonbuild/meson/blob/8a202de6ec763284cbb7160b9d43d5e7e0703f19/mesonbuild/wrap/wrap.py#L490-L502 Change-Id: I7a28f12fc2812c6ed7537b60bc3025c141a05874
2024-08-21build: move to a Cargo workspaceJade Lovelace
This is purely to let Cargo's dependency resolver do stuff for us, we do not actually intend to build this stuff with Cargo to begin with. Change-Id: I4c08d55595c7c27b7096375022581e1e34308a87
2024-08-21libutil/config: unify path setting typesAlois Wohlschlager
There have been multiple setting types for paths that are supposed to be canonicalised, depending on whether zero or one, one, or any number of paths is to be specified. Naturally, they behaved in slightly different ways in the code. Simplify things by unifying them and removing special behaviour (mainly the "multiple paths type can coerce to boolean" thing). Change-Id: I7c1ce95e9c8e1829a866fb37d679e167811e9705
2024-08-21Stop the logger in legacy commands againAlois Wohlschlager
Commit 0dd1d8ca1cdccfc620644a7f690ed35bcd2d1e74 included an accidental revert of 1461e6cdda06f7f461114cce5b415f6d50381311 (actually slightly worse), leading to the progress bar not being stopped properly when a legacy command was invoked with `--log-format bar` (or similar options that show a progress bar). Move the progress bar stopping code to its proper place again to fix this regression. Change-Id: I676333da096d5990b717a387924bb988c9b73fab
2024-08-21Merge "libexpr: Soft-deprecate ancient let syntax" into mainpiegames
2024-08-21libexpr: Soft-deprecate ancient let syntaxpiegames
Change-Id: I6802b26f038578870ea1fa1ed298f0c4b1f29c4a
2024-08-21Merge "flake: fix compiler warning" into mainjade
2024-08-21libexpr: Soft-deprecate __overridespiegames
Change-Id: I787e69e1dad6edc5ccdb747b74a9ccd6e8e13bb3
2024-08-20flake: fix compiler warningJade Lovelace
GCC was complaining, rightfully, about mixed-sign comparisons in there. I removed some extra sign mixing too. Change-Id: I949a618c7405c23d4dc3fd17440ea2d7b5c22c9d
2024-08-20lix-doc: move under src/Jade Lovelace
This is required to make more meson stuff easier/possible, and honestly it *is* now Lix sources anyhow. Change-Id: Ia6c38fabce9aa5c53768745ee38c5cf344f5c226
2024-08-20build: build lix-doc with Meson! 🎉Qyriad
lix-doc is now built with Meson, with lix-doc's dependencies built as Meson subprojects, either fetched on demand with .wrap files, or fetched in advance by Nix with importCargoLock. It even builds statically. Fixes #256. Co-authored-by: Lunaphied <lunaphied@lunaphied.me> Co-authored-by: Jade Lovelace <lix@jade.fyi> Change-Id: I3a4731ff13278e7117e0316bc0d7169e85f5eb0c
2024-08-20Merge "libutil: fix conditional for close_range availability" into mainYureka
2024-08-20libutil: fix conditional for close_range availabilityYureka
This check is wrong and would cause the close_range() function being called even when it's not available Change-Id: Ide65b36830e705fe772196c37349873353622761
2024-08-19flake: don't refetch unmodified inputs by recursive followsMaximilian Bosch
Closes #460 I managed to trigger the issue by having the following inputs (shortened): authentik-nix.url = "github:nix-community/authentik-nix"; authentik-nix.inputs.poetry2nix.inputs.nixpkgs.follows = "nixpkgs"; When evaluating this using nix-eval-jobs --flake .#hydraJobs I got the following error: error: cannot update unlocked flake input 'authentik-nix/poetry2nix' in pure mode The issue we have here is that `authentik-nix/poetry2nix` was written into the `overrideMap` which caused Nix to assume it's a new input and tried to refetch it (#460) or errored out in pure mode (nix-eval-jobs / Hydra). The testcase unfortunately only involves checking for the output log and makes sure that something *is* logged on the first fetch so that the test doesn't rot when the logging changes since I didn't manage to trigger the error above with the reproducer from #460. In fact, I only managed to trigger the `cannot update unlocked flake input` error in this context with `nix-eval-jobs`. Change-Id: Ifd00091eec9a0067ed4bb3e5765a15d027328807
2024-08-19libstore: un-enable_shared_from_this Goaleldritch horrors
it's no longer needed for anything, and not even a great idea. Change-Id: Ia7a59e1e3f9d8f4ad2ac3b054e38485157c210a6
2024-08-19libstore: make Worker::childStarted privateeldritch horrors
this can be a proper WorkResult now. childTerminated is unfortunately a lot more stubborn and won't be made private for quite a while yet. once we can get rid of the Worker poll loop that *should* be possible though Change-Id: I2218df202da5cb84e852f6a37e4c20367495b617
2024-08-19libstore: turn HookReply into a variant typeeldritch horrors
we'll need this once we want to pass extra information out of accepting replies, such as fd sets or possibly even async output reader promises. Change-Id: I5e2f18cdb80b0d2faf3067703cc18bd263329b3f
2024-08-19libstore: downsize hook pipeseldritch horrors
don't keep fds open we're not using. currently this does not cause any problems, but it does increase the size of our fd table needlessly and in the future, when we have proper async processing, having builderOut open in the daemon once the hook has been fully started is problematic Change-Id: I6e7fb773b280b042873103638d3e04272ca1e4fc
2024-08-19libstore: run childStarted as late as possibleeldritch horrors
otherwise we *technically* give away the output fds before we've read them. Change-Id: I6ad0d6a1bb553ecfcdd7708f50d34142a425374d
2024-08-19libstore: move respect-timeoutiness to goal methodeldritch horrors
this is useless to do on the face of it, but it'll make it easier to convert the entire output handling to use async io and promises soon Change-Id: I2d1eb62c4bbf8f57bd558b9599c08710a389b1a8
2024-08-18libstore: clarify that build log fd and hook log fd are differenteldritch horrors
only DerivationGoal can set the hook to anything at all. it always sets buildOutFD to something that is not related to fromHook in any way, and mixing the two would have rather dire consequences for log consistency. Change-Id: Ida86727fd1cd5e1ecd78f07f3bde330a346658a8
2024-08-18libstore: remove DerivationGoal::isReadDesceldritch horrors
all derivation goals need a log fd of some description. let's save this single fd in a dedicated pointer field for all subclasses so that later we have just the one spot to change if we turn this into async promises Change-Id: If223adf90909247363fb823d751cae34d25d0c0b
2024-08-18libutil: Optimize feature checkspiegames
Instead of doing a linear search on an std::set, we use a bitset enum. Change-Id: Ide537f6cffdd16d06e59aaeb2e4ac0acb6493421
2024-08-18libstore: don't ignore max-build-log-size for ssh-ngeldritch horrors
Change-Id: Ieab14662bea6e6f5533325f0e945147be998f9a2
2024-08-18libstore: add explicit in-build-slot-ness to goalseldritch horrors
we don't need to expose information about how busy a Worker is if the worker can instead tell its work items whether they are in a slot. in the future we might use this to not start items waiting for a slot if no slots are currently available, but that requires more preparation. Change-Id: Ibe01ac536da7e6d6f80520164117c43e772f9bd9
2024-08-18libstore: remove method without definitioneldritch horrors
Change-Id: I676411752a4b1777045d7211ac1176693f1a3d7d
2024-08-18libstore: remove unused includes in worker codeeldritch horrors
Change-Id: I6c7fccc4e710e23a22faae2669cb75f2f6da27b4
2024-08-18libstore: make Worker::removeGoal privateeldritch horrors
Change-Id: I8583d9ff752f702a10ec52b0330b0d4d4d2614fa
2024-08-17libexpr: Deprecate URL literalspiegames
Closes #437. Change-Id: I9f67fc965bb4a7e7fd849e5067ac1cb3bab064cd
2024-08-17libexpr: Introduce Deprecated featurespiegames
They are like experimental features, but opt-in instead of opt-out. They will allow us to gracefully remove language features. See #437 Change-Id: I9ca04cc48e6926750c4d622c2b229b25cc142c42
2024-08-13Merge "libutil: Add bindPath function from libstore" into mainArtemis Tosini
2024-08-10libutil: deal with Linux systems that do not implement close_rangeJade Lovelace
Seems a little bit Rich that musl does not implement close_range because they suspect that the system call itself is a bad idea, so they uhhhh are considering not implementing a wrapper. Let's just fix the problem at hand by writing our own wrapper. Change-Id: I1f8e5858e4561d58a5450503d9c4585aded2b216
2024-08-10nix: remove explosions if you have a window size less than fourJade Lovelace
Turns out strings do not like being resized to -4. This was discovered while messing with the tests to remove unbuffer and trying stdbuf instead. Turns out that was not the right approach. This basically rewrites the handling of this case to be much more correct, and fixes a bug where with small window sizes where it would ALSO truncate the attr names in addition to the optional descriptions. Change-Id: Ifd1beeaffdb47cbb5f4a462b183fcb6c0ff6c524
2024-08-10fix: check if it is a Real terminal, not just if it is a terminalJade Lovelace
This will stop printing stuff to dumb terminals that they don't support. I've overall audited usage of isatty and replaced the ones with intent to mean "is a Real terminal" with checking for that. I've also caught a case of carelessly assuming "is a tty" means "should be colour" in nix-env. Change-Id: I6d83725d9a2d932ac94ff2294f92c0a1100d23c9
2024-08-09libutil: rename and optimize closeMostFDseldritch horrors
this is only used to close non-stdio files in derivation sandboxes. we may as well encode that in its name, drop the unnecessary integer set, and use close_range to deal with the actual closing of files. not only is this clearer, it also makes sandbox setup on linux fast by 1ms each Change-Id: Id90e259a49c7bc896189e76bfbbf6ef2c0bcd3b2
2024-08-09libstore: deprecate the build-hook settingeldritch horrors
implementing a build hook is pretty much impossible without either being a nix, or blindly forwarding the important bits of all build requests to some kind of nix. we've found no uses of build-hook in the wild, and the build-hook protocol (apart from being entirely undocumented) is not able to convey any kind of versioning information between hook and daemon. if we want to upgrade this infrastructure (which we do), this must not stay Change-Id: I1ec4976a35adf8105b8ca9240b7984f8b91e147e
2024-08-09Merge changes Ib7c80826,I636f8a71,I67669b98 into mainjade
* changes: perl: un-autos your conf build: declare all the deps as -isystem darwin: workaround PROC_PIDLISTFDS on processes with no fds
2024-08-09Merge "libexpr: move Value implementations out of eval.cc" into mainQyriad
2024-08-09libutil: allow marking settings as deprecatedeldritch horrors
this is a bit of a hack, but it's apparently the cleanest way of doing this in the absence of any kind of priority/provenance information for values of some given setting. we'll need this to deprecate build-hook. Change-Id: I03644a9c3f17681c052ecdc610b4f1301266ab9e
2024-08-09main: require argv[0]eldritch horrors
sure, linux has been providing argv[0] by default for a while now. other OSes may not be as forthcoming though, and relying on the OS to create a world in which we can just make assumptions we could test for instead is unnecessarily lazy. we *could* default argv0, but that's a little silly. notably we abort instead of returning normally to avoid confusions where a caller interprets our exit status like a Worker build results bitmask. Change-Id: Id73f8cd0a630293b789c59a8c4b0c4a2b936b505
2024-08-09resolve-system-dependencies: remove entirelyeldritch horrors
this hasn't been used since 2020, and hasn't been compiled since may. Change-Id: I865550966630eee6ba18d742ba36f0a90901279d
2024-08-08libexpr: move Value implementations out of eval.ccQyriad
Change-Id: I2ce8a9713533888b3d109a56947156eb3a5ab492
2024-08-08Merge changes I6358a393,I2d9f276b,Idd096dc9 into mainjade
* changes: clang-tidy: write a lint for charptr_cast tree-wide: automated migration to charptr_cast clang-tidy: enforce the new rules
2024-08-08Merge changes I526cceed,Ia4e2f1fa,I22e66972,I9fbd55a9,Ifca22e44 into mainjade
* changes: sqlite: add a Use::fromStrNullable util: implement charptr_cast tree-wide: fix a pile of lints refactor: make HashType and Base enum classes for type safety build: integrate clang-tidy into CI
2024-08-08sqlite: add a Use::fromStrNullableJade Lovelace
There were several usages of the raw sqlite primitives along with C style casts, seemingly because nobody thought to use an optional for getting a string or NULL. Let's fix this API given we already *have* a wrapper. Change-Id: I526cceedc2e356209d8fb62e11b3572282c314e8
2024-08-08tree-wide: automated migration to charptr_castJade Lovelace
The lint did it :3 Change-Id: I2d9f276b01ebbf14101de4257ea13e44ff6fe0a0