aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-10-20gitignore: ignore *.pyc filesLily Ballard
Running our installcheck test suite creates these files. Change-Id: I97ac8f1aa165a491c55dff6b48486db17b75443b
2024-10-20libstore: fix sign comparison warnings in darwin platformLily Ballard
It's not clear to me if `proc_pidinfo()` or `proc_pidfdinfo()` can actually return negative values, the syscall wrappers convert `-1` into zero and the semantics suggest that negative values don't make sense, but just to be safe we'll preserve the int type until we've checked that it's a positive value. Fixes: https://git.lix.systems/lix-project/lix/issues/548 Change-Id: If575aec6b1e27dba63091c7a0316c7b3788747cd
2024-10-19libutil: disallow AsyncCollect relocationseldritch horrors
some promises capture `this`. we could also allocate a shared state, but this thing doesn't really need to ever be moved anyway. so there. Change-Id: I50b5c44684a8ab4e984b1323de21f97ace4a864a
2024-10-19Merge "libstore: always release build/substitution slot tokens" into maineldritch horrors
2024-10-19libstore: always release build/substitution slot tokenseldritch horrors
not doing this can freeze slots until the goal that occupied them is freed (rather than simply complete), and then can freeze the system. fixes #549 Change-Id: I042df04222f8ffbaa18ef4a4eae6cbd6f89b679e
2024-10-19Merge "flake.lock: update everything" into mainjade
2024-10-19Merge "testsuite: Fix tests on systems with a non-master defaultBranch" into ↵jade
main
2024-10-18packaging: use in-tree capnproto derivationJade Lovelace
This is done because the one in nixpkgs has several problems and we don't want to conflict with them fixing those problems: - not building shared libs - not building debuginfo (more critical for us due to https://git.lix.systems/lix-project/lix/issues/549) - not setting the correct cmake build type - not setting the correct cxxflags to build the coroutine library - not building with clang (required for the coroutine library since gcc is known to miscompile coroutines *in kj*). CC: https://git.lix.systems/lix-project/lix/issues/551 Fixes: https://git.lix.systems/lix-project/lix/issues/550 Change-Id: Ia5b78dc1809963fdd1a8203b127a216cb575d751
2024-10-18packaging: remove workaround for clang stdenv assertsJade Lovelace
This workaround was for the stdenv not being set when callPackage'ing package.nix for some of the stranger CI outputs. Change-Id: I2acdd6efa721b90dd3cb04358544d25d591ff084
2024-10-18libexpr: Fix typo in error messagepiegames
Closes #523 Change-Id: Ib5705e405b74d07a8fcf0163847405e9c791c3e3
2024-10-18benchmarks: Fix puritypiegames
Some stuff wasn't working on my machine because it depended on the host environment in some ways, fixed those. Change-Id: Iff4931a9a26c6827978f1ee6434710f406d67a96
2024-10-18libexpr: Optimize complex indented stringspiegames
The old behavior results in lots of concatenations happening for no good reason and is an artifact of the technical limitations of the old parser (combined with some lack of care for such details). Change-Id: I0d78d6220ca6aeaa10bc437e48e08bf7922e0bb3
2024-10-18libexpr: Print interpolations more accurately in `show`piegames
This is only a minor semantical distinction, but we should be able to properly test it, and the parser tests rely on show for that. Change-Id: I25e868cf9544e30cdff17deb5fd50a434e0f367e
2024-10-18libexpr: Rewrite stripIndentation for indented stringspiegames
This commit should faithfully reproduce the old behavior down to the bugs. The new code is a lot more readable, all quirks are well documented, and it is overall much more maintainable. Change-Id: I629585918e4f2b7d296b6b8330235cdc90b7bade
2024-10-18tests/functional-lang: Add test cases for indented stringspiegames
Accidentally came across them elsewhere, wanted them covered in the ind-strings test as well. Change-Id: Iba418a687388ba85516d13d4c9d815744297dc5c
2024-10-18tests/functional-lang: Test indented string parsingpiegames
There is already an eval test for it, but it doesn't test the parser's output Change-Id: I94809080f90ace8aa3f610bcd315e9df0cf5d12f
2024-10-18libexpr: move parser semantics into separate filepiegames
Another preparation for forking off and versioning the parser Change-Id: I7b1225a44a3b81486414c1d37bd3e76a3ab307f9
2024-10-18libexpr: rename grammar to grammar::v1piegames
Let's make some space in the namespace for a v2 https://wiki.lix.systems/books/lix-contributors/page/nix-lang-v2 Change-Id: If56e6dbf680d931233aa822ef91c8832464471e4
2024-10-18tests/functional-lang: Small cleanup of flagfiles handlingpiegames
Flag files now properly work for all tests. Change-Id: If652235960e8cf4120270a1b974c17586a31a157
2024-10-17Merge "fix: macOS build broken by fatal lowdown CLI sandbox setup" into mainjade
2024-10-15Merge "Fix std::terminate call in thread pool" into mainjade
2024-10-14fix: macOS build broken by fatal lowdown CLI sandbox setupJade Lovelace
This failed due to https://github.com/NixOS/nixpkgs/pull/346945, which makes a second lowdown-unsandboxed that works in nix builds, and the regular lowdown has executables that fail closed when the sandbox setup fails. The actual failure here is only visible on nixos-unstable at the moment, not 24.05, but this commit should fix it up for all versions. Fixes: https://git.lix.systems/lix-project/lix/issues/547 Change-Id: I50c0ecb59518ef01a7c0181114c1b4c5a7c6b78b
2024-10-14Merge "Remove ancient `let` from 2 test files" into mainV.
2024-10-14Remove ancient `let` from 2 test filesvigress8
Change-Id: I992bc7f9e1cfcb1e4038fbe6ee04178bbf938556
2024-10-14ssh-ng: better way to keep SSH errors visibleMaximilian Bosch
A better fix than in 104448e75d87d03d2fb0b4ac96d4da72e1dae50d, hence a revert + the fix. It turns out that this commit has the side-effect that when having e.g. `StrictHostKeyChecking=accept-new` for a remote builder, the warnings à la Warning: Permanently added 'builder' (ED25519) to the list of known hosts. actually end up in the derivation's log whereas hostkey verification errors don't, but only in the stderr of the `nix-build` invocation (which was the motivation for the patch). This change writes the stderr from the build-hook to * the daemon's stderr, so that the SSH errors appear in the journal (which was the case before 104448e75d87d03d2fb0b4ac96d4da72e1dae50d) * the client's stderr, as a log message * NOT to the drv log (this is handled via `handleJSONLogMessage`) I tried to fix the issue for legacy-ssh as well, but failed and ultimately decided to not bother. I know that we'll sooner or later replace the entire component, however this is the part of the patch I have working for a while, so I figured I might still submit it for the time being. Change-Id: I21ca1aa0d8ae281d2eacddf26e0aa825272707e5
2024-10-14testsuite: Fix tests on systems with a non-master defaultBranchZebreus
When the git default branch is not set to master the installcheck test suite fails. This patch adjusts the test setup scripts to ignore the system and user git config files. GIT_CONFIG_SYSTEM is set to /dev/null to ignore /etc/gitconfig GIT_CONFIG_GLOBAL is not set because the global config files are loaded from $HOME or $XDG_CONFIG_HOME which we already reset. git documentation: https://git-scm.com/docs/git#Documentation/git.txt-codeGITCONFIGGLOBALcode Change-Id: Ie73bbed1db9419c9885b9d57e4edb7a4047d5cce
2024-10-14Merge changes I327db40f,If762efce into mainjade
* changes: testsuite: use xdist for parallel test running testsuite: add a functional2 test suite based on pytest
2024-10-12worker: respect C-c on `sudo nix-build`Maximilian Bosch
While debugging something else I observed that latest `main` ignores `Control-C` on `sudo nix-build`. After reading through the capnproto docs, it seems as if the promise must be fulfilled to actually terminate the `promise.wait()` below. This also applies to scenarios such as stopping the client (`nix-build`), but the builders on the daemon-side are still running, i.e. closes #540 Co-authored-by: eldritch horrors <pennae@lix.systems> Change-Id: I9634d14df4909fc1b65d05654aad0309bcca8a0a
2024-10-09Fix std::terminate call in thread poolJade Lovelace
So we received a report that the thread pool crashed due to an Interrupted exception. Relevant log tail: copying path '/nix/store/0kal2k73inviikxv9f1ciaj39lkl9a87-etc-os-release' to 'ssh://192.168.0.27'... Lix crashed. This is a bug. We would appreciate if you report it along with what caused it at https://git.lix.systems/lix-project/lix/issues with the following information included: error (ignored): error: interrupted by the user Exception: nix::Interrupted: error: interrupted by the user Relevant stack trace: 4# __cxa_rethrow in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6 5# nix::ignoreExceptionExceptInterrupt(nix::Verbosity) in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so 6# nix::ThreadPool::doWork(bool) in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so 7# 0x00007FA7A00E86D3 in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6 8# 0x00007FA79FE99A42 in /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 9# 0x00007FA79FF1905C in /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 Notably, this is *not* in the main thread, so this implies that the thread didn't get joined properly before their destructors got called. That, in turn, should have only possibly happened because join() threw on a previous iteration of the loop joining threads, I think. Or if it threw while in the ThreadPool destructor. Either way we had better stop letting Interrupted fall out of our child threads! If: - Interrupted was thrown inside the action in the main thread: it would have fallen out of doWork if state->exception was already set and got caught by ThreadPool::process, calling shutdown() and the join loop which would crash the process entirely. - Interrupted was thrown inside the action on a secondary thread: it would have been caught and put into the exception field and then possibly rethrown to fall out of the thread (since it was previously ignoreExceptionExceptInterrupt). The one possible hole in this hypothesis is that there is an "error (ignored)" line in there implying that at least one Interrupted got eaten by an ignoreExceptionInDestructor. It's also unclear whether this got reordered because of stderr buffering. Fixes: https://git.lix.systems/lix-project/lix/issues/542 Change-Id: I322cf050da660af78f5cb0e08ec6e6d27d09ac76
2024-10-09testsuite: use xdist for parallel test runningJade Lovelace
This is capped at 12 because 3.7 seconds of startup is painful enough and 5.5 seconds with 24 was more annoying. Change-Id: I327db40fd98deaa5330cd9cf6de99fb07b2c1cb0
2024-10-09testsuite: add a functional2 test suite based on pytestJade Lovelace
I am tired of bad shell scripts, let me write bad python quickly instead. It's definitely, $100%, better. This is not planned as an immediate replacement of the old test suite, but we::jade would not oppose tests getting ported. What is here is a mere starting point and there is a lot more functionality that we need. Fixes: https://git.lix.systems/lix-project/lix/issues/488 Change-Id: If762efce69030bb667491b263b874c36024bf7b6
2024-10-09Merge "Remove static initializers for `RegisterLegacyCommand`" into mainjade
2024-10-09Merge changes Ib27cb43d,I03687b8b into mainjade
* changes: testsuite: override NIX_CONF_DIR and NIX_USER_CONF_FILES Remove some outdated `make test` invocation suggestions
2024-10-08Merge "Add release note for CTRL-C improvements" into mainrebecca “wiggles” turner
2024-10-08Set c++ version to c++23Lulu
I followed @pennae's advice and moved the constructor definition of `AttrName` from the header file `nixexpr.hh` to `nixexpr.cc`. Change-Id: I733f56c25635b366b11ba332ccec38dd7444e793
2024-10-08Fix gcc warning -Wmissing-field-initializersLulu
The approach that was taken here was to add default values to the type definitions rather than specify them whenever they are missing. Now the only remaining warning is '-Wunused-parameter' which @jade said is usually counterproductive and that we can just disable it: https://git.lix.systems/lix-project/lix/issues/456#issuecomment-6617 So this change adds the flags '-Wall', '-Wextra' and '-Wno-unused-parameter', so that all warnings are enabled except for '-Wunused-parameter'. Change-Id: Ic223a964d67ab429e8da804c0721ba5e25d53012
2024-10-08Merge "Avoid calling memcpy when len == 0 in filetransfer.cc" into mainLulu
2024-10-08Fix gcc warning -Wsign-compareLulu
Add the compile flag '-Wsign-compare' and adapt the code to fix all cases of this warning. Change-Id: I26b08fa5a03e4ac294daf697d32cf9140d84350d
2024-10-08Avoid calling memcpy when len == 0 in filetransfer.ccLulu
There was a bug report about a potential call to `memcpy` with a null pointer which is not reproducible: https://git.lix.systems/lix-project/lix/issues/492 This occurred in `src/libstore/filetransfer.cc` in `InnerSource::read`. To ensure that this doesn't happen, an early return is added before calling `memcpy` if the length of the data to be copied is 0. This change also adds a test that ensures that when `InnerSource::read` is called with an empty file, it throws an `EndOfFile` exception. Change-Id: Ia18149bee9a3488576c864f28475a3a0c9eadfbb
2024-10-05libstore: remove Worker::{childStarted, goalFinished}eldritch horrors
these two functions are now nearly trivial and much better inline into makeGoalCommon. keeping them separate also separates information about goal completion flows and how failure information ends up in `Worker`. Change-Id: I6af86996e4a2346583371186595e3013c88fb082
2024-10-05libstore: remove Worker::removeGoaleldritch horrors
we can use our newfound powers of Goal::work Is A Real Promise to remove completed goals from continuation promises. apart from being much easier to follow it's also a lot more efficient because we have the iterator to the item we are trying to remove, skipping a linear search of the cache. Change-Id: Ie0190d051c5f4b81304d98db478348b20c209df5
2024-10-05libstore: remove Goal::notifyeldritch horrors
Goal::work() is a fully usable promise that does not rely on the worker to report completion conditions. as such we no longer need the `notify` field that enabled this interplay. we do have to clear goal caches when destroying the worker though, otherwise goal promises may (incorrectly) keep goals alive due to strong shared pointers created by childStarted. Change-Id: Ie607209aafec064dbdf3464fe207d70ba9ee158a
2024-10-05libstore: move Goal::ex to WorkResulteldritch horrors
yet another duplicated field. it's the last one though. Change-Id: I352df8d306794d262d8c9066f3be78acd40e82cf
2024-10-05libstore: move Goal::buildResult to WorkResulteldritch horrors
derivation goals still hold a BuildResult member variable since parts of these results of accumulated in different places, but the Goal class now no longer has such a field. substitution goals don't need it at all, and derivation goals should also be refactored to not drop their buildResult Change-Id: Ic6d3d471cdbe790a6e09a43445e25bedec6ed446
2024-10-05libstore: move Goal::exitCode to WorkResulteldritch horrors
the field is simply duplicated between the two, and now that we can return WorkResults from Worker::run we no longer need both of them. Change-Id: I82fc47d050b39b7bb7d1656445630d271f6c9830
2024-10-05libstore: return goal results from Worker::run()eldritch horrors
this will be needed to move all interesting result fields out of Goal proper and into WorkResult. once that is done we can treat goals as a totally internal construct of the worker mechanism, which also allows us to fully stop exposing unclear intermediate state to Worker users. Change-Id: I98d7778a4b5b2590b7b070bdfc164a22a0ef7190
2024-10-05libstore: remove Worker::topGoalseldritch horrors
since we now propagate goal exceptions properly we no longer need to check topGoals for a reason to abort early. any early abort reasons, whether by exception or a clean top goal failure, can now be handled by inspecting the goal result in the main loop. this greatly reduces goal-to-goal interactions that do not happen at the main loop level. since the underscore-free name is now available for use as variables we'll migrate to that where we currently use `_topGoals` for locals. Change-Id: I5727c5ea7799647c0a69ab76975b1a03a6558aa6
2024-10-05libstore: propagate goal exceptions using promiseseldritch horrors
drop childException since it's no longer needed. also makes waitForInput, childFinished, and childTerminated redundant. Change-Id: I05d88ffd323c5b5c909ac21056162f69ffb0eb9f
2024-10-05libstore: have goals promise WorkResults, not voideldritch horrors
Change-Id: Idd218ec1572eda84dc47accc0dcd8a954d36f098
2024-10-05libstore: extract Worker::goalFinished specificseldritch horrors
there's no reason to have the worker set information on goals that the goals themselves return from their entry point. doing this in the goal `work()` function is much cleaner, and a prerequisite to removing more implicit strong shared references to goals that are currently running. Change-Id: Ibb3e953ab8482a6a21ce2ed659d5023a991e7923