aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-10-14fix: macOS build broken by fatal lowdown CLI sandbox setupJade Lovelace
This failed due to https://github.com/NixOS/nixpkgs/pull/346945, which makes a second lowdown-unsandboxed that works in nix builds, and the regular lowdown has executables that fail closed when the sandbox setup fails. The actual failure here is only visible on nixos-unstable at the moment, not 24.05, but this commit should fix it up for all versions. Fixes: https://git.lix.systems/lix-project/lix/issues/547 Change-Id: I50c0ecb59518ef01a7c0181114c1b4c5a7c6b78b
2024-10-14Merge changes I327db40f,If762efce into mainjade
* changes: testsuite: use xdist for parallel test running testsuite: add a functional2 test suite based on pytest
2024-10-12worker: respect C-c on `sudo nix-build`Maximilian Bosch
While debugging something else I observed that latest `main` ignores `Control-C` on `sudo nix-build`. After reading through the capnproto docs, it seems as if the promise must be fulfilled to actually terminate the `promise.wait()` below. This also applies to scenarios such as stopping the client (`nix-build`), but the builders on the daemon-side are still running, i.e. closes #540 Co-authored-by: eldritch horrors <pennae@lix.systems> Change-Id: I9634d14df4909fc1b65d05654aad0309bcca8a0a
2024-10-09testsuite: use xdist for parallel test runningJade Lovelace
This is capped at 12 because 3.7 seconds of startup is painful enough and 5.5 seconds with 24 was more annoying. Change-Id: I327db40fd98deaa5330cd9cf6de99fb07b2c1cb0
2024-10-09testsuite: add a functional2 test suite based on pytestJade Lovelace
I am tired of bad shell scripts, let me write bad python quickly instead. It's definitely, $100%, better. This is not planned as an immediate replacement of the old test suite, but we::jade would not oppose tests getting ported. What is here is a mere starting point and there is a lot more functionality that we need. Fixes: https://git.lix.systems/lix-project/lix/issues/488 Change-Id: If762efce69030bb667491b263b874c36024bf7b6
2024-10-09Merge "Remove static initializers for `RegisterLegacyCommand`" into mainjade
2024-10-09Merge changes Ib27cb43d,I03687b8b into mainjade
* changes: testsuite: override NIX_CONF_DIR and NIX_USER_CONF_FILES Remove some outdated `make test` invocation suggestions
2024-10-08Merge "Add release note for CTRL-C improvements" into mainrebecca “wiggles” turner
2024-10-08Set c++ version to c++23Lulu
I followed @pennae's advice and moved the constructor definition of `AttrName` from the header file `nixexpr.hh` to `nixexpr.cc`. Change-Id: I733f56c25635b366b11ba332ccec38dd7444e793
2024-10-08Fix gcc warning -Wmissing-field-initializersLulu
The approach that was taken here was to add default values to the type definitions rather than specify them whenever they are missing. Now the only remaining warning is '-Wunused-parameter' which @jade said is usually counterproductive and that we can just disable it: https://git.lix.systems/lix-project/lix/issues/456#issuecomment-6617 So this change adds the flags '-Wall', '-Wextra' and '-Wno-unused-parameter', so that all warnings are enabled except for '-Wunused-parameter'. Change-Id: Ic223a964d67ab429e8da804c0721ba5e25d53012
2024-10-08Merge "Avoid calling memcpy when len == 0 in filetransfer.cc" into mainLulu
2024-10-08Fix gcc warning -Wsign-compareLulu
Add the compile flag '-Wsign-compare' and adapt the code to fix all cases of this warning. Change-Id: I26b08fa5a03e4ac294daf697d32cf9140d84350d
2024-10-08Avoid calling memcpy when len == 0 in filetransfer.ccLulu
There was a bug report about a potential call to `memcpy` with a null pointer which is not reproducible: https://git.lix.systems/lix-project/lix/issues/492 This occurred in `src/libstore/filetransfer.cc` in `InnerSource::read`. To ensure that this doesn't happen, an early return is added before calling `memcpy` if the length of the data to be copied is 0. This change also adds a test that ensures that when `InnerSource::read` is called with an empty file, it throws an `EndOfFile` exception. Change-Id: Ia18149bee9a3488576c864f28475a3a0c9eadfbb
2024-10-05libstore: remove Worker::{childStarted, goalFinished}eldritch horrors
these two functions are now nearly trivial and much better inline into makeGoalCommon. keeping them separate also separates information about goal completion flows and how failure information ends up in `Worker`. Change-Id: I6af86996e4a2346583371186595e3013c88fb082
2024-10-05libstore: remove Worker::removeGoaleldritch horrors
we can use our newfound powers of Goal::work Is A Real Promise to remove completed goals from continuation promises. apart from being much easier to follow it's also a lot more efficient because we have the iterator to the item we are trying to remove, skipping a linear search of the cache. Change-Id: Ie0190d051c5f4b81304d98db478348b20c209df5
2024-10-05libstore: remove Goal::notifyeldritch horrors
Goal::work() is a fully usable promise that does not rely on the worker to report completion conditions. as such we no longer need the `notify` field that enabled this interplay. we do have to clear goal caches when destroying the worker though, otherwise goal promises may (incorrectly) keep goals alive due to strong shared pointers created by childStarted. Change-Id: Ie607209aafec064dbdf3464fe207d70ba9ee158a
2024-10-05libstore: move Goal::ex to WorkResulteldritch horrors
yet another duplicated field. it's the last one though. Change-Id: I352df8d306794d262d8c9066f3be78acd40e82cf
2024-10-05libstore: move Goal::buildResult to WorkResulteldritch horrors
derivation goals still hold a BuildResult member variable since parts of these results of accumulated in different places, but the Goal class now no longer has such a field. substitution goals don't need it at all, and derivation goals should also be refactored to not drop their buildResult Change-Id: Ic6d3d471cdbe790a6e09a43445e25bedec6ed446
2024-10-05libstore: move Goal::exitCode to WorkResulteldritch horrors
the field is simply duplicated between the two, and now that we can return WorkResults from Worker::run we no longer need both of them. Change-Id: I82fc47d050b39b7bb7d1656445630d271f6c9830
2024-10-05libstore: return goal results from Worker::run()eldritch horrors
this will be needed to move all interesting result fields out of Goal proper and into WorkResult. once that is done we can treat goals as a totally internal construct of the worker mechanism, which also allows us to fully stop exposing unclear intermediate state to Worker users. Change-Id: I98d7778a4b5b2590b7b070bdfc164a22a0ef7190
2024-10-05libstore: remove Worker::topGoalseldritch horrors
since we now propagate goal exceptions properly we no longer need to check topGoals for a reason to abort early. any early abort reasons, whether by exception or a clean top goal failure, can now be handled by inspecting the goal result in the main loop. this greatly reduces goal-to-goal interactions that do not happen at the main loop level. since the underscore-free name is now available for use as variables we'll migrate to that where we currently use `_topGoals` for locals. Change-Id: I5727c5ea7799647c0a69ab76975b1a03a6558aa6
2024-10-05libstore: propagate goal exceptions using promiseseldritch horrors
drop childException since it's no longer needed. also makes waitForInput, childFinished, and childTerminated redundant. Change-Id: I05d88ffd323c5b5c909ac21056162f69ffb0eb9f
2024-10-05libstore: have goals promise WorkResults, not voideldritch horrors
Change-Id: Idd218ec1572eda84dc47accc0dcd8a954d36f098
2024-10-05libstore: extract Worker::goalFinished specificseldritch horrors
there's no reason to have the worker set information on goals that the goals themselves return from their entry point. doing this in the goal `work()` function is much cleaner, and a prerequisite to removing more implicit strong shared references to goals that are currently running. Change-Id: Ibb3e953ab8482a6a21ce2ed659d5023a991e7923
2024-10-05libstore: check for interrupts in parallel promiseeldritch horrors
this simplifies the worker loop, and lets us remove it entirely later. note that ideally only one promise waiting for interrupts should exist in the entire system. not one per event loop, one per *process*. extra interrupt waiters make interrupt response nondeterministic and as such aren't great for user experience. if anything wants to react to aborts caused by explicit interruptions, or anything else, those things would be much better served using RAII guards such as Finally (or KJ_DEFER). Change-Id: I41d035ff40172d536e098153c7375b0972110d51
2024-10-05libstore: remove Goal::StillAliveeldritch horrors
this was a triumph. i'm making a note here: huge success. it's hard to overstate my satisfaction! i'm not even angry. i'm being so sincere ri actually, no. we *are* angry. this was one dumbass odyssey. nobody has asked for this. but not doing it would have locked us into old, broken protocols forever or (possibly worse) forced us to write our own async framework building on the old did-you-mean-continuations in Worker. if we had done that we'd be locked into ever more, and ever more complex, manual state management all over the place. this just could not stand. Change-Id: I43a6de1035febff59d2eff83be9ad52af4659871
2024-10-05Add release note for CTRL-C improvementsRebecca Turner
I'm very excited for cl/2016, so others will probably be excited also! Let's add a release note. Change-Id: Ic84a4444241aafce4cb6d5a6d1dddb47e7a7dd7b
2024-10-05Merge "Split ignoreException to avoid suppressing CTRL-C" into mainrebecca “wiggles” turner
2024-10-05fix build test error count checkseldritch horrors
with async runtime scheduling we can no longer guarantee exact error counts for builds that do not set keepGoing. the old behavior can be recovered with a number of hacks that affect scheduling, but none of those are very easy to follow now advisable. exact error counts will like not be needed for almost all uses except tests, and *those* had better check the actual messages rather than how many they got. more messages can even help to avoid unnecessary rebuilds for most users. Change-Id: I1c9aa7a401227dcaf2e19975b8cb83c5d4f85d64
2024-10-05doc: install the HTML manual againAlois Wohlschlager
In 0e6b3435a14a304b8833c27d2911de7ac4e731d4, installation of the HTML manual was accidentally dropped: setting install_dir on a custom_target only sets the directory where something is going to be installed if it is installed at all, but does not itself trigger installation. The latter has to be explicitly requested, which is just what we do here to get the manual back. Change-Id: Iff8b791de7e7cb4c8d747c2a9b1154b5fcc32fe0
2024-10-04testsuite: override NIX_CONF_DIR and NIX_USER_CONF_FILESJade Lovelace
The test suite can load the global configuration files under certain circumstances, and, though we would really rather it didn't ever do that at all, we should at least break the mechanism. Fixes: https://git.lix.systems/lix-project/lix/issues/474 Change-Id: Ib27cb43dd5dfaa70ac491c395b5ba308fd7bd289
2024-10-04Remove some outdated `make test` invocation suggestionsJade Lovelace
These should be meson. Change-Id: I03687b8b03f50fb1684e7ffcd487be855052d6c2
2024-10-04libstore: forbid addWantedGoals when finishedeldritch horrors
due to event loop scheduling behavior it's possible for a derivation goal to fully finish (having seen all paths it was asked to create), but to not notify the worker of this in time to prevent another goal asking the recently-finished goal for more outputs. if this happened the finished goal would ignore the request for more outputs since it considered itself fully done, and the delayed result reporting would cause the requesting goal to assume its request had been honored. if the requested goal had finished *properly* the worker would recreate it instead of asking for more outputs, and this would succeed. it is thus safe to always recreate goals once they are done, so we now do. Change-Id: Ifedd69ca153372c623abe9a9b49cd1523588814f
2024-10-04Merge "editorconfig: Add meson.build" into mainrebecca “wiggles” turner
2024-10-04Merge "internal-api-docs: allow Doxygen to build regardless of workdir" into ↵Olivia Crain
main
2024-10-04Merge "build: let meson add compiler flags for libstdc++ assertions" into mainOlivia Crain
2024-10-01editorconfig: Add meson.buildRebecca Turner
Change-Id: Ibb59ddc21f5d3ef7fb4c900e3413e426c201334d
2024-10-01Remove static initializers for `RegisterLegacyCommand`Rebecca Turner
This moves the "legacy"/"nix2" commands under a new `src/legacy/` directory, instead of being scattered around in a bunch of different directories. A new `liblegacy` build target is defined, and the `nix` binary is linked against it. Then, `RegisterLegacyCommand` is replaced with `LegacyCommand::add` calls in functions like `registerNixCollectGarbage()`. These registration functions are called explicitly in `src/nix/main.cc`. See: https://git.lix.systems/lix-project/lix/issues/359 Change-Id: Id450ffc3f793374907599cfcc121863b792aac1a
2024-10-01Split ignoreException to avoid suppressing CTRL-CRobert Hensing
This splits `ignoreException` into `ignoreExceptionExceptInterrupt` (which ignores all exceptions except `Interrupt`, which indicates a SIGINT/CTRL-C) and `ignoreExceptionInDestructor` (which ignores all exceptions, so that destructors do not throw exceptions). This prevents many cases where Nix ignores CTRL-C entirely. See: https://github.com/NixOS/nix/issues/7245 Upstream-PR: https://github.com/NixOS/nix/pull/11618 Change-Id: Ie7d2467eedbe840d1b9fa2e88a4e88e4ab26a87b
2024-10-01libstore: turn DerivationGoal::work into *one* promiseeldritch horrors
Change-Id: Ic2f7bc2bd6a1879ad614e4be81a7214f64eb0e85
2024-10-01libstore: turn DrvOutputSubstitutionGoal::work into *one* promiseeldritch horrors
Change-Id: I2d4dcedff0a278d2d8f3d264a9186dfb399275e2
2024-10-01libstore: make PathSubstitutionGoal::work *one* promiseeldritch horrors
Change-Id: I38cfe8c7059251b581f1013c4213804f36b985ea
2024-10-01libstore: turn Worker::updateStatistics into a promiseeldritch horrors
we'll now loop to update displayed statistics, and use this loop to limit the update rate to 50 times per second. we could have updated much more frequently before this (once per iteration of `runImpl`), much faster than would ever be useful in practice. aggressive stats updates can even impede progress due to terminal or network delays. Change-Id: Ifba755a2569f73c919b1fbb06a142c0951395d6d
2024-10-01libstore: remove Worker::wakeUp()eldritch horrors
Worker::run() is now entirely based on the kj event loop and promises, so we need not handle awakeness of goals manually any more. every goal can instead, once it has finished a partial work call, defer itself to being called again in the next iteration of the loop. same end effect. Change-Id: I320eee2fa60bcebaabd74d1323fa96d1402c1d15
2024-10-01libstore: turn periodic gc attempt into a promiseeldritch horrors
notably we will check whether we want to do GC at all only once during startup, and we'll only attempt GC every ten seconds rather than every time a goal has finished a partial work call. this shouldn't cause any problems in practice since relying on auto-gc is not deterministic and stores in which builds can fill all remaining free space in merely ten seconds are severely troubled even when gargage collection runs a lot. Change-Id: I1175a56bf7f4e531f8be90157ad88750ff2ddec4
2024-10-01libstore: turn Worker::run() main loop into a promiseeldritch horrors
Change-Id: Ib112ea9a3e67d5cb3d7d0ded30bbd25c96262470
2024-10-01libstore: turn waitForInput into a promiseeldritch horrors
Change-Id: I8355d8d3f6c43a812990c1912b048e5735b07f7b
2024-10-01Revert "libstore: remove worker removeGoal"Raito Bezarius
Revert submission 1946 Reason for revert: regression in building (found via bisection) Reported by users: > error: path '/nix/store/04ca5xwvasz6s3jg0k7njz6rzi0d225w-jq-1.7.1-dev' does not exist in the store Reverted changes: /q/submissionid:1946 Change-Id: I6f1a4b2f7d7ef5ca430e477fc32bca62fd97036b
2024-09-30Merge "fix(nix fmt): remove the default "." argument" into mainJonas Chevalier
2024-09-29libstore: merge ContinueImmediately and StillAliveeldritch horrors
nothing needs to signal being still active but not actively pollable, only that immediate polling for the next goal work phase is in order. Change-Id: Ia43c1015e94ba4f5f6b9cb92943da608c4a01555