aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2024-04-05Revert "libutil: remove Pool::Handle::bad"eldritch horrors
This reverts commit 792844fb861ea7367ac2316c78fec055363f2f9e. Change-Id: I3ca208b62edfd5cd1199478f75cd2edf19a364f6
2024-04-05Revert "libstore: remove one Resource::good flag"eldritch horrors
This reverts commit 87249eb579bf57f4f09e9fca100588a4d6b90b4c. Change-Id: Ide4c6e00c4155216a17e46671ff47151d7bb85b4
2024-04-05Revert "libstore: using throwing finally in withFramedSink"eldritch horrors
This reverts commit 491caad6f62c21ffbcdebe662e63ec0f72e6f3a2. this is not actually legal for nix! throwing exceptions in destructors is fine, but the way nix is set up we'll end up throwing the exception we received from the remote *twice* in some cases, and such cases will cause an immediate terminate without active exception. Change-Id: I74c46b9f26fd791086e4193ec60eb1deb9a5bb2a
2024-04-05Revert "libutil: drop Fs{Source,Sink}::good"eldritch horrors
This reverts commit 1340807e30dba4b3972c31f02861bbaeaeb60e61. Change-Id: I34d2a80eb3c3e9d79cb02b92cd1189da32d18cb6
2024-03-31libutil: drop Fs{Source,Sink}::goodeldritch horrors
setting this only on exceptions caused by actual fd access is not sufficient to diagnose all errors (such as SerialisationError) in some cases. this usually does not have any negative effects since those errors will end up killing the process in another way. this is not a reliable assumption though and we should be using proper error handling (and closing connections more often, preferring to close over keeping something open that might be in a weird state) Change-Id: I1b792cd7ad8ba9ff0f6bd174945ab2575ff2208e
2024-03-31Merge changes Ib62d3d68,Ic3e7affe into mainjade
* changes: Make things that can throw not noexcept anymore Fix various clang-tidy lints
2024-03-31libstore: using throwing finally in withFramedSinkeldritch horrors
the duplication of exception handling was added without justification, so we can only assume that it was done like this because Finally could not throw exceptions safely. since this has now been rectified we will deduplicate this handler code again. Change-Id: I40721f3378c0fd9f34e2914a16d383f6e2713b40
2024-03-31libstore: remove one Resource::good flageldritch horrors
usage of this flag previously kept connections open much longer than necessary, and at the same time obscured that a connection was being dropped when it *was* set. new variable names clarify this somewhat. Change-Id: I11f6f08f37a5e4dc04ea6c6036ea589154b121c6
2024-03-31libutil: remove Pool::Handle::badeldritch horrors
it was used incorrectly (not swapped on handle move), only used in one place (that is now handled with exception handling detection in Handle itself), and if ever reintroduced should be replaced with a different, more understandable mechanism (like an explicit dropAsInvalid method). Change-Id: Ie3e5d5cfa81d335429cb2ee5c3ad85c74a9df17b
2024-03-31libutil: remove Pool::flushBadeldritch horrors
this was never actually used, and bad design in the first place—why should a bad resource be put back into the idle pool? just drop it. Change-Id: Idab8774bee19dadae0209d404c4fb86dd4aeba1e
2024-03-29Add `pre-commit` checksRebecca Turner
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer` (which makes sure that every file ends with exactly one newline character). Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29Make things that can throw not noexcept anymoreJade Lovelace
This does involve making a large number of destructors able to throw, because we had to change it high in the class hierarchy. Oh well. Change-Id: Ib62d3d6895b755f20322bb8acc9bf43daf0174b2
2024-03-29Fix various clang-tidy lintsJade Lovelace
* some things that can throw are marked noexcept yet the linter seems to think not. Maybe they can't throw in practice. I would rather not have the UB possibility in pretty obvious cold paths. * various default-case-missing complaints * a fair pile of casts from integer to character, which are in fact deliberate. * an instance of <https://clang.llvm.org/extra/clang-tidy/checks/bugprone/move-forwarding-reference.html> * bugprone-not-null-terminated-result on handing a string to curl in chunks of bytes. our usage is fine. * reassigning a unique_ptr by CRIMES instead of using release(), then using release() and ignoring the result. wild. let's use release() for its intended purpose. Change-Id: Ic3e7affef12383576213a8a7c8145c27e662513d
2024-03-29meson: fix state-dir default valueeldritch horrors
the autoconf build system defaults to /nix/var, not /nix/var/nix. the latter is only used in libstore, so we'll move the extra segment there. Change-Id: Idfbc988ee302355982abdcd51d6d7b5d5d661c0d
2024-03-29meson: add missing explicit dependency on nlohmann_jsonWinter
Without this, the Meson setup won't bail out if nlohmann_json is missing, leading to subpar DX (and maybe worse, but I'm not entirely sure). Change-Id: I5913111060226b540dcf003257c99a08e84da0de
2024-03-29meson: install missing/generated headerseldritch horrors
one headers (args/root.hh) was simply missing, and the generated headers were not installed. not all of them *should* be installed either, only a select few (and sadly this needs a custom target for each one, it seems) Change-Id: I37b25517895d0e5e521abc1202fa65624de57ed1
2024-03-29meson: install pkg-config files for librarieseldritch horrors
Change-Id: I14b9d81d09f188eacfb9c68bcfb84751c18e3779
2024-03-29Merge "feat: unprivileged read-only open of SQLite DB" into mainRaito Bezarius
2024-03-27HOT SALE: 15% off your build times!Jade Lovelace
This was achieved by running maintainers/buildtime_report.sh on the build directory of a meson build, then asking "why the heck is json eating our build times", and strategically moving the json using bits out of widely included headers. It turns out that putting literally any metrics whatsoever into the build had immediate and predictable results. Results are 1382.5s frontend time -> 1175.4s frontend time, back end time approximately invariant. Related: https://git.lix.systems/lix-project/lix/issues/159 Change-Id: I7edea95c8536203325c8bb4dae5f32d727a21b2d
2024-03-26libstore/filetransfer: use Lix UA and unnix error messageRaito Bezarius
Once this commit lands, we are even more visible in analytics FWIW. Change-Id: Id7e0c162315d0f191edbea9cb5fb82ce363704b9 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-24libstore: despecialcase protocol version checkeldritch horrors
protocol versions are sent as u64. on the peer we read them as uint64, check that the upper half is 0, and throw an exception if not. we then read an arbitrary amount of data from the peer and dump it to the user terminal. this is a little bit ridiculous, can never happen in correct implementation, and is severly untested. let us just drop it entirely. Change-Id: Ibd2f53a765341ed6439d40d9d1eac11e79c6b5e3
2024-03-24libstore: un-inline copyNAR expansionseldritch horrors
these are copies of copyNAR with only some variables renamed. Change-Id: I98ddd7a98250fa5d304e18e1debf417e9f7768dd
2024-03-23feat: unprivileged read-only open of SQLite DBRaito Bezarius
If the state SQLite database is configured to use a write-ahead-log, it creates WAL files in the state directory. When the state SQLite database is closed by the `nix-daemon` after builds, those files are removed. When an unprivileged user would like to open _in read only_ that database, they cannot do so because they would need to create those WAL files and they do not have the permission to do so. For this, SQLite offers a "persistent WAL" feature [1] to leave the WAL files around, even after closing the database. This CL enable the persistent WAL mode. Fixes: https://github.com/NixOS/nix/issues/10300 [1]: https://www.sqlite.org/wal.html Change-Id: Id8ae534d7d2290457af28782e5215222ae051fe5 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-22build: optionally build and install with mesonQyriad
This commit adds several meson.build, which successfully build and install Lix executables, libraries, and headers. Meson does not yet build docs, Perl bindings, or run tests, which will be added in following commits. As such, this commit does not remove the existing build system, or make it the default, and also as such, this commit has several FIXMEs and TODOs as notes for what should be done before the existing autoconf + make buildsystem can be removed and Meson made the default. This commit does not modify any source files. A Meson-enabled build is also added as a Hydra job, and to `nix flake check`. Change-Id: I667c8685b13b7bab91e281053f807a11616ae3d4
2024-03-19libutil: remove exception handling workingness checkeldritch horrors
within lix itself this problem is caught by the test suite. outside of lix itself three cases can be had: either the problem is fully inside lix libs, fully inside user code, or it exists at the boundary. the first is caught by the test suite, the second isn't caught at all, and the third is something lix should not be responsible for. Change-Id: I95aa35d8cb6f0ef5816a2941c467bc0c15916063
2024-03-18Merge changes I72c945ca,I2138bb4d,Ib96749f3 into mainjade
* changes: Release notes for builtins.nixVersion change un-nixes ur lix, a little issue importer: list issues that are *not* closed when finding existing issues
2024-03-18un-nixes ur lix, a littleJade Lovelace
I didn't really go attack the docs because we need to pull a bunch of PRs. I went looking for strings in the code that called lix nix. Change-Id: I2138bb4dd239096bc530946b281db7f875195b39
2024-03-18libutil: make AutoCloseFD a better resourceeldritch horrors
add a reset() method to close the wrapped fd instead of assigning magic constants. also make the from-fd constructor explicit so you can't accidentally assign the *wrong* magic constant, or even an unrelated integer that also just happens to be an fd by pure chance. Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-17Delete hasPrefix and hasSuffix from the codebaseJade Lovelace
These now have equivalents in the standard lib in C++20. This change was performed with a custom clang-tidy check which I will submit later. Executed like so: ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
2024-03-15Allow dlopen of plugins to failJade Lovelace
It happens with some frequency that plugins that might be unimportant to the evaluation at hand mismatch with the nix version, leading to spurious load failures. Let's make these non fatal. Change-Id: Iba10e951d171725ccf1a121bcd9be1e1d6ad69eb
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-09Merge pull request #9925 from 9999years/fmt-cleanupeldritch horrors
Cleanup `fmt.hh` (cherry picked from commit 47a1dbb4b8e7913cbb9b4d604728b912e76e4ca0) Change-Id: Id076a45cb39652f437fe3f8bda10c310a9894777
2024-03-09libexpr: Support structured error classeseldritch horrors
While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`. (cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732) Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
2024-03-07Merge pull request #9560 from ↵eldritch horrors
obsidiansystems/serve-proto-unkeyed-valid-path-info-serializer Factor out `ServeProto::Serialiser<UnkeyedValidPathInfo>` and test (cherry picked from commit 139982997eec493a0f74105c427953f6be77da6d) Change-Id: I28e4ba5a681a90d81915a56e6dbaa5456d64f96d
2024-03-07Merge pull request #8544 from edolstra/handle-missing-gc-socketeldritch horrors
LocalStore: :addTempRoot(): Handle ENOENT (cherry picked from commit 7115edc85af060ef235ac0270245ab46cc828f7c) Change-Id: Ie6b1596049c3fde09b98f2f0727899f98e48e6b1
2024-03-06Merge pull request #9280 from R-VdP/rvdp/fix_remote_logging_phase_reportingeldritch horrors
Include phase reporting in log file for ssh-ng builds (cherry picked from commit b1e7d7cad625095656fff05ac4aedeb12135110a) Change-Id: I4076669b0ba160412f7c628ca9113f9abbc8c303
2024-03-07Copy the output of fixed-output derivations before registering themPuck Meerburg
It is possible to exfiltrate a file descriptor out of the build sandbox of FODs, and use it to modify the store path after it has been registered. To avoid that issue, don't register the output of the build, but a copy of it (that will be free of any leaked file descriptor). Test that we can't leverage abstract unix domain sockets to leak file descriptors out of the sandbox and modify the path after it has been registered. (cherry picked from commit 2dadfeb690e7f4b8f97298e29791d202fdba5ca6) (tests cherry picked from commit c854ae5b3078ac5d99fa75fe148005044809e18c) Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Theophane Hufschmitt <theophane.hufschmitt@tweag.io> Co-authored-by: Tom Bereknyei <tomberek@gmail.com> Change-Id: I87cd58f1c0a4f7b7a610d354206b33301e47b1a4
2024-03-05Merge pull request #9634 from 9999years/combine-abstract-pos-and-poseldritch horrors
Combine `AbstractPos`, `PosAdapter`, and `Pos` (cherry picked from commit 113499d16fc87d53b73fb62fe6242154909756ed) === this is a bit cursed because originally it was based on InputAccessor code that we don't have and moved/patched features we likewise don't have (fetchToStore caching, all the individual accessors, ContentAddressMethod). the commit is adjusted accordingly to match (remove caching, ignore accessors, use FileIngestionMethod). note that `state.rootPath . CanonPath == abs` and computeStorePathForPath works relative to cwd, so the slight rewrite in the moved fetchToStore is legal. Change-Id: I05fd340c273f0bcc8ffabfebdc4a88b98083bce5
2024-03-05Merge pull request #9443 from ivan770/reproducibilityeldritch horrors
doc: fix machine-specific capabilities leaking (cherry picked from commit dda0e34ecf16bb1c736d585414122a7e3587db70) Change-Id: I3d07cc5039ee954b215a7a27caa3bf7359d92c26
2024-03-04Merge pull request #9992 from edolstra/fix-warningeldritch horrors
Fix "may be used uninitialized" warning (cherry picked from commit c4ebb82da4eade975e874da600dc50e9dec610cb) Change-Id: I0ce99bf102ad12902f7055c29a56e665b8320ca8
2024-03-04Merge pull request #9974 from edolstra/less-chatty-post-build-hookeldritch horrors
runPostBuildHook(): Be less chatty (cherry picked from commit 28a988bdde7b20abff9000162015500c44a5812b) Change-Id: Ibb49b71bf631d04d08970d719f091221b15c5036
2024-03-04Merge pull request #9924 from 9999years/rename-yellowtxteldritch horrors
Rename `yellowtxt` -> `magentatxt` (cherry picked from commit fb78a99e04206e7b1df84a362bb87d3300b41855) Change-Id: I9ade553d9f499e6713aeff3463c9a653a880a051
2024-03-04Merge pull request #9848 from obsidiansystems/default-system-features-staticeldritch horrors
Make `StoreConfig::getDefaultSystemFeatures` a static method (cherry picked from commit 5a9513cdbae31ea5e6f6e7afa7b3c2e3a9a26474) Change-Id: Ia9c0ae2b7de419bd60aea8bf905154b96c428276
2024-03-04Merge pull request #9841 from obsidiansystems/float-speed-factoreldritch horrors
Convert `Machine::speedFactor` from a non-neg int to a non-neg float (cherry picked from commit 69d0ae27e376e7c7c4f237716b0149223b8a805a) Change-Id: I2afb5cf9e4fe1384985c58353946135c3d102b42
2024-03-04Merge pull request #9838 from obsidiansystems/systemTypes-seteldritch horrors
Make `Machine::systemTypes` a set not vector (cherry picked from commit f1b030415376e81c5804647c055d71eaba4aa725) Change-Id: I6d4f5c0bfc226e9bd66c58c360cd99e3fac9a129
2024-03-04Merge pull request #9804 from edolstra/missing-nar-crasheldritch horrors
Fix crash when NAR is missing from binary cache (cherry picked from commit 3b20cca9625a1701a10a883735e7315185629563) Change-Id: I50ff18f4a6de69c323473b4a8e3e098d1f365145
2024-03-04Merge pull request #9798 from edolstra/remote-store-eofeldritch horrors
Print a more helpful message if the daemon crashes (cherry picked from commit 32706b14a7531c2c21b9f96da083a540a0031ec4) Change-Id: Ief7c465bca7666e2b7e7c9d1dd0c01c5f9014146
2024-03-04Merge pull request #9797 from edolstra/fix-error-messageeldritch horrors
Store: :buildPaths(): Fix display of store paths (cherry picked from commit b5ed36e6633cac844fe4388dcc0cc8055a18ef9e) Change-Id: Ic6008491088dc6febd4a1e44dc2dbb96c47661f4
2024-03-04Merge pull request #9763 from L-as/avoid-unnecessary-copyeldritch horrors
Avoid unnecessary copy of goal log (cherry picked from commit 268c49264ae2657c602f84f9bff3eb31306253ea) Change-Id: I5d9de9e43a7ddcb09d25bbc7367c9d83f58c7a75
2024-03-04Merge pull request #9662 from shlevy/flat-fixed-references-asserteldritch horrors
Improve error message for fixed-outputs with references. (cherry picked from commit ff6de4a9ee6c3862db9ee5f09ff9c3f43ae7a088) Change-Id: I733c49760b9a3f1b76a6bece3b250b8579cd6cac