aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2024-03-04Merge pull request #9687 from edolstra/withFramedSink-ctrl-c-hangeldritch horrors
withFramedSink(): Receive interrupts on the stderr thread (cherry picked from commit 965cfe96886c988c3aa94bfc7fefdd37325f4536) Change-Id: I8320a96957c01ec0e3450d1b3ae38a3baff78d49
2024-03-04Merge pull request #9673 from pennae/drv-parse-optseldritch horrors
optimize derivation parsing (cherry picked from commit 3511430902941f0f26dc71313a54bb5096f57305) Change-Id: I00f76dcd464a5811944613731501af504b6e8c29
2024-03-04Merge pull request #9666 from unblevable/dervation-typoeldritch horrors
Fix "dervation" typo (cherry picked from commit a21c762dab365049b77af95355ee4236d173e216) Change-Id: Ib6c0521758eb23115cfa243b9f3a34bb6e249e5f
2024-03-04Merge pull request #9582 from pennae/misc-optseldritch horrors
a packet of small optimizations (cherry picked from commit ee439734e924eb337a869ff2e48aff8b989198bc) Change-Id: I125d870710750a32a0dece48f39a3e9132b0d023
2024-03-04Merge pull request #9641 from amarshall/darwin-sandbox-stderreldritch horrors
Allow access to /dev/stderr in Darwin sandbox (cherry picked from commit c6d7013583c568590aff285fb7414d1675a745f4) Change-Id: I5657f6f4ee9dad8c978bad0d71f5cac51584e4f2
2024-03-04Merge pull request #4093 from matthewbauer/eval-systemeldritch horrors
Add eval-system option (cherry picked from commit 071dbbee33af9f27338c3e53e4ea067dbfa14010) Change-Id: Ia81358c8cfb60241da07a4d0e84b9ee62a18a53f
2024-03-04Merge pull request #9600 from SharzyL/fix_nix_copyeldritch horrors
fix: nix copy ssh-ng:// not respecting --substitute-on-destination (cherry picked from commit b1c633c6bb2e2dbc8a2891edf0d0c6f9d31d890b) Change-Id: I77356d14a9011d6dc4cf64776995f7590d918874
2024-03-04Merge pull request #9589 from ↵eldritch horrors
obsidiansystems/floating-content-addressing-derivations-eval-store Fix building CA derivations with and eval store (cherry picked from commit dfc0cee7024a082d90a4f68296f55a82dfd52126) Change-Id: I28feb5a36d4fe75f0ed3e3e2db6eb56b67d0f371
2024-03-04Merge pull request #9588 from obsidiansystems/queryDerivationOutputMap-evalStoreeldritch horrors
Give `Store::queryDerivationOutputMap` and `evalStore` argument (cherry picked from commit 8cddda4f892cb42be43e9bd87aa0111572617e78) Change-Id: I394e7e11c3f2e0cd3dbe0f48d757c14c09835e44
2024-03-04Merge pull request #9587 from amjoseph-nixpkgs/pr/wopBuildDerivation/explaineldritch horrors
libstore/daemon.cc: note trust model difference in readDerivation()s (cherry picked from commit 5c917c32048ef185ea0eec352c3505485aa3212c) Change-Id: I9945bc84e9529b005eafdc5c08b5bf1553335340
2024-03-04Merge pull request #9563 from obsidiansystems/tryResolve-evalStoreeldritch horrors
Give `Derivation::tryResolve` an `evalStore` argument (cherry picked from commit 36ca6adc60511dc822870f2df43c0a578e481925) Change-Id: If76b185a01ffa982e4c49cf333a9b5fbf9edebfe
2024-03-04Merge pull request #9233 from bouk/bouk/apply-config-innereldritch horrors
config: add included files into parsedContents before applying (cherry picked from commit 82359eba6b692691ef08a71196ef25a61bc4d3d3) Change-Id: Idde3177010fec7b8bafe6088c3c23d5caf491845
2024-03-04Merge pull request #9465 from obsidiansystems/build-direldritch horrors
Use `buildprefix` in a few more places (cherry picked from commit b6a3fde6b7a416929553e6be36fc991680ddf9ef) Change-Id: I2790663fa9f8242ac2db6582b7e421d2fdf42942
2024-03-04Merge pull request #9400 from hercules-ci/refactors-from-5e3986f59cbeldritch horrors
Refactors from 5e3986f59cb (cherry picked from commit e540d48c4fb460e5e577d8b8b33e8eca9959c49b) Change-Id: I5b21b770a0c20ec2ec9845d3a97a524f1b0135ee
2024-03-04Merge pull request #6469 from gbpdt/fix/skipped_build_lockingeldritch horrors
Unlock output paths when a derivation is already built (cherry picked from commit 7ba4e073e8622ca86b52e03d68476e80250ab62f) Change-Id: I9de077679290d5141a610ac43d99d3a43acff87c
2024-03-04Merge pull request #7348 from thufschmitt/dont-use-vlaseldritch horrors
Remove the usage of VLAs in the code (cherry picked from commit ac4431e9d016e62fb5dc9ae36833bd0c6cdadeec) Change-Id: Ifbf5fbfc2e27122362a2aaea4b62c7cf3ca46b1a
2024-03-04Merge pull request #9293 from ThinkChaos/ssh-misc-improvmentseldritch horrors
SSH small improvments (cherry picked from commit dcc49744ed0c3b91f905b0f3304d3ca700f9f717) Change-Id: Ie87910df3ab739d0ff983c085190da1d108e96e2
2024-03-04Merge pull request #9289 from edolstra/fix-warningseldritch horrors
Fix gcc warnings (cherry picked from commit 66cb364f581486e0c426b35149ac13d19f7842bc) Change-Id: I1474dbc18a4beaaf1bce16d4abbcc99806b79ff1
2024-03-04Merge pull request #9271 from keszybz/if-fixeldritch horrors
fix: gcc complains about if which doesn't guard the indented statement (cherry picked from commit 8d663462938a333a4e81cce1005437f141cd11fa) Change-Id: Ifa2e65502de4000935549dde82ab1b5867e2f0ed
2024-03-04Merge pull request #6258 from obsidiansystems/gcc-bug-ergonomicseldritch horrors
Remove bug-avoiding `StoreConfig *` casts for settings (cherry picked from commit e3febfcd532adb23ca05ac465a2b907d6f1a3529) Change-Id: Ifeae276582fdbc781a38581df9de3da67a7e7bf9
2024-03-04Merge pull request #9246 from trofi/clarify-builder-uid-exhaustioneldritch horrors
local-derivation-goal.cc: slightly clarify waiting message (cherry picked from commit 232e23667226a3cfff9de1828f819844c27e7539) Change-Id: Idd271d6921caba203db9971f27e156e533664935
2024-03-04Merge pull request #9238 from tfc/small-improvements2eldritch horrors
Small improvements 2 (cherry picked from commit 1dc6a65d361456d0e082065eb189d9b67d93aacd) Change-Id: I8ed573c496068c4a92f53c1caafa49da59d75967
2024-03-04Merge pull request #8965 from Artturin/bindfilesinchrooteldritch horrors
Bindmount files instead of hardlinking or copying to chroot (cherry picked from commit 622191c2b53882a1675fed5066ff8090b4f01827) Change-Id: I278ec1baacdfa9044992b58fdec8f14d6d7d09ce
2024-03-04Merge pull request #9229 from tfc/small-improvementseldritch horrors
Remove warnings, small improvements (cherry picked from commit 5ac87a75dd65c19c50976559a6f7810c315cd9d5) Change-Id: I88349b6e954398dde83c845f42d41a9dd89ba9e0
2024-03-04Merge pull request #6223 from obsidiansystems/worker-proto-with-versioneldritch horrors
Give `nix daemon` and `nix-store --serve` protocols separate serializers with version info (cherry picked from commit 8b68bbb77745fda0d14939b6c23d31cc89da41ce) Change-Id: Ia3d3b9fbaf9f0ae62ab225020b7d14790e793655
2024-03-04Merge pull request #9157 from obsidiansystems/protocol-versionsRobert Hensing
Add protocol versions to `{Worker,Serve}Proto::*Conn` (cherry picked from commit 4d17c59d8d059a5b39f1d1da2b58f2ec8da44861) Change-Id: I497af39deb792e50c157a1305d8c9e722798740b
2024-03-04Merge pull request #9172 from tfc/bad-moveseldritch horrors
Fix/remove some bad std::moves (cherry picked from commit 8c049a9f044569ebda70231709f6f15d3073894a) Change-Id: I720273378d2506a13883acee28abd096d099b0d4
2024-03-04Merge pull request #9137 from obsidiansystems/serve-protocoleldritch horrors
Introduce separate Serve protocol serialisers (cherry picked from commit d070d8b7460f412a657745698dba291c66792402) Change-Id: Ibcc8639e8997bcd07f7a5318330a147bcadc411b
2024-03-04Merge pull request #8047 from lovesegfault/always-allow-substituteseldritch horrors
feat: add always-allow-substitutes (cherry picked from commit da2b59a08878b3c6c7074595e3b6d26b6928b4c1) Change-Id: I50481cd8fe643c673c610fec28bad84519a4d650
2024-03-04Merge pull request #9099 from obsidiansystems/common-protoeldritch horrors
Factor out bits of the worker protocol to use elsewhere (cherry picked from commit 4b1a97338f517f45e6169d3d8845c5caa5724e97) Change-Id: If93afa0f8b1cf9b0e705b34fa71e6fd708752758
2024-03-04Merge pull request #9094 from obsidiansystems/test-protoeldritch horrors
Test the rest of the worker protocol serializers (cherry picked from commit 2f1c16dfa2378fd8616bff1b9b7cd0b4d42af69b) Change-Id: Idfd72d32b21d14a260e02f65531d287cef7464d2
2024-03-04Merge pull request #8923 from obsidiansystems/test-protoeldritch horrors
Unit test some worker protocol serializers (cherry picked from commit c6faef61a6f31c71146aee5d88168e861df9a22a) Change-Id: I99e36f5f17eb7642211a4e42a16b143424f164b4
2024-03-04Merge pull request #9041 from trofi/profiles-signeldritch horrors
src/libstore/profiles.cc: fix comparison of sign difference (cherry picked from commit 1da1642527f73089947b7bd2de46147a44700ac8) Change-Id: I8f23d32ad66b797ff58eb2bd15b2f7741f47cd99
2023-12-01Move tests to separate directories, and documentJohn Ericson
Today, with the tests inside a `tests` intermingled with the corresponding library's source code, we have a few problems: - We have to be careful that wildcards don't end up with tests being built as part of Nix proper, or test headers being installed as part of Nix proper. - Tests in libraries but not executables is not right: - It means each executable runs the previous unit tests again, because it needs the libraries. - It doesn't work right on Windows, which doesn't want you to load a DLL just for the side global variable . It could be made to work with the dlopen equivalent, but that's gross! This reorg solves these problems. There is a remaining problem which is that sibbling headers (like `hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end up shadowing each other. This PR doesn't solve that. That is left as future work for a future PR. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> (cherry picked from commit 91b6833686a6a6d9eac7f3f66393ec89ef1d3b57) (cherry picked from commit a61e42adb528b3d40ce43e07c79368d779a8b624)
2023-12-01Put functional tests in `tests/functional`John Ericson
I think it is bad for these reasons when `tests/` contains a mix of functional and integration tests - Concepts is harder to understand, the documentation makes a good unit vs functional vs integration distinction, but when the integration tests are just two subdirs within `tests/` this is not clear. - Source filtering in the `flake.nix` is more complex. We need to filter out some of the dirs from `tests/`, rather than simply pick the dirs we want and take all of them. This is a good sign the structure of what we are trying to do is not matching the structure of the files. With this change we have a clean: ```shell-session $ git show 'HEAD:tests' tree HEAD:tests functional/ installer/ nixos/ ``` (cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
2023-11-16libstore: Add apple-virt to system features when availableRobert Hensing
I'm sure that we'll adjust the implementation over time, but this at least discerns between an apple silicon bare metal machine and a tart VM. (cherry picked from commit 9277eb276bf0a942e88fcf499f6a6b9c262be853)
2023-10-08StorePath: reject names starting with '.'edef
This has been the behaviour before Nix 2.4. It was dropped in a rewrite in 759947bf72c134592f0ce23d385e48095bd0a301, allowing the creation of store paths that aren't considered valid by older Nix versions or other Nix tooling. Nix 2.4 didn't ship in NixOS until 22.05, and stdenv.mkDerivation in nixpkgs drops leading periods since April 2022, so it's unlikely anyone is relying on the current lax behaviour. Closes #9091. Change-Id: I4a57bd9899e1b0dba56870ae5a1b680918a18ce9 (cherry picked from commit 24bda0c7b381e1a017023c6f7cb9661fae8560bd)
2023-10-02Revert "Adapt scheduler to work with dynamic derivations"John Ericson
This reverts commit 5e3986f59cb58f48186a49dcec7aa317b4787522. This un-implements RFC 92 but fixes the critical bug #9052 which many people are hitting. This is a decent stop-gap until a minimal reproduction of that bug is found and a proper fix can be made. Mostly fixed #9052, but I would like to leave that issue open until we have a regression test, so I can then properly fix the bug (unbreaking RFC 92) later. (cherry picked from commit 8440afbed756254784d9fea3eaab06649dffd390)
2023-09-19Disable rapidcheck tests in the coverage runEelco Dolstra
https://hydra.nixos.org/build/233688539
2023-09-07Special-case error message to add extra informationJohn Ericson
The Derivation parser and old ATerm unfortunately leaves few ways to get nice errors when an old version of Nix encounters a new version of the format. The most likely scenario for this to occur is with a new client making a derivation that the old daemon it is communicating with cannot understand. The extensions we just created for dynamic derivation deps will add a version field, solving the problem going forward, but there is still the issue of what to do about old versions of Nix up to now. The solution here is to carefully catch the bad error from the daemon that is likely to indicate this problem, and add some extra context to it. There is another "Ugly backwards compatibility hack" in `remote-store.cc` that also works by transforming an error. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-07Allow dynamic derivation deps in `inputDrvs`John Ericson
We use the same nested map representation we used for goals, again in order to save space. We might someday want to combine with `inputDrvs`, by doing `V = bool` instead of `V = std::set<OutputName>`, but we are not doing that yet for sake of a smaller diff. The ATerm format for Derivations also needs to be extended, in addition to the in-memory format. To accomodate this, we added a new basic versioning scheme, so old versions of Nix will get nice errors. (And going forward, if the ATerm format changes again the errors will be even better.) `parsedStrings`, an internal function used as part of parsing derivations in A-Term format, used to consume the final `]` but expect the initial `[` to already be consumed. This made for what looked like unbalanced brackets at callsites, which was confusing. Now it consumes both which is hopefully less confusing. As part of testing, we also created a unit test for the A-Term format for regular non-experimental derivations too. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com> Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Apply suggestions from code review Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-06Improve derivation parsingJohn Ericson
- Don't assert: Derivation ATerms are not necessarily produced by Nix, and parsers should always throw graceful errors - Improve error message from `static void except(..)`, shows both what we expected and what we actually got. The intention is that we backport it, and then hopefully a few people might get slightly better errors if they try out new experimental drv files (for RFC 92) with an old version of Nix.
2023-09-06Merge pull request #8927 from obsidiansystems/test-derivation-atermJohn Ericson
Test and begin documentation of the ATerm format for derivations
2023-09-06Fix globals.hh typoChristina Sørensen
2023-09-05Test and begin documentation of the ATerm format for derivationsJohn Ericson
Wanted to do this before the last dynamic derivations PR when I introduce a variation, to make sure I wasn't changing the old version by mistake.
2023-09-01Fix warning 'catching polymorphic type by value'Eelco Dolstra
2023-08-25Adapt scheduler to work with dynamic derivationsJohn Ericson
To avoid dealing with an optional `drvPath` (because we might not know it yet) everywhere, make an `CreateDerivationAndRealiseGoal`. This goal just builds/substitutes the derivation file, and then kicks of a build for that obtained derivation; in other words it does the chaining of goals when the drv file is missing (as can already be the case) or computed (new case). This also means the `getDerivation` state can be removed from `DerivationGoal`, which makes the `BasicDerivation` / in memory case and `Derivation` / drv file file case closer together. The map type is factored out for clarity, and because we will soon hvae a second use for it (`Derivation` itself). Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-25Use `Worker::makeDerivationGoal` lessJohn Ericson
We're about to split up `DerivationGoal` a bit. At that point `makeDerivationGoal` will mean something more specific than it does today. (Perhaps a future rename will make this clearer.) On the other hand, the more public `Worker::makeGoal` function will continue to work exactly as before. So by moving some call sites to use that instead, we preemptively avoid issues in the next step.
2023-08-25Throw `MissingRealisation` not plain `Error` in both `resolveDerivedPath`John Ericson
Now we are consistent with the other `resolveDerivedPath`, and other such functions.
2023-08-25Introduce `OutputName` and `OutputNameView` type aliasesJohn Ericson
Hopefully they make the code easier to understand!
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-22 17:47:59 +0000