aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/store-api.cc
AgeCommit message (Collapse)Author
2024-06-20Merge "libstore: fix queryValidPaths concurrency" into mainjade
2024-06-19libutil: return a source from readFileeldritch horrors
don't consume a sink, return a source instead. the only reason to not do this is a very slight reduction in dynamic allocations, but since we are going to *at least* do disk io that will not be a lot of overhead anyway Change-Id: Iae2f879ec64c3c3ac1d5310eeb6a85e696d4614a
2024-06-18libstore: fix queryValidPaths concurrencyJade Lovelace
The lock usage was obviously wrong so it was entirely serialized. This has the predicted speedups, the only question is whether it is sound because it's exposing a bunch of new code to actual concurrency. I did audit all the stores' queryPathInfoUncached implementations and they all look *intended* to be thread safe, but whether that is actually sound or not: lol lmao. I am highly confident in the s3 one because it is calling s3 sdk methods that are thread safe and has no actual state. Others are using Pool and look to be *supposed* to be thread safe, but unsure if they actually are. Change-Id: I0369152a510e878b5ac56c9ac956a98d48cd5fef
2024-06-18store-api: fix/clarify capture lifetimes in copyPathsJade Lovelace
This seems to fix a use of stack after return. Change-Id: If690a6defb9a3225684685132cf78b227e271447
2024-06-16libstore: refuse to serialise ancient protocolsJade Lovelace
We don't want to deal with these at all, let's stop doing so. (marking this one as the fix commit since its immediate predecessors aren't the complete fix) Fixes: https://git.lix.systems/lix-project/lix/issues/325 Change-Id: Ieea1b0b8ac0f903d1e24e5b3e63cfe12eeec119d
2024-06-16Harmonise the Store::queryPathInfoUncached interfacejulia
This: - Consistently returns `nullptr` for a non-existent store path, instead of a mix of `nullptr` and throwing exceptions. - If a store returns "bad" store paths in response to a request (e.g. incorrect hash or name), don't cache this result. This removes some duplication of code at the cache-access layer of queryPathInfo() checking this, and ­allows us to provide more specific errors. Part of #270. Change-Id: I86612c6499b1a37ab872c712c2304d6a3ff19edb
2024-06-01chore: rebrand Nix to Lix when it makes senseRaito Bezarius
Here's my guide so far: $ rg '((?!(recursive).*) Nix (?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))' -g '!doc/' --pcre2 All items from this query have been tackled. For the documentation side: that's for https://git.lix.systems/lix-project/lix/issues/162. Additionally, all remaining references to github.com/NixOS/nix which were not relevant were also replaced. Fixes: https://git.lix.systems/lix-project/lix/issues/148. Fixes: https://git.lix.systems/lix-project/lix/issues/162. Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-29util.hh: Delete remaining file and clean up headersTom Hubrecht
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29util.{hh,cc}: Split out users.{hh,cc}Tom Hubrecht
Change-Id: I1bd92479a2cb7e5c2c2e1541b80474adb05ea0df
2024-05-24libstore: parse the buildMode instead of unchecked castJade Lovelace
Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
2024-05-10libutil: remove callback.hheldritch horrors
it's no longer used. it really shouldn't have existed this long since it was just a mashup of both std::promise and std::packaged_task in a shape that makes composition unnecessarily difficult. all but a single case of Callback pattern calls were fully synchronous anyway, and even this sole outlier was by far not important enough to justify the extra complexity. Change-Id: I208aec4572bf2501cdbd0f331f27d505fca3a62f
2024-05-09libstore: de-callback-ify Store::queryRealisationeldritch horrors
Change-Id: I8d74745c519518f163f51dfaa39063836f17599e
2024-05-09libstore: un-callback-ify Store::queryRealisationUncachedeldritch horrors
Change-Id: I4a328f46eaac3bb8b19ddc091306de83348be9cf
2024-05-09libstore: de-callback-ify Store::queryPathInfoUncachedeldritch horrors
Change-Id: I23a156aaff5328f67ca16ccd85c0ea1711b21e35
2024-05-06fix fallback chroot store creation after b247ef72dQyriad
When /nix/var (or, more precisely, NIX_STATE_DIR) does not exist at all, Lix falls back to creating an adhoc chroot store in XDG_DATA_HOME. b247ef72d[1] changed the way Store classes are initialized, and in the migration, a `params2` was accidentally changed to `params`. This commit restores the correct behavior, and in lieu of a single *character* fix, this commit also changes the variable name to something more reasonable. Fixes #274. [1]: b247ef72dc7bcc857288c0ddcceb3e42f76a78f1 n.b., this code might deserve some more looking at anyway. this fallback store creation throws away *all* Store params passed to openFromNonUri() in favor of an entirely new set which only contains the `root` param, which may or may not be the correct behavior Change-Id: Ibea559b88a50e6d6e75a1f87d9d7816cabb2a8f3
2024-04-27Fix progress bar on copyPathsPuck Meerburg
This variable should not be shared between activities. Change-Id: I4eee89bc7acb320a3972dc3a55bfb087d3a9eb3a
2024-04-23libstore: Create platform LocalStore subclassesArtemis Tosini
This creates new subclasses of LocalStore for each OS to include platform-specific functionality. Currently this just includes garbage collector roots but it could be extended to sandboxing as well. In order to make sure that the generic LocalStore is not accidentally constructed, its constructor is protected. A Fallback is provided which implements no functionality except constructors. Change-Id: I836a28e90b68309873f75afb83e0f1b2e2c89fb3
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-04Merge pull request #9992 from edolstra/fix-warningeldritch horrors
Fix "may be used uninitialized" warning (cherry picked from commit c4ebb82da4eade975e874da600dc50e9dec610cb) Change-Id: I0ce99bf102ad12902f7055c29a56e665b8320ca8
2024-03-04Merge pull request #9804 from edolstra/missing-nar-crasheldritch horrors
Fix crash when NAR is missing from binary cache (cherry picked from commit 3b20cca9625a1701a10a883735e7315185629563) Change-Id: I50ff18f4a6de69c323473b4a8e3e098d1f365145
2024-03-04Merge pull request #9662 from shlevy/flat-fixed-references-asserteldritch horrors
Improve error message for fixed-outputs with references. (cherry picked from commit ff6de4a9ee6c3862db9ee5f09ff9c3f43ae7a088) Change-Id: I733c49760b9a3f1b76a6bece3b250b8579cd6cac
2024-03-04Merge pull request #9588 from obsidiansystems/queryDerivationOutputMap-evalStoreeldritch horrors
Give `Store::queryDerivationOutputMap` and `evalStore` argument (cherry picked from commit 8cddda4f892cb42be43e9bd87aa0111572617e78) Change-Id: I394e7e11c3f2e0cd3dbe0f48d757c14c09835e44
2024-03-04Merge pull request #9289 from edolstra/fix-warningseldritch horrors
Fix gcc warnings (cherry picked from commit 66cb364f581486e0c426b35149ac13d19f7842bc) Change-Id: I1474dbc18a4beaaf1bce16d4abbcc99806b79ff1
2024-03-04Merge pull request #6223 from obsidiansystems/worker-proto-with-versioneldritch horrors
Give `nix daemon` and `nix-store --serve` protocols separate serializers with version info (cherry picked from commit 8b68bbb77745fda0d14939b6c23d31cc89da41ce) Change-Id: Ia3d3b9fbaf9f0ae62ab225020b7d14790e793655
2023-08-02Add infra for experimental store implemenationsJohn Ericson
This is analogous to that for experimental settings and flags that we have also added as of late.
2023-07-21Merge pull request #8724 from ↵John Ericson
obsidiansystems/queryPartialDerivationOutputMap-evalStore Give `queryPartialDerivationOutputMap` an `evalStore` parameter
2023-07-20Give `queryPartialDerivationOutputMap` an `evalStore` parameterJohn Ericson
This makes it more useful. In general, the derivation will be in one store, and the realisation info is in another. This also helps us avoid duplication. See how `resolveDerivedPath` is now simpler because it uses `queryPartialDerivationOutputMap`. In #8369 we get more flavors of derived path, and need more code to resolve them all, and this problem only gets worse. The fact that we need a new method to deal with the multiple dispatch is unfortunate, but this generally relates to the fact that `Store` is a sub-par interface, too bulky/unwieldy and conflating separate concerns. Solving that is out of scope of this PR. This is part of the RFC 92 work. See tracking issue #6316
2023-07-07Simplify `ContentAddress`John Ericson
Whereas `ContentAddressWithReferences` is a sum type complex because different varieties support different notions of reference, and `ContentAddressMethod` is a nested enum to support that, `ContentAddress` can be a simple pair of a method and hash. `ContentAddress` does not need to be a sum type on the outside because the choice of method doesn't effect what type of hashes we can use. Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-04-07Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-04-02Move `querySubstitutablePathInfos` from `LocalStore` to `Store`John Ericson
The code is not local-store-specific, so we should share it with all stores. More uniform behavior is better, and a less store-specific functionality is more maintainable. This fixes a FIXME added in f73d911628 by @edolstra himself.
2023-03-31Replace unnecessary Sync<uint64_t> with std::atomic<uint64_t>Patrick Jackson
2023-03-30Only lock oncePatrick Jackson
2023-03-30Fix data race in copyPathsPatrick Jackson
2023-03-30Use "raw pattern" for content address typesJohn Ericson
We weren't because this ancient PR predated it! This is actually a new version of the pattern which addresses some issues identified in #7479.
2023-03-30Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
Also improve content-address.hh API docs.
2023-03-20Move enabled experimental feature to libutil structJohn Ericson
This is needed in subsequent commits to allow the settings and CLI args infrastructure itself to read this setting.
2023-03-10Merge pull request #8015 from tweag/progress-during-nix-copyThéophane Hufschmitt
Display progress when running copyPaths (nix copy)
2023-03-09Display progress when running copyPaths (nix copy)Alexander Bantyev
`nix copy` operations did not show progress. This is quite confusing. Add a `progressSink` which displays the progress during `copyPaths`, pretty much copied from `copyStorePath`. Fixes https://github.com/NixOS/nix/issues/8000
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2023-02-28Always set valid in path-info --json outputMatthew Kenigsberg
Currently the valid key is only present when the path is invalid, which makes checking path validity more complex than it should be. With this change, the valid key can always be used to check if a path is valid
2023-02-28No inheritance for `TextInfo` and `FixedOutputInfo`John Ericson
2023-02-28Revert "Remove some designated initializers"John Ericson
This reverts commit ee9eb83a842eb97d0180fd9d349d30ff27fdb485.
2023-02-28Clarify store path grammar and improve comment on `makeType`John Ericson
2023-02-28Fix typo in the method nameJohn Ericson
2023-02-02Merge branch 'master' into path-infoJohn Ericson
2023-02-01Remove an unused captureEelco Dolstra
2023-02-01Remove some designated initializersJohn Ericson
With the switch to C++20, the rules became more strict, and we can no longer initialize base classes. Make them comments instead. (BTW https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html this offers some new syntax for this use-case. Hopefully this will be adopted and we can eventually use it.)
2023-01-30Avoid some `StorePath` <-> `Path` round tripsJohn Ericson
Avoid needless work and throwing away invariants. These conversions date back to when `StorePath` was in Rust and there were issues with it missing utility methods.
2023-01-23Don't add `StorePathDescriptor` for nowJohn Ericson
We don't need it yet, we can add it back later.
2023-01-14Make `ValidPathInfo` have plain `StorePathSet` references like beforeJohn Ericson
This change can wait for another PR.