aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/remote-store.cc
AgeCommit message (Collapse)Author
2020-12-02read(): Use char * instead of unsigned char *Eelco Dolstra
This gets rid of some pointless casts.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-11-17Merge pull request #4180 from Ma27/ssh-ng-substituteEelco Dolstra
Allow substituting paths when building remotely using `ssh-ng://`
2020-11-16Fix deadlock in nix-store when max-connections=1Jake Waksbaum
This fixes a bug I encountered where `nix-store -qR` will deadlock when the `--include-outputs` flag is passed and `max-connections=1`. The deadlock occurs because `RemoteStore::queryDerivationOutputs` takes the only connection from the connection pool and uses it to check the daemon version. If the version is new enough, it calls `Store::queryDerivationOutputs`, which eventually calls `RemoteStore::queryPartialDerivationOutputMap`, where we take another connection from the connection pool to check the version again. Because we still haven't released the connection from the caller, this waits for a connection to be available, causing a deadlock. This diff solves the issue by using `getProtocol` to check the protocol version in the caller `RemoteStore::queryDerivationOutputs`, which immediately frees the connection back to the pool before returning the protocol version. That way we've already freed the connection by the time we call `RemoteStore::queryPartialDerivationOutputMap`.
2020-11-05Allow substituting paths when building remotely using `ssh-ng://`Maximilian Bosch
Until now, it was not possible to substitute missing paths from e.g. `https://cache.nixos.org` on a remote server when building on it using the new `ssh-ng` protocol. This is because every store implementation except legacy `ssh://` ignores the substitution flag passed to `Store::queryValidPaths` while the `legacy-ssh-store` substitutes the remote store using `cmdQueryValidPaths` when the remote store is opened with `nix-store --serve`. This patch slightly modifies the daemon protocol to allow passing an integer value suggesting whether to substitute missing paths during `wopQueryValidPaths`. To implement this on the daemon-side, the substitution logic from `nix-store --serve` has been moved into a protected method named `Store::substitutePaths` which gets currently called from `LocalStore::queryValidPaths` and `Store::queryValidPaths` if `maybeSubstitute` is `true`. Fixes #2770
2020-11-02Merge pull request #4207 from hercules-ci/fix-RemoteStore-filterSource-deadlockEelco Dolstra
Fix RemoteStore pool deadlock in filterSource etc
2020-11-02Don't send eval-related settings to the daemonEelco Dolstra
2020-10-30Fix RemoteStore pool deadlock in filterSource etcRobert Hensing
2020-10-11Split out uds-remote-store.{cc.hh}John Ericson
2020-10-07Serialize exceptions from the daemon to the clientEelco Dolstra
2020-10-06Remove static variable name clashesEelco Dolstra
This was useful for an experiment with building Nix as a single compilation unit. It's not very useful otherwise but also doesn't hurt...
2020-09-30nix::worker_proto -> worker_protoJohn Ericson
2020-09-30Revert "Use template structs instead of phantoms"John Ericson
This reverts commit 9ab07e99f527d1fa3adfa02839da477a1528d64b.
2020-09-22Merge remote-tracking branch 'upstream/master' into templated-daemon-protocolJohn Ericson
2020-09-22RemoteStore::addCAToStore(): Don't hold connection while calling queryPathInfo()Eelco Dolstra
This leads to a deadlock if we're at the connection limit.
2020-09-22Style fixesEelco Dolstra
2020-09-22Merge branch 'add-ca-to-store' of https://github.com/hercules-ci/nixEelco Dolstra
2020-09-22Merge remote-tracking branch 'upstream/master' into templated-daemon-protocolJohn Ericson
2020-09-21Move Callback into its own headerEelco Dolstra
This gets rid of the inclusion of <future> in util.hh, cutting compilation time by ~20s (CPU time). Issue #4045.
2020-09-21wopAddToStore: return ValidPathInfoRobert Hensing
A ValidPathInfo is created anyway. By returning it we can save a roundtrip and we have a nicer interface.
2020-09-21wopAddToStore: add RepairFlagRobert Hensing
2020-09-21Move FramedSink next to FramedSourceRobert Hensing
2020-09-21Refactor wopAddToStore to make wopAddTextToStore obsoleteRobert Hensing
2020-09-21Overhaul wopAddToStoreRobert Hensing
2020-09-17Move FramedSource and FramedSink, extract withFramedSinkRobert Hensing
2020-09-16Merge remote-tracking branch 'upstream/master' into ↵John Ericson
remove-storetype-delegate-regStore
2020-09-16Fix build issues with gccregnat
2020-09-16Document the new store hierarchyregnat
2020-09-16Properly filter the stores according to their declared uriSchemesregnat
When opening a store, only try the stores whose `uriSchemes()` include the current one
2020-09-16Correctly call all the parent contructors of the storesregnat
Using virtual inheritance means that only the default constructors of the parent classes will be called, which isn't what we want
2020-09-16Separate store configs from the implemsregnat
Rework the `Store` hierarchy so that there's now one hierarchy for the store configs and one for the implementations (where each implementation extends the corresponding config). So a class hierarchy like ``` StoreConfig-------->Store | | v v SubStoreConfig----->SubStore | | v v SubSubStoreConfig-->SubSubStore ``` (with virtual inheritance to prevent DDD). The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
2020-09-16Fix the registration of storesregnat
2020-09-16Make the store plugins more introspectableregnat
Directly register the store classes rather than a function to build an instance of them. This gives the possibility to introspect static members of the class or choose different ways of instantiating them.
2020-09-15Merge remote-tracking branch 'upstream/master' into ↵John Ericson
remove-storetype-delegate-regStore
2020-09-15gracefully handle old daemon versionsregnat
Add a fallback path in `queryPartialDerivationOutputMap` for daemons that don't support it. Also upstreams a couple methods from `SSHStore` to `RemoteStore` as this is needed to handle the fallback path.
2020-09-13Fix unspecified behaviour in readStorePathCAMapaszlig
When deploying a Hydra instance with current Nix master, most builds would not run because of errors like this: queue monitor: error: --- Error --- hydra-queue-runner error: --- UsageError --- nix-daemon not a content address because it is not in the form '<prefix>:<rest>': /nix/store/...-somedrv The last error message is from parseContentAddress, which expects a colon-separated string, however what we got here is a store path. Looking at the worker protocol, the following message sent to the Nix daemon caused the error above: 0x1E -> wopQuerySubstitutablePathInfos 0x01 -> Number of paths 0x16 -> Length of string "/nix/store/...-somedrv" 0x00 -> Length of string "" Looking at writeStorePathCAMap, the store path is indeed the first field that's transmitted. However, readStorePathCAMap expects it to be the *second* field *on my machine*, since expression evaluation order is a classic form of unspecified behaviour[1] in C++. This has been introduced in https://github.com/NixOS/nix/pull/3689, specifically in commit 66a62b3189c8c9b0965850e6b3c9b0fda0b50fd8. [1]: https://en.wikipedia.org/wiki/Unspecified_behavior#Order_of_evaluation_of_subexpressions Signed-off-by: aszlig <aszlig@nix.build>
2020-08-27RemoteStore::addToStore(): Fix race between stderrThread and NAR writerEelco Dolstra
As pointed out by @B4dM4n, the call to to.flush() on stderrThread is unsafe because the NAR writer thread is also writing to 'to'. Fixes #3943.
2020-08-21tabs -> spacesJohn Ericson
Sorry I let the tab sneak in there in the first place.
2020-08-20Merge remote-tracking branch 'upstream/master' into templated-daemon-protocolJohn Ericson
2020-08-20Merge pull request #3859 from obsidiansystems/drv-outputs-map-allow-missingEelco Dolstra
`queryDerivationOutputMap` no longer assumes all outputs have a mapping
2020-08-20Rename drv output querying functionsJohn Ericson
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap` - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
2020-08-19Use `RemoteStore` to open connection for proxying daemonJohn Ericson
Removes duplicate websocket opening code, and also means we should be able to to ssh-ssh-... daemon relays, not just uds-uds-... ones.
2020-08-19Merge branch 'master' of github.com:NixOS/nix into ↵John Ericson
remove-storetype-delegate-regStore
2020-08-19Merge branch 'master' of github.com:NixOS/nix into templated-daemon-protocolJohn Ericson
2020-08-07Merge branch 'drv-outputs-map-allow-missing' of ↵John Ericson
github.com:obsidiansystems/nix into templated-daemon-protocol
2020-08-07Specialize `std::optional<StorePath>` so this is backwards compatibleJohn Ericson
While I am cautious to break parametricity, I think it's OK in this cases---we're not about to try to do some crazy polymorphic protocol anytime soon.
2020-08-06Use template structs instead of phantomsCarlo Nucera
2020-08-06Merge branch 'drv-outputs-map-allow-missing-namespace' of ↵Carlo Nucera
github.com:obsidiansystems/nix into templated-daemon-protocol
2020-08-06Minimize the usage of `Hash::dummy`John Ericson
2020-08-05WIP: Put the worker protocol `read` and `write` in a namespace to disambigJohn Ericson
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 14:03:11 +0000