aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/remote-store.cc
AgeCommit message (Collapse)Author
2020-06-02WIPJohn Ericson
2020-05-28Merge branch 'master' of github.com:NixOS/nix into enum-classCarlo Nucera
2020-03-29Replace some `bool recursive` with a new `FileIngestionMethod` enumJohn Ericson
2020-03-29Use `enum struct` and drop prefixesJohn Ericson
This does a few enums; the rest will be gotten in subsequent commits.
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-10-29queryPathInfoUncached(): Return const ValidPathInfoEelco Dolstra
2019-10-11ssh-ng: Don't set CPU affinity on the remoteEelco Dolstra
Fixes #3138.
2019-10-04Revert "std::uncaught_exception() -> std::uncaught_exceptions()"Eelco Dolstra
This reverts commit 6b83174ffffbdfc3f876d94d5178e0b83f675cae because it doesn't work on macOS yet. https://hydra.nixos.org/build/102617587
2019-09-23Merge pull request #3104 from zimbatm/no-show-trace-forwardingEelco Dolstra
libstore: don't forward --show-trace
2019-09-22libstore: don't forward --show-tracezimbatm
2019-09-13std::uncaught_exception() -> std::uncaught_exceptions()Eelco Dolstra
The former is deprecated in C++17. Fixes a clang warning.
2019-09-03Add some noexceptsEelco Dolstra
This is to assert that callback functions should never throw (since the context in which they're called may not be able to handle the exception).
2019-08-28Don't send certain setting overrides to the daemonEelco Dolstra
These are already handled separately. This fixes warnings like warning: ignoring the user-specified setting 'max-jobs', because it is a restricted setting and you are not a trusted user when using the -j flag.
2019-03-14findRoots(): Add 'censor' parameterEelco Dolstra
This is less brittle than filtering paths after the fact in nix-daemon.
2019-03-10Make roots a map of store paths to pinning linksGuillaume Maudoux
This new structure makes more sense as there may be many sources rooting the same store path. Many profiles can reference the same path but this is even more true with /proc/<pid>/maps where distinct pids can and often do map the same store path. This implementation is also more efficient as the `Roots` map contains only one entry per rooted store path.
2018-10-27Merge branch 'nix-doctor' of https://github.com/LnL7/nixEelco Dolstra
2018-10-16RemoteStore: Close connection if an exception occursEelco Dolstra
Fixes #2075.
2018-09-02Store: expose the protocol version used by a storeDaiderd Jordan
2018-05-30RemoteStore::addToStore(): Ensure capacity is decreased on exceptionsEelco Dolstra
2018-05-30Fix deadlock in RemoteStore::queryPathInfoUncached()Eelco Dolstra
2018-05-30Simplify the callback mechanismEelco Dolstra
2018-05-30Modularize config settingsEelco Dolstra
Allow global config settings to be defined in multiple Config classes. For example, this means that libutil can have settings and evaluator settings can be moved out of libstore. The Config classes are registered in a new GlobalConfig class to which config files etc. are applied. Relevant to https://github.com/NixOS/nix/issues/2009 in that it removes the need for ad hoc handling of useCaseHack, which was the underlying cause of that issue.
2018-05-30Make 'nix copy --to daemon' run in constant memory (daemon side)Eelco Dolstra
Continuation of 97002b684c902dadcd351a67208f9c2a88ff8f8f. This makes the daemon use constant memory. For example, it reduces the daemon's maximum RSS on $ nix copy --from ~/my-nix --to daemon /nix/store/1n7x0yv8vq6zi90hfmian84vdhd04bgp-blender-2.79a from 264 MiB to 7 MiB. We now use a TunnelSource to prevent the connection from ending up in an undefined state if an exception is thrown while the NAR is being sent. Issue https://github.com/NixOS/nix/issues/1681.
2018-03-21Also make the backwards compatible case of RemoteStore::addToStore run in ↵Eelco Dolstra
constant memory Just because we can.
2018-03-21Make 'nix copy --to daemon' run in constant memoryEelco Dolstra
2018-01-26remote_store: register for NIX_REMOTE=unix://pathSpencer Baugh
This allows overriding the socket path so the daemon may be listening at an arbitrary Unix domain socket location. Fixes #1800
2017-10-24Remove the remote-builds optionEelco Dolstra
This is superfluous since you can now just set "builders" to empty, e.g. "--builders ''".
2017-09-14RemoteStore: Add option to drop old connections from the poolEelco Dolstra
This is a hack to make hydra-queue-runner free its temproots periodically, thereby ensuring that garbage collection of the corresponding paths is not blocked until the queue runner is restarted. It would be better if temproots could be released earlier than at process exit. I started working on a RAII object returned by functions like addToStore() that releases temproots. However, this would be a pretty massive change so I gave up on it for now.
2017-08-28Give activities a verbosity level againEelco Dolstra
And print them (separately from the progress bar) given sufficient -v flags.
2017-08-28Tunnel progress messages from the daemon to the clientEelco Dolstra
This makes the progress bar work for non-root users.
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-07-03Replace a few bool flags with enumsEelco Dolstra
Functions like copyClosure() had 3 bool arguments, which creates a severe risk of mixing up arguments. Also, implement copyClosure() using copyPaths().
2017-05-02build-remote: Fix fallback to other machines when connecting failsEelco Dolstra
Opening an SSHStore or LegacySSHStore does not actually establish a connection, so the try/catch block here did nothing. Added a Store::connect() method to test whether a connection can be established.
2017-04-26Add Store nesting to fix import-from-derivation within filterSourceShea Levy
2017-04-20Read per-user settings from ~/.config/nix/nix.confEelco Dolstra
2017-04-13Convert Settings to the new config systemEelco Dolstra
This makes all config options self-documenting. Unknown or unparseable config settings and --option flags now cause a warning.
2017-04-13Add a Config class to simplify adding configuration settingsEelco Dolstra
The typical use is to inherit Config and add Setting<T> members: class MyClass : private Config { Setting<int> foo{this, 123, "foo", "the number of foos to use"}; Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"}; MyClass() : Config(readConfigFile("/etc/my-app.conf")) { std::cout << foo << "\n"; // will print 123 unless overriden } }; Currently, this is used by Store and its subclasses for store parameters. You now get a warning if you specify a non-existant store parameter in a store URI.
2017-04-06Implement RemoteStore::queryMissing()Eelco Dolstra
This provides a significant speedup, e.g. 64 s -> 12 s for nix-build --dry-run -I nixpkgs=channel:nixos-16.03 '<nixpkgs/nixos/tests/misc.nix>' -A test on a cold local and CloudFront cache. The alternative is to use lots of concurrent daemon connections but that seems wasteful.
2017-03-16copyPaths(): Use queryValidPaths() to reduce SSH latencyEelco Dolstra
2017-03-03Cache connection failuresEelco Dolstra
2017-03-03Improve SSH handlingEelco Dolstra
* Unify SSH code in SSHStore and LegacySSHStore. * Fix a race starting the SSH master. We now wait synchronously for the SSH master to finish starting. This prevents the SSH clients from starting their own connections. * Don't use a master if max-connections == 1. * Add a "max-connections" store parameter. * Add a "compress" store parameter.
2017-03-01RemoteStore::addToStore(): Send NAR rather than string containing NAREelco Dolstra
This allows the NAR to be streamed in the future (though we're not doing that yet).
2017-03-01Handle importing NARs containing files greater than 4 GiBEelco Dolstra
Also templatize readInt() to work for various integer types.
2017-02-22RemoteStore::addToStore(): Pass content-addressability assertionEelco Dolstra
... and use this in Downloader::downloadCached(). This fixes $ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
2017-02-07Add a LegacySSHStore that uses nix-store --serveEelco Dolstra
This is useful for nix-copy-closure.
2017-01-16AutoDeleteArray -> std::unique_ptrEelco Dolstra
Also, switch to C++14 for std::make_unique.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-11-09Merge branch 'ssh-store' of https://github.com/shlevy/nixEelco Dolstra