aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2017-04-13Convert Settings to the new config systemEelco Dolstra
This makes all config options self-documenting. Unknown or unparseable config settings and --option flags now cause a warning.
2017-04-13Merge branch 'rework-options' of https://github.com/copumpkin/nixEelco Dolstra
2017-04-13Add a Config class to simplify adding configuration settingsEelco Dolstra
The typical use is to inherit Config and add Setting<T> members: class MyClass : private Config { Setting<int> foo{this, 123, "foo", "the number of foos to use"}; Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"}; MyClass() : Config(readConfigFile("/etc/my-app.conf")) { std::cout << foo << "\n"; // will print 123 unless overriden } }; Currently, this is used by Store and its subclasses for store parameters. You now get a warning if you specify a non-existant store parameter in a store URI.
2017-04-10Merge pull request #1316 from copumpkin/nix-retries-defaultEelco Dolstra
Default to 5 download retries
2017-04-10Add CURLE_WRITE_ERROR as a transient error conditionDan Peebles
We've observed it failing downloads in the wild and retrying the same URL a few moments later seemed to fix it.
2017-04-10Default to 5 download retriesDan Peebles
This should help certain downloaders that don't request anything special for the number of retries, like nix-channel.
2017-04-10Minor cleanupEelco Dolstra
Also, possible fix for #1310 on 32-bit systems.
2017-04-10Allow "auto" as a store URIEelco Dolstra
Using the empty string is likely to be ambiguous in some contexts.
2017-04-06Retry downloads on transient SSL errors tooDan Peebles
2017-04-06Implement RemoteStore::queryMissing()Eelco Dolstra
This provides a significant speedup, e.g. 64 s -> 12 s for nix-build --dry-run -I nixpkgs=channel:nixos-16.03 '<nixpkgs/nixos/tests/misc.nix>' -A test on a cold local and CloudFront cache. The alternative is to use lots of concurrent daemon connections but that seems wasteful.
2017-04-06Add a method to allow hydra-queue-runner to flush the path info cacheEelco Dolstra
2017-04-06nix-daemon: Disable path info cacheEelco Dolstra
This is useless because the client also caches path info, and can cause problems for long-running clients like hydra-queue-runner (i.e. it may return cached info about paths that have been garbage-collected).
2017-04-04Allow default sandbox paths to be overridenEelco Dolstra
E.g. you can now redirect /etc/resolv.conf to a different file.
2017-04-04Make /var/run/nscd/socket optionalEelco Dolstra
Not every distribution uses nscd.
2017-03-31Sandbox: Fix /dev/ptmx on recent kernelsEelco Dolstra
This fixes "No such file or directory" when opening /dev/ptmx (e.g. http://hydra.nixos.org/build/51094249). The reason appears to be some changes to /dev/ptmx / /dev/pts handling between Linux 4.4 and 4.9. See https://patchwork.kernel.org/patch/7832531/. The fix is to go back to mounting a proper /dev/pts instance inside the sandbox. Happily, this now works inside user namespaces, even for unprivileged users. So NIX_REMOTE=local?root=/tmp/nix nix-build \ '<nixpkgs/nixos/tests/misc.nix>' -A test works for non-root users. The downside is that the fix breaks sandbox builds on older kernels (probably pre-4.6), since mounting a devpts fails inside user namespaces for some reason I've never been able to figure out. Builds on those systems will fail with error: while setting up the build environment: mounting /dev/pts: Invalid argument Ah well.
2017-03-31Merge branch 'builtins.exec'Shea Levy
2017-03-31Retry curl error 16Eelco Dolstra
2017-03-30Add exec primop behind allow-unsafe-native-code-during-evaluation.Shea Levy
Execute a given program with the (optional) given arguments as the user running the evaluation, parsing stdout as an expression to be evaluated. There are many use cases for nix that would benefit from being able to run arbitrary code during evaluation, including but not limited to: * Automatic git fetching to get a sha256 from a git revision * git rev-parse HEAD * Automatic extraction of information from build specifications from other tools, particularly language-specific package managers like cabal or npm * Secrets decryption (e.g. with nixops) * Private repository fetching Ideally, we would add this functionality in a more principled way to nix, but in the mean time 'builtins.exec' can be used to get these tasks done. The primop is only available when the 'allow-unsafe-native-code-during-evaluation' nix option is true. That flag also enables the 'importNative' primop, which is strictly more powerful but less convenient (since it requires compiling a plugin against the running version of nix).
2017-03-28Ignore broken "Deriver: unknown-deriver" fields in .narinfoEelco Dolstra
These were generated by a legacy tool.
2017-03-21LocalFSStore::getBuildLog(): Handle corrupted logsEelco Dolstra
2017-03-21Only use cache.nixos.org when the store is /nix/storeEelco Dolstra
This is consistent with the behaviour of the old download-from-binary-cache substituter.
2017-03-21Require signatures by defaultEelco Dolstra
This corresponds to the NixOS default.
2017-03-21Restore cache.nixos.org as the default substituterEelco Dolstra
Fixes #1283.
2017-03-21Move istringstream_nocopy to a separate fileEelco Dolstra
2017-03-21Honor $NIX_SSHOPTS againEelco Dolstra
NixOps needs this.
2017-03-20Merge branch 'darwin-s3-binary-cache-store' of https://github.com/shlevy/nixEelco Dolstra
2017-03-16ssh:// -> ssh-ng://, legacy-ssh:// -> ssh://Eelco Dolstra
2017-03-16copyPaths(): Use queryValidPaths() to reduce SSH latencyEelco Dolstra
2017-03-16copyPaths(): Don't query path info for a path the target already hasEelco Dolstra
For example, this cuts "nix-copy-closure --from" on a NixOS system closure from 15.9s to 0.5s.
2017-03-16LegacySSHStore: Provide a faster implementation of computeFSClosure()Eelco Dolstra
This avoids the latency of the standard implementation, which can make a huge difference (e.g. 16.5s -> 0.5s on a NixOS system closure).
2017-03-16Fix nix-copy-closure --toEelco Dolstra
2017-03-16Remove "killing process <pid>" messagesEelco Dolstra
They convey no useful information.
2017-03-15More precise compression settingsEelco Dolstra
2017-03-15runProgram(): Distinguish between empty input and no inputEelco Dolstra
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2017-03-15Compress NAR listings using the "text-compression" methodEelco Dolstra
So if "text-compression=br", the .ls file in S3 will get a Content-Encoding of "br". Brotli appears to compress better than xz for this kind of file and is natively supported by browsers.
2017-03-15S3BinaryCacheStore: Set Content-TypeEelco Dolstra
This is necessary for serving log files to browsers.
2017-03-15S3BinaryCacheStore: Support compression of narinfo and log filesEelco Dolstra
You can now set the store parameter "text-compression=br" to compress textual files in the binary cache (i.e. narinfo and logs) using Brotli. This sets the Content-Encoding header; the extension of compressed files is unchanged. You can separately specify the compression of log files using "log-compression=br". This is useful when you don't want to compress narinfo files for backward compatibility.
2017-03-15Add support for brotli compressionEelco Dolstra
Build logs on cache.nixos.org are compressed using Brotli (since this allows them to be decompressed automatically by Chrome and Firefox), so it's handy if "nix log" can decompress them.
2017-03-15BinaryCacheStore: Implement getBuildLog()Eelco Dolstra
We assume that build logs are stored under log/<drv>, e.g. /nix/store/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv maps to https://cache.nixos.org/log/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv
2017-03-15Store: Add a method for getting build logsEelco Dolstra
This allows various Store implementations to provide different ways to get build logs. For example, BinaryCacheStore can get the build logs from the binary cache. Also, remove the log-servers option since we can use substituters for this.
2017-03-12Merge branch 'configurable-aws-region'Shea Levy
2017-03-08Add option to disable import-from-derivation completely, even if the drv is ↵Shea Levy
already realized
2017-03-06Properly set the caFile for aws-sdk-cpp s3Shea Levy
2017-03-06istringstream_nocopy: Implement in a standards-compliant way.Shea Levy
Fixes the problem mentioned in e6a61b8da788efbbbb0eb690c49434b6b5fc9741 See #1135
2017-03-03Add aws-region param to S3 store URLsShea Levy
2017-03-03Cache connection failuresEelco Dolstra
2017-03-03SSHMaster: Make thread-safeEelco Dolstra
2017-03-03build-remote: Don't use a SSH masterEelco Dolstra
This is unnecessary because we make only one connection.
2017-03-03Improve SSH handlingEelco Dolstra
* Unify SSH code in SSHStore and LegacySSHStore. * Fix a race starting the SSH master. We now wait synchronously for the SSH master to finish starting. This prevents the SSH clients from starting their own connections. * Don't use a master if max-connections == 1. * Add a "max-connections" store parameter. * Add a "compress" store parameter.
2017-03-03Fix fatal "broken pipe" error when $NIX_BUILD_HOOK is missingEelco Dolstra