aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/store-api.hh
AgeCommit message (Collapse)Author
2017-02-22RemoteStore::addToStore(): Pass content-addressability assertionEelco Dolstra
... and use this in Downloader::downloadCached(). This fixes $ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
2017-02-07Merge branch 'nix-copy-closure-c++' of https://github.com/shlevy/nixEelco Dolstra
2017-02-07copyStorePath(): Don't require signatures for "trusted" storesEelco Dolstra
For example, SSH stores could be trusted.
2017-02-07Provide default implementations for a couple of Store methodsEelco Dolstra
2017-01-26exportReferencesGraph: Export more complete info in JSON formatEelco Dolstra
This writes info about every path in the closure in the same format as ‘nix path-info --json’. Thus it also includes NAR hashes and sizes. Example: [ { "path": "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10", "narHash": "sha256:0ckdc4z20kkmpqdilx0wl6cricxv90lh85xpv2qljppcmz6vzcxl", "narSize": 197648, "references": [ "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10", "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24" ], "closureSize": 20939776 }, { "path": "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24", "narHash": "sha256:1nfn3m3p98y1c0kd0brp80dn9n5mycwgrk183j17rajya0h7gax3", "narSize": 20742128, "references": [ "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24" ], "closureSize": 20742128 } ] Fixes #1134.
2017-01-20nix-copy-closure: Implement in C++.Shea Levy
Tests fail currently because the database is not given proper hashes in the VM
2017-01-19Merge pull request #981 from shlevy/build-remote-c++Eelco Dolstra
build-remote: Implement in C++
2016-12-07Keep track of the exact build start/stop timesEelco Dolstra
2016-12-07Add an option to make non-determinism non-fatalEelco Dolstra
That is, when build-repeat > 0, and the output of two rounds differ, then print a warning rather than fail the build. This is primarily to let Hydra check reproducibility of all packages.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-11-10Store::computeFSClosure(): Support a set of pathsEelco Dolstra
This way, callers can exploits the parallelism of computeFSClosure() when they have multiple paths that they need the (combined) closure of.
2016-11-10build-remote: Implement in C++Shea Levy
2016-11-09copyClosure() / copyStorePath(): Expose dontCheckSigsEelco Dolstra
Needed by Hydra.
2016-11-09Merge branch 'ssh-store' of https://github.com/shlevy/nixEelco Dolstra
2016-10-26Restore the CachedFailure status codeEelco Dolstra
The removal of CachedFailure caused the value of TimedOut to change, which broke timed-out handling in Hydra (so timed-out builds would show up as "aborted" and would be retried, e.g. at http://hydra.nixos.org/build/42537427).
2016-10-21Remove addPathToAccessorEelco Dolstra
2016-10-21BinaryCacheStore: Optionally write a NAR listingEelco Dolstra
The store parameter "write-nar-listing=1" will cause BinaryCacheStore to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’ added to the binary cache. This file contains an XZ-compressed JSON file describing the contents of the NAR, excluding the contents of regular files. E.g. { "version": 1, "root": { "type": "directory", "entries": { "lib": { "type": "directory", "entries": { "Mcrt1.o": { "type": "regular", "size": 1288 }, "Scrt1.o": { "type": "regular", "size": 3920 }, } } } ... } } (The actual file has no indentation.) This is intended to speed up the NixOS channels programs index generator [1], since fetching gazillions of large NARs from cache.nixos.org is currently a bottleneck for updating the regular (non-small) channel. [1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
2016-10-07Implement generic Store::queryValidPaths()Eelco Dolstra
2016-10-07Add copyClosure utility function for HydraEelco Dolstra
2016-10-07importPaths(): Fix accessor support for HydraEelco Dolstra
2016-09-16Make computeFSClosure() single-threaded againEelco Dolstra
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-02Factor out the unix domain socket-specific code from RemoteStoreShea Levy
2016-09-02Factor a function to get the store type from a URI out of the main ↵Shea Levy
RegisterStoreImplementation
2016-09-02Merge openStore and openStoreAt with default argumentsShea Levy
2016-08-10Mark content-addressed paths in the Nix database and in .narinfoEelco Dolstra
This allows such paths to be imported without signatures.
2016-08-10Add a "root" parameter to local storesEelco Dolstra
This makes it easier to create a diverted store, i.e. NIX_REMOTE="local?root=/tmp/root" instead of NIX_REMOTE="local?real=/tmp/root/nix/store&state=/tmp/root/nix/var/nix" NIX_LOG_DIR=/tmp/root/nix/var/log
2016-07-26makeFixedOutputPath(): Drop superfluous HashType argumentEelco Dolstra
2016-06-03Support sandbox builds by non-root usersEelco Dolstra
This allows an unprivileged user to perform builds on a diverted store (i.e. where the physical store location differs from the logical location). Example: $ NIX_LOG_DIR=/tmp/log NIX_REMOTE="local?real=/tmp/store&state=/tmp/var" nix-build -E \ 'with import <nixpkgs> {}; runCommand "foo" { buildInputs = [procps nettools]; } "id; ps; ifconfig; echo $out > $out"' will do a build in the Nix store physically in /tmp/store but logically in /nix/store (and thus using substituters for the latter).
2016-06-02Make derivationFromPath work on diverted storesEelco Dolstra
2016-06-02Allow setting the state directory as a store parameterEelco Dolstra
E.g. "local?store=/tmp/store&state=/tmp/var".
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-05-31nix-copy-closure / build-remote.pl: Disable signature checkingEelco Dolstra
This restores the Nix 1.11 behaviour.
2016-05-30Re-implement the WantMassQuery property of binary cachesEelco Dolstra
2016-05-30Fix repair during substitutionEelco Dolstra
2016-05-04Add a Store::addToStore() variant that accepts a NAREelco Dolstra
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04Remove OpenSSL-based signingEelco Dolstra
2016-05-04Add a copyStorePath() utility functionEelco Dolstra
2016-04-29BinaryCacheStore: Make the signing key a parameterEelco Dolstra
2016-04-29Allow parameters in store URIsEelco Dolstra
This is to allow store-specific configuration, e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
2016-04-29Eliminate the substituter mechanismEelco Dolstra
Substitution is now simply a Store -> Store copy operation, most typically from BinaryCacheStore to LocalStore.
2016-04-21Implement S3BinaryCacheStore::queryAllValidPaths()Eelco Dolstra
This allows commands like "nix verify --all" or "nix path-info --all" to work on S3 caches. Unfortunately, this requires some ugly hackery: when querying the contents of the bucket, we don't want to have to read every .narinfo file. But the S3 bucket keys only include the hash part of each store path, not the name part. So as a special exception queryAllValidPaths() can now return store paths *without* the name part, and queryPathInfo() accepts such store paths (returning a ValidPathInfo object containing the full name).
2016-04-20Cache path info lookups in SQLiteEelco Dolstra
This re-implements the binary cache database in C++, allowing it to be used by other Store backends, in particular the S3 backend.
2016-04-19Move path info caching from BinaryCacheStore to StoreEelco Dolstra
Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-15BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-04-08Remove failed build cachingEelco Dolstra
This feature was implemented for Hydra, but Hydra no longer uses it.
2016-04-07nix verify-paths: Add ‘--sigs-needed <N>’ flagEelco Dolstra
This specifies the number of distinct signatures required to consider each path "trusted". Also renamed ‘--no-sigs’ to ‘--no-trust’ for the flag that disables verifying whether a path is trusted (since a path can also be trusted if it has no signatures, but was built locally).
2016-04-05Add "nix copy-sigs" commandEelco Dolstra
This imports signatures from one store into another. E.g. $ nix copy-sigs -r /run/current-system -s https://cache.nixos.org/ imported 595 signatures
2016-03-30Improve the SQLite wrapper APIEelco Dolstra
In particular, this eliminates a bunch of boilerplate code.
2016-03-24Move signature support from NarInfo to ValidPathInfoEelco Dolstra