aboutsummaryrefslogtreecommitdiff
path: root/src/libutil
AgeCommit message (Collapse)Author
2016-09-16Make computeFSClosure() single-threaded againEelco Dolstra
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-16printMsg(): Don't check for interruptsEelco Dolstra
Having the logger function potentially throw exceptions is Heisenbuggy.
2016-09-14Add a toLower utility functionEelco Dolstra
2016-09-14Move some .drv parsing functions out of utilEelco Dolstra
2016-08-30Fix 32-bit buildEelco Dolstra
2016-08-29nix path-info: Add --json flagEelco Dolstra
Also, factor out JSON generation from value-to-json.{cc,hh}, and support producing indented JSON.
2016-07-21Fix assertion failureEelco Dolstra
2016-07-21NarInfo::NarInfo(): Ensure that we get a NAR size/hashEelco Dolstra
2016-07-21Fix assertion failure in ThreadPool::enqueue()Eelco Dolstra
2016-07-13FdSink: Restore move constructor/assignmentShea Levy
2016-07-13Make Buffered{Source,Sink} move-safeShea Levy
2016-07-11Modernize AutoCloseFDShea Levy
2016-06-09Use O_CLOEXEC in most placesEelco Dolstra
2016-06-02createTempDir(): Don't do a chown on LinuxEelco Dolstra
It's not needed and can cause problems in a user namespace.
2016-06-02Allow setting the state directory as a store parameterEelco Dolstra
E.g. "local?store=/tmp/store&state=/tmp/var".
2016-05-31use $(LIBLZMA_LIBS) instead of -llzmaDmitry Kalinkin
This is needed in case of non-standard lzma installation path that will be specified in pkgconfig manifest as extra -L option for LDFLAGS.
2016-05-04Cleanup: Remove singleton()Eelco Dolstra
2016-05-04Do compression in a sinkEelco Dolstra
2016-05-04Add a Store::addToStore() variant that accepts a NAREelco Dolstra
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04Fix segfault in xz compression/decompressionEelco Dolstra
2016-05-04Add a copyStorePath() utility functionEelco Dolstra
2016-04-29Support Git repos in the Nix pathEelco Dolstra
E.g. $ nix-build -I nixpkgs=git://github.com/NixOS/nixpkgs '<nixpkgs>' -A hello This is not extremely useful yet because you can't specify a branch/revision.
2016-04-29BinaryCacheStore: Support bzip2 compressionEelco Dolstra
2016-04-29BinaryCacheStore: Support "none" compression methodEelco Dolstra
2016-04-29Eliminate the substituter mechanismEelco Dolstra
Substitution is now simply a Store -> Store copy operation, most typically from BinaryCacheStore to LocalStore.
2016-04-25Improved logging abstractionEelco Dolstra
This also gets rid of --log-type, since the nested log type isn't useful in a multi-threaded situation, and nobody cares about the "pretty" log type.
2016-04-22Factor out parallel processing of work items that have dependenciesEelco Dolstra
2016-04-22ThreadPool: Start doing work as soon as work items are enqueuedEelco Dolstra
2016-04-22Make compression interruptibleEelco Dolstra
2016-04-21nix --help: Show short flagsEelco Dolstra
2016-04-20Cache path info lookups in SQLiteEelco Dolstra
This re-implements the binary cache database in C++, allowing it to be used by other Store backends, in particular the S3 backend.
2016-04-19Move path info caching from BinaryCacheStore to StoreEelco Dolstra
Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-15BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-04-08Make LocalStore thread-safeEelco Dolstra
Necessary for multi-threaded commands like "nix verify-paths".
2016-03-31Add missing -pthreadEelco Dolstra
https://hydra.nixos.org/build/33908385
2016-03-29Improve SIGINT handling in multi-threaded programsEelco Dolstra
The flag remembering whether an Interrupted exception was thrown is now thread-local. Thus, all threads will (eventually) throw Interrupted. Previously, one thread would throw Interrupted, and then the other threads wouldn't see that they were supposed to quit.
2016-03-29Add "nix verify-paths" commandEelco Dolstra
Unlike "nix-store --verify-path", this command verifies signatures in addition to store path contents, is multi-threaded (especially useful when verifying binary caches), and has a progress indicator. Example use: $ nix verify-paths --store https://cache.nixos.org -r $(type -p thunderbird) ... [17/132 checked] checking ‘/nix/store/rawakphadqrqxr6zri2rmnxh03gqkrl3-autogen-5.18.6’
2016-03-15Fix Darwin buildEelco Dolstra
http://hydra.nixos.org/build/33279996
2016-03-04Eliminate some large string copyingEelco Dolstra
2016-03-02Fix reading symlinksEelco Dolstra
The st_size field of a symlink doesn't have to be correct, e.g. for /dev/fd symlinks.
2016-03-02FdSource: track number of bytes readEelco Dolstra
2016-02-25Merge branch 'master' into new-cliEelco Dolstra
2016-02-25Add NAR / Store accessor abstractionEelco Dolstra
This is primary to allow hydra-queue-runner to extract files like "nix-support/hydra-build-products" from NARs in binary caches.
2016-02-25Fix short boolean flagsEelco Dolstra
2016-02-25Merge branch 'master' into new-cliEelco Dolstra
2016-02-24deletePath(): Succeed if path doesn't existEelco Dolstra
Also makes it robust against concurrent deletions.
2016-02-24BinaryCacheStore: Implement addToStore()Eelco Dolstra
So now you can do $ NIX_REMOTE=file:///tmp/binary-cache nix-instantiate '<nixpkgs>' -A hello and lots of other operations.
2016-02-24TypoEelco Dolstra
2016-02-24Move BinaryCacheStore / LocalBinaryCacheStore from HydraEelco Dolstra
So you can now do: $ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...
2016-02-24std::condition_variable_any -> std::condition_variableEelco Dolstra
The latter is supposed to be more efficient.