aboutsummaryrefslogtreecommitdiff
path: root/src/libutil
AgeCommit message (Collapse)Author
2017-03-16Remove "killing process <pid>" messagesEelco Dolstra
They convey no useful information.
2017-03-15runProgram(): Distinguish between empty input and no inputEelco Dolstra
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2017-03-15S3BinaryCacheStore: Support compression of narinfo and log filesEelco Dolstra
You can now set the store parameter "text-compression=br" to compress textual files in the binary cache (i.e. narinfo and logs) using Brotli. This sets the Content-Encoding header; the extension of compressed files is unchanged. You can separately specify the compression of log files using "log-compression=br". This is useful when you don't want to compress narinfo files for backward compatibility.
2017-03-15Fix assertion failure on SIGINTEelco Dolstra
nix: src/libutil/compression.cc:142: virtual nix::XzSink::~XzSink(): Assertion `finished' failed.
2017-03-15Fix deadlock in runProgram() when input is larger than the pipe buffer sizeEelco Dolstra
2017-03-15Add support for brotli compressionEelco Dolstra
Build logs on cache.nixos.org are compressed using Brotli (since this allows them to be decompressed automatically by Chrome and Firefox), so it's handy if "nix log" can decompress them.
2017-03-08Add missing header <sstream>Adrien Devresse
2017-03-06Revert fa125b9b28bea25a4eeb4d39a71a481563127cb9Eelco Dolstra
This causes quadratic performance.
2017-03-03Pool: Don't hang if creating a connection failsEelco Dolstra
2017-03-03Improve SSH handlingEelco Dolstra
* Unify SSH code in SSHStore and LegacySSHStore. * Fix a race starting the SSH master. We now wait synchronously for the SSH master to finish starting. This prevents the SSH clients from starting their own connections. * Don't use a master if max-connections == 1. * Add a "max-connections" store parameter. * Add a "compress" store parameter.
2017-03-01TeeSink: Pre-reserve string spaceEelco Dolstra
When receiving a very large file, this can prevent the string from having tobe copied, which temporarily doubles memory consumption.
2017-03-01RemoteStore::addToStore(): Send NAR rather than string containing NAREelco Dolstra
This allows the NAR to be streamed in the future (though we're not doing that yet).
2017-03-01readString(): Read directly into std::stringEelco Dolstra
When reading a huge string, this halves memory consumption. (Strictly speaking, this appears only valid in C++17, but who cares...)
2017-03-01Handle importing NARs containing files greater than 4 GiBEelco Dolstra
Also templatize readInt() to work for various integer types.
2017-02-16Support netrc in <nix/fetchurl.nix>Eelco Dolstra
This allows <nix/fetchurl.nix> to fetch private Git/Mercurial repositories, e.g. import <nix/fetchurl.nix> { url = https://edolstra@bitbucket.org/edolstra/my-private-repo/get/80a14018daed.tar.bz2; sha256 = "1mgqzn7biqkq3hf2697b0jc4wabkqhmzq2srdymjfa6sb9zb6qs7"; } where /etc/nix/netrc contains: machine bitbucket.org login edolstra password blabla... This works even when sandboxing is enabled. To do: add unpacking support (i.e. fetchzip functionality).
2017-02-14Add support for s3:// URIsEelco Dolstra
This adds support for s3:// URIs in all places where Nix allows URIs, e.g. in builtins.fetchurl, builtins.fetchTarball, <nix/fetchurl.nix> and NIX_PATH. It allows fetching resources from private S3 buckets, using credentials obtained from the standard places (i.e. AWS_* environment variables, ~/.aws/credentials and the EC2 metadata server). This may not be super-useful in general, but since we already depend on aws-sdk-cpp, it's a cheap feature to add.
2017-02-08Include config.h implicitly with '-include config.h' in CFLAGSTuomas Tynkkynen
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not every compilation unit includes config.h, we currently compile half of Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS unset. This causes major havoc with the Settings class on e.g. 32-bit ARM, where different compilation units disagree with the struct layout. E.g.: diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc @@ -166,6 +166,8 @@ void Settings::update() _get(useSubstitutes, "build-use-substitutes"); + fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); _get(buildUsersGroup, "build-users-group"); diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn) void RemoteStore::setOptions(Connection & conn) { + fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); conn.to << wopSetOptions Gave me: at Settings::update(): &useSubstitutes = 0xb6e5c5cb at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7 That was not a fun one to debug!
2017-02-07Move SavingSourceAdapter to serialise.hhEelco Dolstra
2017-02-01Restore default signal handling in child processesEelco Dolstra
In particular, this fixes Ctrl-C in nix-shell sessions.
2017-01-26Fix interrupt handlingEelco Dolstra
2017-01-24Work around a bug in clang and older versions of gccEelco Dolstra
http://hydra.nixos.org/build/46597440 https://llvm.org/bugs/show_bug.cgi?id=28096
2017-01-19Merge pull request #981 from shlevy/build-remote-c++Eelco Dolstra
build-remote: Implement in C++
2017-01-19Kill builds when we get EOF on the log FDEelco Dolstra
This closes a long-time bug that allowed builds to hang Nix indefinitely (regardless of timeouts) simply by doing exec > /dev/null 2>&1; while true; do true; done Now, on EOF, we just send SIGKILL to the child to make sure it's really gone.
2017-01-19CleanupEelco Dolstra
2017-01-17Handle SIGINT etc. via a sigwait() signal handler threadEelco Dolstra
This allows other threads to install callbacks that run in a regular, non-signal context. In particular, we can use this to signal the downloader thread to quit. Closes #1183.
2017-01-16AutoCloseDir: Use std::unique_ptrEelco Dolstra
2017-01-16AutoDeleteArray -> std::unique_ptrEelco Dolstra
Also, switch to C++14 for std::make_unique.
2016-12-13Merge branch 'base32-overflow' of git://github.com/vcunat/nixShea Levy
2016-12-08Fix S3BinaryCacheStoreEelco Dolstra
It failed with AWS error uploading ‘6gaxphsyhg66mz0a00qghf9nqf7majs2.ls.xz’: Unable to parse ExceptionName: MissingContentLength Message: You must provide the Content-Length HTTP header. possibly because the istringstream_nocopy introduced in 0d2ebb4373e509521f27a6e8f16bfd39d05b2188 doesn't supply the seek method that the AWS library expects. So bring back the old version, but only for S3BinaryCacheStore.
2016-12-06Use a steady clock for timeoutsEelco Dolstra
Fixes #1146.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-11-17TypoEelco Dolstra
2016-11-17Fix binary-cache-store buildShea Levy
2016-11-17istringstream_nocopy: Implement in a standards-compliant wayShea Levy
Fixes #1135.
2016-11-16S3BinaryCacheStore:: Eliminate a string copy while uploadingEelco Dolstra
This cuts hydra-queue-runner's peak memory usage by about a third.
2016-11-10build-remote: Implement in C++Shea Levy
2016-11-07Add missing #includeEelco Dolstra
2016-11-06Detect and disallow base32 hash overflowVladimír Čunát
Example (before this commit): $ nix-hash --type sha256 --to-base16 4n0igfxbd3kqvvj2k2xgysrp63l4v2gd110fwkk4apfpm0hvzwh0 \ | xargs nix-hash --type sha256 --to-base32 0n0igfxbd3kqvvj2k2xgysrp63l4v2gd110fwkk4apfpm0hvzwh0 It's a real-life example: https://github.com/NixOS/nixpkgs/pull/20208/files#r86695567
2016-10-19Merge branch 'regex' of https://github.com/groxxda/nixEelco Dolstra
2016-10-19Fix Darwin buildEelco Dolstra
Done slightly differently from https://github.com/NixOS/nix/pull/1093.
2016-10-18replace own regex class with std::regexAlexander Ried
2016-10-12Add some functions needed by hydraEelco Dolstra
2016-10-06toJSON(): Support some more typesEelco Dolstra
2016-10-06Merge pull request #998 from veprbl/rx_chmod_fixEelco Dolstra
override rx directory permissions in deletePath()
2016-09-21nix-build, nix-shell: Don't print error message if nix-store/nix-instantiate ↵Eelco Dolstra
fails
2016-09-21printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra
2016-09-21Some notational convenience for formatting stringsEelco Dolstra
We can now write throw Error("file '%s' not found", path); instead of throw Error(format("file '%s' not found") % path); and similarly printError("file '%s' not found", path); instead of printMsg(lvlError, format("file '%s' not found") % path);
2016-09-20TweakEelco Dolstra
2016-09-20nix-shell: Fix $PATH handling in the impure caseEelco Dolstra
We were passing "p=$PATH" rather than "p=$PATH;", resulting in some invalid shell code. Also, construct a separate environment for the child rather than overwriting the parent's.