aboutsummaryrefslogtreecommitdiff
path: root/src/libutil/compression.cc
AgeCommit message (Collapse)Author
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-04-21remove 'format' from Error constructor callsBen Burdette
2019-12-13Improve gzip error messageEelco Dolstra
2019-12-13Initial gzip supportTom Bereknyei
Closes #3256
2018-09-17Fix build failure if parallel xz is not availableEelco Dolstra
2018-08-21Fix warnings in compression.ccEelco Dolstra
2018-08-06Fix Brotli decompression in 'nix log'Eelco Dolstra
This didn't work anymore since decompression was only done in the non-coroutine case. Decompressors are now sinks, just like compressors. Also fixed a bug in bzip2 API handling (we have to handle BZ_RUN_OK rather than BZ_OK), which we didn't notice because there was a missing 'throw': if (ret != BZ_OK) CompressionError("error while compressing bzip2 file");
2018-08-06Require libbrotliEelco Dolstra
2018-05-02Fix bzip2 compression of files > 4 GiBEelco Dolstra
Bzip2's 'avail_in' parameter is declared as an unsigned int, so assigning a size_t length to it led to silent truncation. Fixes #2111.
2018-03-22Fix buildEelco Dolstra
2018-03-16decompress(): Use a Source and SinkEelco Dolstra
This allows decompression to happen in O(1) memory.
2018-02-11compression: make parallel sink separate classWill Dietz
2018-02-11compression: print warning if parallel requested but not supportedWill Dietz
2018-02-09configure.ac: check if lzma has MT support, fix deb build/etc.Will Dietz
2018-02-07make multi threaded compression configurable and use single threadedAmineChikhaoui
by default.
2018-02-07set block size to 0 to let the lzma lib choose the right one, addAmineChikhaoui
some comments about possible improvements wrt memory usage/threading.
2018-02-06support multi threaded xz encoder, this might be particularly useful inAmineChikhaoui
the case of hydra where the overhead of single threaded encoding is more noticeable e.g most of the time spent in "Sending inputs"/"Receiving outputs" is due to compression while the actual upload to the binary cache seems to be negligible.
2017-12-30use libbrotli directly when availableWill Dietz
* Look for both 'brotli' and 'bro' as external command, since upstream has renamed it in newer versions. If neither are found, current runtime behavior is preserved: try to find 'bro' on PATH. * Limit amount handed to BrotliEncoderCompressStream to ensure interrupts are processed in a timely manner. Testing shows negligible performance impact. (Other compression sinks don't seem to require this)
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-03-22Fix xz decompressionEelco Dolstra
Fixes #1285.
2017-03-21LocalFSStore::getBuildLog(): Handle corrupted logsEelco Dolstra
2017-03-21Don't hang in decompression if bzip2 data ends prematurelyEelco Dolstra
2017-03-15runProgram(): Distinguish between empty input and no inputEelco Dolstra
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2017-03-15S3BinaryCacheStore: Support compression of narinfo and log filesEelco Dolstra
You can now set the store parameter "text-compression=br" to compress textual files in the binary cache (i.e. narinfo and logs) using Brotli. This sets the Content-Encoding header; the extension of compressed files is unchanged. You can separately specify the compression of log files using "log-compression=br". This is useful when you don't want to compress narinfo files for backward compatibility.
2017-03-15Fix assertion failure on SIGINTEelco Dolstra
nix: src/libutil/compression.cc:142: virtual nix::XzSink::~XzSink(): Assertion `finished' failed.
2017-03-15Add support for brotli compressionEelco Dolstra
Build logs on cache.nixos.org are compressed using Brotli (since this allows them to be decompressed automatically by Chrome and Firefox), so it's handy if "nix log" can decompress them.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-05-04Do compression in a sinkEelco Dolstra
2016-05-04Add a Store::addToStore() variant that accepts a NAREelco Dolstra
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04Fix segfault in xz compression/decompressionEelco Dolstra
2016-04-29BinaryCacheStore: Support bzip2 compressionEelco Dolstra
2016-04-29BinaryCacheStore: Support "none" compression methodEelco Dolstra
2016-04-22Make compression interruptibleEelco Dolstra
2016-04-15BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-02-15Add xz compression functionEelco Dolstra
This is used by the Hydra queue runner, but since it may also be useful for the C++ rewrite of nix-push, I'm putting it here.
2016-02-15decompressXZ: Ensure that lzma_end() is calledEelco Dolstra
Otherwise we might leak memory.
2016-01-05Fixed unresolved 'BUFSIZ'FrankHB
`BUFSIZ` is defined in header `<cstdio>`.
2015-10-30<nix/fetchurl.nix>: Support xz-compressed NARsEelco Dolstra