aboutsummaryrefslogtreecommitdiff
path: root/src/libutil/serialise.cc
AgeCommit message (Collapse)Author
2024-05-29util.hh: Delete remaining file and clean up headersTom Hubrecht
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-07libutil: throw EndOfFile at sourceToSink endeldritch horrors
... how has this never broken anything before Change-Id: If3789c02028e8f929481514f63d76b0b46bfc182
2024-04-05Revert "libutil: drop Fs{Source,Sink}::good"eldritch horrors
This reverts commit 1340807e30dba4b3972c31f02861bbaeaeb60e61. Change-Id: I34d2a80eb3c3e9d79cb02b92cd1189da32d18cb6
2024-03-31libutil: drop Fs{Source,Sink}::goodeldritch horrors
setting this only on exceptions caused by actual fd access is not sufficient to diagnose all errors (such as SerialisationError) in some cases. this usually does not have any negative effects since those errors will end up killing the process in another way. this is not a reliable assumption though and we should be using proper error handling (and closing connections more often, preferring to close over keeping something open that might be in a weird state) Change-Id: I1b792cd7ad8ba9ff0f6bd174945ab2575ff2208e
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-09Merge pull request #9925 from 9999years/fmt-cleanupeldritch horrors
Cleanup `fmt.hh` (cherry picked from commit 47a1dbb4b8e7913cbb9b4d604728b912e76e4ca0) Change-Id: Id076a45cb39652f437fe3f8bda10c310a9894777
2024-03-04Merge pull request #9798 from edolstra/remote-store-eofeldritch horrors
Print a more helpful message if the daemon crashes (cherry picked from commit 32706b14a7531c2c21b9f96da083a540a0031ec4) Change-Id: Ief7c465bca7666e2b7e7c9d1dd0c01c5f9014146
2024-03-04Merge pull request #9673 from pennae/drv-parse-optseldritch horrors
optimize derivation parsing (cherry picked from commit 3511430902941f0f26dc71313a54bb5096f57305) Change-Id: I00f76dcd464a5811944613731501af504b6e8c29
2024-02-22Fix bad_format_string error when builder stdout contains % (#10063)github-actions[bot]
(cherry picked from commit e2b6821ca0147f36bcb9404aab080f80746984c8) Co-authored-by: roblabla <unfiltered@roblab.la>
2023-04-14Merge pull request #8176 from tweag/rename-confusing-write-methodEelco Dolstra
Rename and protect `BufferedSink::write`
2023-04-07Always disable GC in a coroutine unless the patch is appliedYorick van Pelt
2023-04-07DisableGC: replace by CoroutineContext, std::shared_ptr<void>Yorick van Pelt
2023-04-07Disable GC inside coroutines on mac OSYorick van Pelt
2023-04-07Rename and protect `BufferedSink::write`Théophane Hufschmitt
The `write` name is ambiguous and could lead to some funny bugs like https://github.com/NixOS/nix/pull/8173#issuecomment-1500009480. So rename it to the more explicit `writeUnbuffered`. Besides, this method shouldn't be (and isn't) used outside of the class implementation, so mark it `protected`. This makes it more symetrical to `BufferedSource` which uses a `protected readUnbuffered` method.
2023-03-08Revert "Disable GC during coroutine execution + test"Théophane Hufschmitt
2023-03-08Merge pull request #7725 from yorickvP/check-coro-gcThéophane Hufschmitt
Disable GC during coroutine execution + test
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2023-03-01Always disable GC in a coroutine unless the patch is appliedYorick van Pelt
2023-03-01DisableGC: replace by CoroutineContext, std::shared_ptr<void>Yorick van Pelt
2023-03-01Disable GC inside coroutines on mac OSYorick van Pelt
2022-12-07Trivial changes from the lazy-trees branchEelco Dolstra
2022-08-17Remove warnLargeDump()Eelco Dolstra
This message was unhelpful (#1184) and probably misleading since memory is O(1) in most cases now.
2022-04-08Remove unused Error.name fieldEelco Dolstra
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2022-01-17serialise.hh: Use std::string_viewEelco Dolstra
2021-09-14Fix clang warningEelco Dolstra
2021-03-10Use libarchive for all compressionYorick van Pelt
2021-01-21Improve error formattingEelco Dolstra
Changes: * The divider lines are gone. These were in practice a bit confusing, in particular with --show-trace or --keep-going, since then there were multiple lines, suggesting a start/end which wasn't the case. * Instead, multi-line error messages are now indented to align with the prefix (e.g. "error: "). * The 'description' field is gone since we weren't really using it. * 'hint' is renamed to 'msg' since it really wasn't a hint. * The error is now printed *before* the location info. * The 'name' field is no longer printed since most of the time it wasn't very useful since it was just the name of the exception (like EvalError). Ideally in the future this would be a unique, easily googleable error ID (like rustc). * "trace:" is now just "…". This assumes error contexts start with something like "while doing X". Example before: error: --- AssertionError ---------------------------------------------------------------------------------------- nix at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix 6| 7| x = assert false; 1; | ^ 8| assertion 'false' failed ----------------------------------------------------- show-trace ----------------------------------------------------- trace: while evaluating the attribute 'x' of the derivation 'hello-2.10' at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix 191| // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) { 192| name = "${attrs.pname}-${attrs.version}"; | ^ 193| } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) { Example after: error: assertion 'false' failed at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix 6| 7| x = assert false; 1; | ^ 8| … while evaluating the attribute 'x' of the derivation 'hello-2.10' at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix 191| // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) { 192| name = "${attrs.pname}-${attrs.version}"; | ^ 193| } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
2020-12-02read(): Use char * instead of unsigned char *Eelco Dolstra
This gets rid of some pointless casts.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-11-10Fix stack overflow introduced in #4206Ricardo M. Correia
2020-10-30Fix memory corruption caused by GC-invisible coroutine stacksRobert Hensing
Crucially this introduces BoehmGCStackAllocator, but it also adds a bunch of wiring to avoid making libutil depend on bdw-gc. Part of the solutions for #4178, #4200
2020-10-07Serialize exceptions from the sandbox process to the parentEelco Dolstra
Fixes #4118.
2020-09-17Add Source.drainInto(Sink)Robert Hensing
2020-07-15Revert "LocalStore::addToStoreFromDump copy in chunks"John Ericson
This reverts commit 592851fb67cd15807109d6f65fb81f6af89af966. We don't need this extra feature anymore
2020-07-15Get rid of `LocalStore::addToStoreCommon`John Ericson
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking a store and then doing nothing too fancy with it. `LocalStore::addToStore(...Path...)` is now just a simple wrapper with a bare-bones sinkToSource of the right dump command.
2020-07-14LocalStore::addToStoreFromDump copy in chunksJohn Ericson
Rather than copying byte-by-byte, we let the coroutine know how much data we would like it to send back to us.
2020-06-15Get rid of explicit ErrorInfo constructorsEelco Dolstra
2020-05-11switch from printError warnings to logWarningsBen Burdette
2019-03-14experimental/optional -> optionalEelco Dolstra
2018-09-26sinkToSource(): Start the coroutine lazilyEelco Dolstra
In particular this causes copyStorePath() from HttpBinaryCacheStore to only start a download if needed. E.g. if the destination LocalStore goes to sleep waiting for the path lock and another process creates the path, then LocalStore::addToStore() will never read from the source so we don't have to do the download.
2018-09-26Make NAR header check more robustEelco Dolstra
Changes std::bad_alloc into bad archive: input doesn't look like a Nix archive
2018-08-21Improve 'coroutine has finished' error messageEelco Dolstra
2018-07-31Add a check for broken Boost versionsEelco Dolstra
In some Boost versions, coroutines don't propagate exceptions properly, causing Nix to fail with the exception 'coroutine has finished'.
2018-05-21serialise: fix buffer size used, hide method for internal use onlyWill Dietz
Fixes #2169.
2018-03-19serialise.cc: remove pessimising moveWill Dietz
from clang6: src/libutil/serialise.cc:189:23: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
2018-03-16Reduce substitution memory consumptionEelco Dolstra
copyStorePath() now pipes the output of srcStore->narFromPath() directly into dstStore->addToStore(). The sink used by the former is converted into a source usable by the latter using boost::coroutine2. This is based on [1]. This reduces the maximum resident size of $ nix build --store ~/my-nix/ /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1 --substituters file:///tmp/binary-cache-xz/ --no-require-sigs from 418592 KiB to 53416 KiB. (The previous commit also reduced the runtime from ~4.2s to ~3.4s, not sure why.) A further improvement will be to download files into a Sink. [1] https://github.com/NixOS/nix/compare/master...Mathnerd314:dump-fix-coroutine#diff-dcbcac55a634031f9cc73707da6e4b18 Issue #1969.
2018-02-13Fix #1762Linus Heckemann
nix-store --export, nix-store --dump, and nix dump-path would previously fail silently if writing the data out failed, because a) FdSink::write ignored exceptions, and b) the commands relied on FdSink's destructor, which ignores exceptions, to flush the data out. This could cause rather opaque issues with installing nixos, because nix-store --export would happily proceed even if it couldn't write its data out (e.g. if nix-store --import on the other side of the pipe failed). This commit adds tests that expose these issues in the nix-store commands, and fixes them for all three.
2017-03-01readString(): Read directly into std::stringEelco Dolstra
When reading a huge string, this halves memory consumption. (Strictly speaking, this appears only valid in C++17, but who cares...)