aboutsummaryrefslogtreecommitdiff
path: root/src/libutil/archive.cc
AgeCommit message (Collapse)Author
2024-09-13archive: refactor bad mutable-state API in the NAR parse listenerJade Lovelace
Remove the mutable state stuff that assumes that one file is being written a time. It's true that we don't write multiple files interleaved, but that mutable state is evil. Change-Id: Ia1481da48255d901e4b09a9b783e7af44fae8cff
2024-09-11archive: rename ParseSink to NARParseVisitorJade Lovelace
- Rename the listener to not be called a "sink". If it were a "sink" it would be eating bytes and conform with any of the Nix sink stuff (maybe FileHandle should be a Sink itself! but that's a later CL's problem). This is a parser listener. - Move the RetrieveRegularNARSink thing into store-api.cc, which is its only usage, and fix it to actually do what it is stated to do: crash if its invariants are violated. It's, of course, used to erm, unpack single-file NAR files, generated via a horrible contraption of sources and sinks that looks like a plumbing blueprint. Refactoring that is a future task. - Add a description of the invariants of NARParseVisitor in preparation of refactoring it. Change-Id: Ifca1d74d2947204a1f66349772e54dad0743e944
2024-08-04tree-wide: fix various lint warningsJade Lovelace
Change-Id: I0fc80718eb7e02d84cc4b5d5deec4c0f41116134
2024-07-16libstore: rewrite the nar parser as a contents generatoreldritch horrors
this is not completely necessary at this point because the parser right now already returns a generator to pass through all input data it read, but the nar parser *was* very lax and would accept nars that weren't in canonical form (defined as the form dumpPath would return). nar hashing depends on these things, and as such rewriting the parser now allows us to reject non-canonical nars that extract to the same store contents as their canonical counterpart but have different nar hashes despite that. Change-Id: Iccd319e3bd5912d8297014c84c495edc59019bb7
2024-07-11libstore: turn copyNAR into a generatoreldritch horrors
Change-Id: Id452f6a03faa1037ff13af0f63e32883966ff40d
2024-07-11libstore: turn the NAR parser into a passthrough generatoreldritch horrors
this will let us turn copyNAR into a generator as well, which in turn is necessary to turn the users of copyNAR into generators without resorting to sinkToSource coroutines. currently this uses the SerializingTransform in all cases, even for copyNAR where it is not necessary. should this be a performance problem we can easily swap out the transform for one which does not produce any bytes of its own, but that should not be necessary. Change-Id: I7e685879318fcbb78d8b88abfddd7752360eb0ce
2024-07-05libstore: convert dumpPath to a generatoreldritch horrors
Change-Id: Ic4cf5562504aa29130304469936f958c0426e5ef
2024-05-29util.hh: Delete remaining file and clean up headersTom Hubrecht
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29util.{hh,cc}: Split out file-system.{hh,cc}Tom Hubrecht
Change-Id: Ifa89a529e7e34e7291eca87d802d2f569cf2493e
2024-05-22libstore: remove unused copyPath functioneldritch horrors
Change-Id: Ibfb0848d8b337bd38947a745c240838865cea401
2024-03-18libutil: make AutoCloseFD a better resourceeldritch horrors
add a reset() method to close the wrapped fd instead of assigning magic constants. also make the from-fd constructor explicit so you can't accidentally assign the *wrong* magic constant, or even an unrelated integer that also just happens to be an fd by pure chance. Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-04Merge pull request #9673 from pennae/drv-parse-optseldritch horrors
optimize derivation parsing (cherry picked from commit 3511430902941f0f26dc71313a54bb5096f57305) Change-Id: I00f76dcd464a5811944613731501af504b6e8c29
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2022-12-07Trivial changes from the lazy-trees branchEelco Dolstra
2022-09-22archive: check close errors when extracting narssqualus
2022-03-15libfetchers/path: set `lastModified` to path's mtimeMaximilian Bosch
When importing e.g. a local `nixpkgs` in a flake to test a change like { inputs.nixpkgs.url = path:/home/ma27/Projects/nixpkgs; outputs = /* ... */ } then the input is missing a `lastModified`-field that's e.g. used in `nixpkgs.lib.nixosSystem`. Due to the missing `lastMoified`-field, the mtime is set to 19700101: result -> /nix/store/b7dg1lmmsill2rsgyv2w7b6cnmixkvc1-nixos-system-nixos-22.05.19700101.dirty With this change, the `path`-fetcher now sets a `lastModified` attribute to the `mtime` just like it's the case in the `tarball`-fetcher already. When building NixOS systems with `nixpkgs` being a `path`-input and this patch, the output-path now looks like this: result -> /nix/store/ld2qf9c1s98dxmiwcaq5vn9k5ylzrm1s-nixos-system-nixos-22.05.20220217.dirty
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-01-07dump(): Use emplace()Eelco Dolstra
2021-10-02fix creation of NAR files >4GB on 32-bit platformskvtb
`size_t` is 32-bit on 32-bit platforms, while the file size can be bigger
2020-12-02read(): Use char * instead of unsigned char *Eelco Dolstra
This gets rid of some pointless casts.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-11-26archive: disable preallocate-contents by defaultDominique Martinet
using fallocate() to preallocate files space does more harm than good: - breaks compression on btrfs - has been called "not the right thing to do" by xfs developers (because delayed allocation that most filesystems implement leads to smarter allocation than what the filesystem needs to do if we upfront fallocate files)
2020-10-06Remove static variable name clashesEelco Dolstra
This was useful for an experiment with building Nix as a single compilation unit. It's not very useful otherwise but also doesn't hurt...
2020-09-23Merge pull request #4054 from edolstra/fix-4021Eelco Dolstra
registerOutputs(): Don't call canonicalisePathMetaData() twice
2020-09-23lstat() cleanupEelco Dolstra
2020-09-23libutil/archive: add preallocate-contents optionDominique Martinet
Make archive preallocation (fallocate) optional because some filesystems like btrfs do not behave as expected with fallocate. See #3550.
2020-08-13Use `TeeSink` and `TeeSouce` in a few more placesJohn Ericson
2020-07-30unsigned long long -> uint64_tEelco Dolstra
2020-07-30receiveContents(): unsigned int -> size_tEelco Dolstra
2020-06-29NAR parser: Fix missing name field checkEelco Dolstra
Discovered by @Kloenk.
2020-04-21remove 'format' from Error constructor callsBen Burdette
2019-10-27Add O(1)-memory copyPath() functionEelco Dolstra
2019-03-01archive.cc: ignore more posix_fallocate "not supported" error codesWill Dietz
Fixes w/musl.
2018-09-26Make NAR header check more robustEelco Dolstra
Changes std::bad_alloc into bad archive: input doesn't look like a Nix archive
2018-05-30Modularize config settingsEelco Dolstra
Allow global config settings to be defined in multiple Config classes. For example, this means that libutil can have settings and evaluator settings can be moved out of libstore. The Config classes are registered in a new GlobalConfig class to which config files etc. are applied. Relevant to https://github.com/NixOS/nix/issues/2009 in that it removes the need for ad hoc handling of useCaseHack, which was the underlying cause of that issue.
2018-03-22TweakEelco Dolstra
2018-03-22Merge branch 'fix/avoid-large-stack-buffers' of https://github.com/dtzWill/nixEelco Dolstra
2018-03-21Make 'nix copy --from ssh://...' run in constant memoryEelco Dolstra
For instance, this reduced the memory consumption of $ nix copy --from ssh://localhost --to ~/my-nix /nix/store/1n7x0yv8vq6zi90hfmian84vdhd04bgp-blender-2.79a from 632 MiB to 16 MiB.
2018-03-02don't allocate large buffers on the stackWill Dietz
2017-10-30builtins.fetchgit: Support importing a working treeEelco Dolstra
For example, you can write src = fetchgit ./.; and if ./. refers to an unclean working tree, that tree will be copied to the Nix store. This removes the need for "cleanSource".
2017-08-16nix copy: Revive progress barEelco Dolstra
2017-07-31Merge branch 'macOS' of https://github.com/davidak/nixEelco Dolstra
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30replace "Mac OS X" with "macOS"davidak
except in older release notes where the name was actually Mac OS X.
2017-02-08Include config.h implicitly with '-include config.h' in CFLAGSTuomas Tynkkynen
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not every compilation unit includes config.h, we currently compile half of Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS unset. This causes major havoc with the Settings class on e.g. 32-bit ARM, where different compilation units disagree with the struct layout. E.g.: diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc @@ -166,6 +166,8 @@ void Settings::update() _get(useSubstitutes, "build-use-substitutes"); + fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); _get(buildUsersGroup, "build-users-group"); diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn) void RemoteStore::setOptions(Connection & conn) { + fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); conn.to << wopSetOptions Gave me: at Settings::update(): &useSubstitutes = 0xb6e5c5cb at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7 That was not a fun one to debug!
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-10-19Fix Darwin buildEelco Dolstra
Done slightly differently from https://github.com/NixOS/nix/pull/1093.
2016-09-21printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra