aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/optimise-store.cc
AgeCommit message (Collapse)Author
2018-06-14Fix optimise-store.cc to skip more paths on macOSMatthew Justin Bauer
I hate to make this such a large check but the lack of documentation means we really have no idea what's allowed. All of them reported so far have been within ".app/Contents" directories. That appears to be a safe starting point. However, I would not be surprised to also find more paths that are disallowed for instance in .framework or .bundle directories. Fixes #2031 Fixes #2229
2018-03-06rand() -> random(), since we use srandom().Will Dietz
rand() requires we call srand() instead, but might as well use random().
2017-08-28Give activities a verbosity level againEelco Dolstra
And print them (separately from the progress bar) given sufficient -v flags.
2017-08-16nix optimise-store: Show how much space has been freedEelco Dolstra
2017-08-16nix optimise-store: AddEelco Dolstra
This replaces "nix-store --optimise". Main difference is that it has a progress indicator.
2017-08-10Style fixEelco Dolstra
2017-07-31Merge branch 'macOS' of https://github.com/davidak/nixEelco Dolstra
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30replace "OS X" with "macOS"davidak
2017-07-07Merge pull request #1445 from matthewbauer/macos-skip-hardlinkEelco Dolstra
Don’t hardlink disallowed paths in OS X.
2017-07-06Don’t hardlink disallowed paths in OS X.Matthew Bauer
Fixes #1443
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-05-16Improve progress indicatorEelco Dolstra
2017-05-01Reduce severity of EMLINK warningsEelco Dolstra
Fixes #1357.
2017-02-08Include config.h implicitly with '-include config.h' in CFLAGSTuomas Tynkkynen
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not every compilation unit includes config.h, we currently compile half of Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS unset. This causes major havoc with the Settings class on e.g. 32-bit ARM, where different compilation units disagree with the struct layout. E.g.: diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc @@ -166,6 +166,8 @@ void Settings::update() _get(useSubstitutes, "build-use-substitutes"); + fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); _get(buildUsersGroup, "build-users-group"); diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn) void RemoteStore::setOptions(Connection & conn) { + fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); conn.to << wopSetOptions Gave me: at Settings::update(): &useSubstitutes = 0xb6e5c5cb at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7 That was not a fun one to debug!
2017-01-16AutoCloseDir: Use std::unique_ptrEelco Dolstra
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-11-14Don't rely on %mEelco Dolstra
2016-11-14daemon: Do not error out when deduplication fails due to ENOSPC.Ludovic Courtès
This solves a problem whereby if /gnu/store/.links had enough entries, ext4's directory index would be full, leading to link(2) returning ENOSPC. * nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Upon ENOSPC from link(2), print a message and return instead of throwing a 'SysError'.
2016-09-21printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra
2016-06-02LocalStore: Allow the physical and logical store directories to differEelco Dolstra
This is primarily to subsume the functionality of the copy-from-other-stores substituter. For example, in the NixOS installer, we can now do (assuming we're in the target chroot, and the Nix store of the installation CD is bind-mounted on /tmp/nix): $ nix-build ... --option substituters 'local?state=/tmp/nix/var&real=/tmp/nix/store' However, unlike copy-from-other-stores, this also allows write access to such a store. One application might be fetching substitutes for /nix/store in a situation where the user doesn't have sufficient privileges to create /nix, e.g.: $ NIX_REMOTE="local?state=/home/alice/nix/var&real=/home/alice/nix/store" nix-build ...
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-04-25Improved logging abstractionEelco Dolstra
This also gets rid of --log-type, since the nested log type isn't useful in a multi-threaded situation, and nobody cares about the "pretty" log type.
2015-11-09optimizePath(): Detect some .links corruptionEelco Dolstra
If automatic store optimisation is enabled, and a hard-linked file in the store gets corrupted, then the corresponding .links entry will also be corrupted. In that case, trying to repair with --repair or --repair-path won't work, because the new "good" file will be replaced by a hard link to the corrupted file. We can catch most of these cases by doing a sanity-check on the file sizes.
2015-07-17OCD: foreach -> C++11 ranged forEelco Dolstra
2014-12-14PedantryEelco Dolstra
2014-12-09Explicitly include required C headersMarko Durkovic
2014-09-01Add an 'optimiseStore' remote procedure call.Ludovic Courtès
2014-08-20Use proper quotes everywhereEelco Dolstra
2014-05-15lvlInfo -> lvlTalkativeEelco Dolstra
2014-05-15nix-store --optimise: Remove bogus statisticsEelco Dolstra
2014-05-15Remove tabEelco Dolstra
2014-05-15Shortcut store files before lstatWout Mertens
readdir() already returns the inode numbers, so we don't need to call lstat to know if a file was already linked or not.
2014-05-14Use the inodes given by readdir directlyWout Mertens
2014-05-13Preload linked hashes to speed up lookupsWout Mertens
By preloading all inodes in the /nix/store/.links directory, we can quickly determine of a hardlinked file was already linked to the hashed links. This is tolerant of removing the .links directory, it will simply recalculate all hashes in the store.
2014-05-10Shortcut already-hardlinked fileswmertens
If an inode in the Nix store has more than 1 link, it probably means that it was linked into .links/ by us. If so, skip. There's a possibility that something else hardlinked the file, so it would be nice to be able to override this. Also, by looking at the number of hardlinks for each of the files in .links/, you can get deduplication numbers and space savings.
2013-03-08Revert "Prevent config.h from being clobbered"Eelco Dolstra
This reverts commit 28bba8c44f484eae38e8a15dcec73cfa999156f6.
2013-03-07Prevent config.h from being clobberedEelco Dolstra
2013-02-27Refactoring: Split off the non-recursive canonicalisePathMetaData()Eelco Dolstra
Also, change the file mode before changing the owner. This prevents a slight time window in which a setuid binary would be setuid root.
2013-02-26Security: Don't allow builders to change permissions on files they don't ownEelco Dolstra
It turns out that in multi-user Nix, a builder may be able to do ln /etc/shadow $out/foo Afterwards, canonicalisePathMetaData() will be applied to $out/foo, causing /etc/shadow's mode to be set to 444 (readable by everybody but writable by nobody). That's obviously Very Bad. Fortunately, this fails in NixOS's default configuration because /nix/store is a bind mount, so "ln" will fail with "Invalid cross-device link". It also fails if hard-link restrictions are enabled, so a workaround is: echo 1 > /proc/sys/fs/protected_hardlinks The solution is to check that all files in $out are owned by the build user. This means that innocuous operations like "ln ${pkgs.foo}/some-file $out/" are now rejected, but that already failed in chroot builds anyway.
2013-01-03Clear any immutable bits in the Nix storeEelco Dolstra
Doing this once makes subsequent operations like garbage collecting more efficient since we don't have to call makeMutable() first.
2012-09-19Remove setting of the immutable bitEelco Dolstra
Using the immutable bit is problematic, especially in conjunction with store optimisation. For instance, if the garbage collector deletes a file, it has to clear its immutable bit, but if the file has additional hard links, we can't set the bit afterwards because we don't know the remaining paths. So now that we support having the entire Nix store as a read-only mount, we may as well drop the immutable bit. Unfortunately, we have to keep the code to clear the immutable bit for backwards compatibility.
2012-08-27Merge branch 'master' into no-manifestsEelco Dolstra
2012-08-07Don't warn about maximum link count exceeded on 0-byte filesEelco Dolstra
2012-08-05Fix race condition when two processes create a hard link to a file in .linksEelco Dolstra
This is a problem because one process may set the immutable bit before the second process has created its link. Addressed random Hydra failures such as: error: cannot rename `/nix/store/.tmp-link-17397-1804289383' to `/nix/store/rsvzm574rlfip3830ac7kmaa028bzl6h-nixos-0.1pre-git/upstart-interface-version': Operation not permitted
2012-08-05Fix race condition when two processes create the same link in /nix/store/.linksEelco Dolstra
2012-08-01Merge branch 'master' into no-manifestsEelco Dolstra
2012-08-01DohEelco Dolstra
2012-08-01Make ‘nix-store --optimise’ interruptibleEelco Dolstra