aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/optimise-store.cc
AgeCommit message (Collapse)Author
2015-11-09optimizePath(): Detect some .links corruptionEelco Dolstra
If automatic store optimisation is enabled, and a hard-linked file in the store gets corrupted, then the corresponding .links entry will also be corrupted. In that case, trying to repair with --repair or --repair-path won't work, because the new "good" file will be replaced by a hard link to the corrupted file. We can catch most of these cases by doing a sanity-check on the file sizes.
2015-07-17OCD: foreach -> C++11 ranged forEelco Dolstra
2014-12-14PedantryEelco Dolstra
2014-12-09Explicitly include required C headersMarko Durkovic
2014-09-01Add an 'optimiseStore' remote procedure call.Ludovic Courtès
2014-08-20Use proper quotes everywhereEelco Dolstra
2014-05-15lvlInfo -> lvlTalkativeEelco Dolstra
2014-05-15nix-store --optimise: Remove bogus statisticsEelco Dolstra
2014-05-15Remove tabEelco Dolstra
2014-05-15Shortcut store files before lstatWout Mertens
readdir() already returns the inode numbers, so we don't need to call lstat to know if a file was already linked or not.
2014-05-14Use the inodes given by readdir directlyWout Mertens
2014-05-13Preload linked hashes to speed up lookupsWout Mertens
By preloading all inodes in the /nix/store/.links directory, we can quickly determine of a hardlinked file was already linked to the hashed links. This is tolerant of removing the .links directory, it will simply recalculate all hashes in the store.
2014-05-10Shortcut already-hardlinked fileswmertens
If an inode in the Nix store has more than 1 link, it probably means that it was linked into .links/ by us. If so, skip. There's a possibility that something else hardlinked the file, so it would be nice to be able to override this. Also, by looking at the number of hardlinks for each of the files in .links/, you can get deduplication numbers and space savings.
2013-03-08Revert "Prevent config.h from being clobbered"Eelco Dolstra
This reverts commit 28bba8c44f484eae38e8a15dcec73cfa999156f6.
2013-03-07Prevent config.h from being clobberedEelco Dolstra
2013-02-27Refactoring: Split off the non-recursive canonicalisePathMetaData()Eelco Dolstra
Also, change the file mode before changing the owner. This prevents a slight time window in which a setuid binary would be setuid root.
2013-02-26Security: Don't allow builders to change permissions on files they don't ownEelco Dolstra
It turns out that in multi-user Nix, a builder may be able to do ln /etc/shadow $out/foo Afterwards, canonicalisePathMetaData() will be applied to $out/foo, causing /etc/shadow's mode to be set to 444 (readable by everybody but writable by nobody). That's obviously Very Bad. Fortunately, this fails in NixOS's default configuration because /nix/store is a bind mount, so "ln" will fail with "Invalid cross-device link". It also fails if hard-link restrictions are enabled, so a workaround is: echo 1 > /proc/sys/fs/protected_hardlinks The solution is to check that all files in $out are owned by the build user. This means that innocuous operations like "ln ${pkgs.foo}/some-file $out/" are now rejected, but that already failed in chroot builds anyway.
2013-01-03Clear any immutable bits in the Nix storeEelco Dolstra
Doing this once makes subsequent operations like garbage collecting more efficient since we don't have to call makeMutable() first.
2012-09-19Remove setting of the immutable bitEelco Dolstra
Using the immutable bit is problematic, especially in conjunction with store optimisation. For instance, if the garbage collector deletes a file, it has to clear its immutable bit, but if the file has additional hard links, we can't set the bit afterwards because we don't know the remaining paths. So now that we support having the entire Nix store as a read-only mount, we may as well drop the immutable bit. Unfortunately, we have to keep the code to clear the immutable bit for backwards compatibility.
2012-08-27Merge branch 'master' into no-manifestsEelco Dolstra
2012-08-07Don't warn about maximum link count exceeded on 0-byte filesEelco Dolstra
2012-08-05Fix race condition when two processes create a hard link to a file in .linksEelco Dolstra
This is a problem because one process may set the immutable bit before the second process has created its link. Addressed random Hydra failures such as: error: cannot rename `/nix/store/.tmp-link-17397-1804289383' to `/nix/store/rsvzm574rlfip3830ac7kmaa028bzl6h-nixos-0.1pre-git/upstart-interface-version': Operation not permitted
2012-08-05Fix race condition when two processes create the same link in /nix/store/.linksEelco Dolstra
2012-08-01Merge branch 'master' into no-manifestsEelco Dolstra
2012-08-01DohEelco Dolstra
2012-08-01Make ‘nix-store --optimise’ interruptibleEelco Dolstra
2012-07-30Refactor settings processingEelco Dolstra
Put all Nix configuration flags in a Settings object.
2012-07-26Merge branch 'master' into no-manifestsEelco Dolstra
2012-07-23Handle platforms that don't support linking to a symlinkEelco Dolstra
E.g. Darwin doesn't allow this.
2012-07-23Unlink the right fileEelco Dolstra
2012-07-23Automatically optimise the Nix store when a new path is addedEelco Dolstra
Auto-optimisation is enabled by default. It can be turned off by setting auto-optimise-store to false in nix.conf.
2012-07-23optimiseStore(): Use a content-addressed file store in /nix/store/.linksEelco Dolstra
optimiseStore() now creates persistent, content-addressed hard links in /nix/store/.links. For instance, if it encounters a file P with hash H, it will create a hard link P' = /nix/store/.link/<H> to P if P' doesn't already exist; if P' exist, then P is replaced by a hard link to P'. This is better than the previous in-memory map, because it had the tendency to unnecessarily replace hard links with a hard link to whatever happened to be the first file with a given hash it encountered. It also allows on-the-fly, incremental optimisation.
2012-07-11Rename queryValidPaths() to queryAllValidPaths()Eelco Dolstra
2012-02-15On Linux, make the Nix store really read-only by using the immutable bitEelco Dolstra
I was bitten one time too many by Python modifying the Nix store by creating *.pyc files when run as root. On Linux, we can prevent this by setting the immutable bit on files and directories (as in ‘chattr +i’). This isn't supported by all filesystems, so it's not an error if setting the bit fails. The immutable bit is cleared by the garbage collector before deleting a path. The only tricky aspect is in optimiseStore(), since it's forbidden to create hard links to an immutable file. Thus optimiseStore() temporarily clears the immutable bit before creating the link.
2010-11-16* Store the size of a store path in the database (to be precise, theEelco Dolstra
size of the NAR serialisation of the path, i.e., `nix-store --dump PATH'). This is useful for Hydra.
2010-06-04* Applied a patch from David Brown to prevent `nix-store --optimise'Eelco Dolstra
from failing on rename() on BtrFS.
2009-09-24* And some more.Eelco Dolstra
2009-04-21* Use foreach in a lot of places.Eelco Dolstra
2008-06-18* `nix-store --optimise': handle files with >= 32000 hard links.Eelco Dolstra
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18* Some refactoring: put the GC options / results in separate structs.Eelco Dolstra
* The garbage collector now also prints the number of blocks freed.
2008-06-09* Merged the no-bdb branch (-r10900:HEADEelco Dolstra
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).