aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/gc.cc
AgeCommit message (Collapse)Author
2024-05-18Improve nix-store --delete failure messageAlyssa Ross
On several occasions I've found myself confused when trying to delete a store path, because I am told it's still alive, but nix-store --query --roots doesn't show anything. Let's save future users this confusion by mentioning that a path might be alive due to having referrers, not just roots. (cherry picked from commit 979a019014569eee7d0071605f6ff500b544f6ac) Upstream-PR: https://github.com/NixOS/nix/pull/10733 Change-Id: I54ae839a85f3de3393493fba27fd40d7d3af0516
2024-04-23libstore: Create platform LocalStore subclassesArtemis Tosini
This creates new subclasses of LocalStore for each OS to include platform-specific functionality. Currently this just includes garbage collector roots but it could be extended to sandboxing as well. In order to make sure that the generic LocalStore is not accidentally constructed, its constructor is protected. A Fallback is provided which implements no functionality except constructors. Change-Id: I836a28e90b68309873f75afb83e0f1b2e2c89fb3
2024-03-18libutil: make AutoCloseFD a better resourceeldritch horrors
add a reset() method to close the wrapped fd instead of assigning magic constants. also make the from-fd constructor explicit so you can't accidentally assign the *wrong* magic constant, or even an unrelated integer that also just happens to be an fd by pure chance. Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-07Merge pull request #8544 from edolstra/handle-missing-gc-socketeldritch horrors
LocalStore: :addTempRoot(): Handle ENOENT (cherry picked from commit 7115edc85af060ef235ac0270245ab46cc828f7c) Change-Id: Ie6b1596049c3fde09b98f2f0727899f98e48e6b1
2024-03-04Merge pull request #7348 from thufschmitt/dont-use-vlaseldritch horrors
Remove the usage of VLAs in the code (cherry picked from commit ac4431e9d016e62fb5dc9ae36833bd0c6cdadeec) Change-Id: Ifbf5fbfc2e27122362a2aaea4b62c7cf3ca46b1a
2023-12-01Put functional tests in `tests/functional`John Ericson
I think it is bad for these reasons when `tests/` contains a mix of functional and integration tests - Concepts is harder to understand, the documentation makes a good unit vs functional vs integration distinction, but when the integration tests are just two subdirs within `tests/` this is not clear. - Source filtering in the `flake.nix` is more complex. We need to filter out some of the dirs from `tests/`, rather than simply pick the dirs we want and take all of them. This is a good sign the structure of what we are trying to do is not matching the structure of the files. With this change we have a clean: ```shell-session $ git show 'HEAD:tests' tree HEAD:tests functional/ installer/ nixos/ ``` (cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
2023-07-24Clean up store hierarchy with `IndirectRootStore`John Ericson
See the API doc comments for details.
2023-06-20Merge pull request #8552 from edolstra/fix-eagainEelco Dolstra
GC server: Clear O_NONBLOCK on the right file descriptor
2023-06-20Support opening local store with database on read-only filesystem (#8356)Ben Radford
Previously it was not possible to open a local store when its database is on a read-only filesystem. Obviously a store on a read-only filesystem cannot be modified, but it would still be useful to be able to query it. This change adds a new read-only setting to LocalStore. When set to true, Nix will skip operations that fail when the database is on a read-only filesystem (acquiring big-lock, schema migration, etc), and the store database will be opened in immutable mode. Co-authored-by: Ben Radford <benradf@users.noreply.github.com> Co-authored-by: cidkidnix <cidkidnix@protonmail.com> Co-authored-by: Dylan Green <67574902+cidkidnix@users.noreply.github.com> Co-authored-by: John Ericson <git@JohnEricson.me> Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-20GC server: Clear O_NONBLOCK on the right file descriptorEelco Dolstra
The bug fix in 6d30f9e6fea7d451033653f8f167aef58f7f5347 erroneously cleared O_NONBLOCK on the server rather than client FD (leaving both in an incorrect state). Fixes #8551.
2023-03-09Make findRuntimeRoots() more resilient to disappearing processesEelco Dolstra
I saw this random failure in https://hydra.nixos.org/build/211811692: error: opening /proc/15307/fd: No such process while running nix-collect-garbage in a readfile-context.sh. This is because we're not handling ESRCH errors reading /proc/<pid>/fd. So just move the read inside the try/catch where we do handle it.
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2023-01-03Fix deadlock between auto-GC and addTempRoot()Eelco Dolstra
Previously addTempRoot() acquired the LocalStore state lock and waited for the garbage collector to reply. If the garbage collector is in the same process (as it the case with auto-GC), this would deadlock as soon as the garbage collector thread needs the LocalStore state lock. So now addTempRoot() uses separate Syncs for the state that it needs. As long at the auto-GC thread doesn't call addTempRoot() (which it shouldn't), it shouldn't deadlock. Fixes #3224.
2023-01-03Move creation of the temp roots file into its own functionEelco Dolstra
This also moves the file handle into its own Sync object so we're not holding the _state while acquiring the file lock. There was no real deadlock risk here since locking a newly created file cannot block, but it's still a bit nicer.
2022-11-27Fix random client failures during GC server shutdownEelco Dolstra
We need to close the GC server socket before shutting down the active GC client connections, otherwise a client may (re)connect and get ECONNRESET. But also handle ECONNRESET for resilience. Fixes random failures like GC socket disconnected connecting to '/tmp/nix-shell.y07M0H/nix-test/default/var/nix/gc-socket/socket' sending GC root '/tmp/nix-shell.y07M0H/nix-test/default/store/kb5yzija0f1x5xkqkgclrdzldxj6nnc6-non-blocking' reading GC root from client: error: unexpected EOF reading a line 1 store paths deleted, 0.00 MiB freed error: reading from file: Connection reset by peer in gc-non-blocking.sh.
2022-09-12Address PR feedback on #6694Andrew Brooks
2022-09-06Keep created temp dirs inside store, but protect from GCAndrew Brooks
Implements the approach suggested by feedback on PR #6994, where tempdir paths are created in the store (now with an exclusive lock). As part of this work, the currently-broken and unused `createTempDirInStore` function is updated to create an exclusive lock on the temp directory in the store. The GC now makes a non-blocking attempt to lock any store directories that "look like" the temp directories created by this function, and if it can't acquire one, ignores the directory.
2022-08-03moveFile -> renameFileThéophane Hufschmitt
`move` tends to have this `mv` connotation of “I will copy it for you if needs be”
2022-08-03Create a wrapper around stdlib’s `rename`Théophane Hufschmitt
Directly takes some c++ strings, and gently throws an exception on error (rather than having to inline this logic everywhere)
2022-06-10Add missing rethrows in conditional exception handlersAnders Kaseorg
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-03-17gc: don't visit implicit referrers on garbage collectionSergei Trofimovich
Before the change garbage collector was not considering `.drv` and outputs as alive even if configuration says otherwise. As a result `nix store gc --dry-run` could visit (and parse) `.drv` files multiple times (worst case it's quadratic). It happens because `alive` set was populating only runtime closure without regard for actual configuration. The change fixes it. Benchmark: my system has about 139MB, 40K `.drv` files. Performance before the change: $ time nix store gc --dry-run real 4m22,148s Performance after the change: $ time nix store gc --dry-run real 0m14,178s
2022-03-13nix store gc: account for auto-optimised storeSergei Trofimovich
Before the change on a system with `auto-optimise-store = true`: $ nix store gc --verbose --max 1 deleted all the paths instead of one path (we requested 1 byte limit). It happens because every file in `auto-optimise-store = true` has at least 2 links: file itself and a link in /nix/store/.links/ directory. The change conservatively assumes that any file that has one (as before) or two links (assume auto-potimise mode) will free space. Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-02-28Fix Darwin buildEelco Dolstra
Fixes #6169
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2021-12-13More debug infoEelco Dolstra
2021-12-13Retry on ECONNREFUSEDEelco Dolstra
https://hydra.nixos.org/build/161439235
2021-12-13Explicitly make GC roots client sockets blockingEelco Dolstra
On macOS / BSD, these sockets inherit the non-blocking flag of the server soocket, which is not what we want. https://github.com/dotnet/runtime/issues/25069 https://bugs.python.org/issue7995 https://hydra.nixos.org/build/161439304
2021-11-22Unify #if linuxAlex Shabalin
2021-11-19Fix build warnings on MacOSAlex Shabalin
2021-10-28Use nix::connect() to connect to the garbage collectorEelco Dolstra
2021-10-28Remove unused variableEelco Dolstra
2021-10-15Add a test for the non-blocking GCEelco Dolstra
2021-10-15Fix main GC thread exitingEelco Dolstra
2021-10-15Fix crash when a GC client disconnectsEelco Dolstra
The client thread can't just delete its own thread object from connections, it has to detach it.
2021-10-15Memoize queryReferrers()Eelco Dolstra
2021-10-14Speed up GC by marking entire closures as liveEelco Dolstra
2021-10-14Remove GCStateEelco Dolstra
2021-10-14Move deleteFromStore()Eelco Dolstra
2021-10-14Make the canReachRoots() traversal non-recursiveEelco Dolstra
2021-10-13Fix GC when there are cycles in the referrers graphEelco Dolstra
(where "referrers" includes the reverse of derivation outputs and derivers). Now we do a full traversal to look if we can reach any root. If not, all paths reached can be deleted.
2021-10-13SimplifyEelco Dolstra
2021-10-13Use a thread per connectionEelco Dolstra
2021-10-13Fix auto-gcEelco Dolstra
2021-10-13Non-blocking garbage collectorEelco Dolstra
The garbage collector no longer blocks other processes from adding/building store paths or adding GC roots. To prevent the collector from deleting store paths just added by another process, processes need to connect to the garbage collector via a Unix domain socket to register new temporary roots.
2021-10-13Remove syncWithGC()Eelco Dolstra
2021-10-13Remove trash directoryEelco Dolstra
2021-06-18UDSRemoteStore: Support the 'root' store parameterEelco Dolstra
Useful when we're using a daemon with a chroot store, e.g. $ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot Then the client can now connect with $ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
2020-10-12Handle amount of disk space saved by hard linking being negativevolth
Fixes bogus messages like "currently hard linking saves 17592186044416.00 MiB".
2020-10-09Split out `local-fs-store.hh`John Ericson
This matches the already-existing `local-fs-store.cc`.
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 19:00:47 +0000