aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/pathlocks.cc
AgeCommit message (Collapse)Author
2024-03-18libutil: make AutoCloseFD a better resourceeldritch horrors
add a reset() method to close the wrapped fd instead of assigning magic constants. also make the from-fd constructor explicit so you can't accidentally assign the *wrong* magic constant, or even an unrelated integer that also just happens to be an fd by pure chance. Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-11util.hh: split out signals stuffJade Lovelace
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7 Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2021-10-13Non-blocking garbage collectorEelco Dolstra
The garbage collector no longer blocks other processes from adding/building store paths or adding GC roots. To prevent the collector from deleting store paths just added by another process, processes need to connect to the garbage collector via a Unix domain socket to register new temporary roots.
2020-05-03convert some printError calls to logErrorBen Burdette
2020-04-21remove 'format' from Error constructor callsBen Burdette
2019-08-07pathlocks: add include to fcntl.h for O_CLOEXECWill Dietz
2019-08-02Use BSD instead of POSIX file locksEelco Dolstra
POSIX file locks are essentially incompatible with multithreading. BSD locks have much saner semantics. We need this now that there can be multiple concurrent LocalStore::buildPaths() invocations.
2018-02-12Fix 'deadlock: trying to re-acquire self-held lock'Eelco Dolstra
This was caused by derivations with 'allowSubstitutes = false'. Such derivations will be built locally. However, if there is another SubstitionGoal that has the output of the first derivation in its closure, then the path will be simultaneously built and substituted. There was a check to catch this situation (via pathIsLockedByMe()), but it no longer worked reliably because substitutions are now done in another thread. (Thus the comment 'It can't happen between here and the lockPaths() call below because we're not allowing multi-threading' was no longer valid.) The fix is to handle the path already being locked in both SubstitutionGoal and DerivationGoal.
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-01-26Fix assertion failure when a path is lockedEelco Dolstra
Fixes: nix-store: src/libstore/build.cc:3649: void nix::Worker::run(const Goals&): Assertion `!awake.empty()' failed.
2017-01-26openLockFile: Return an AutoCloseFDEelco Dolstra
2017-01-19Merge pull request #981 from shlevy/build-remote-c++Eelco Dolstra
build-remote: Implement in C++
2016-12-09Probably fix a segfault in PathLocksEelco Dolstra
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-11-10build-remote: Implement in C++Shea Levy
2016-09-21printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra
2016-07-11Modernize AutoCloseFDShea Levy
2016-06-09Use O_CLOEXEC in most placesEelco Dolstra
2016-01-04~PathLocks(): Handle exceptionsEelco Dolstra
Otherwise, since the call to write a "d" character to the lock file can fail with ENOSPC, we can get an unhandled exception resulting in a call to terminate().
2015-07-17OCD: foreach -> C++11 ranged forEelco Dolstra
2014-12-12Ensure we're writing to stderr in the builderEelco Dolstra
http://hydra.nixos.org/build/17862041
2014-08-20Use proper quotes everywhereEelco Dolstra
2012-03-05Set the close-on-exec flag on file descriptorsEelco Dolstra
2011-12-21* Another case of lock file permissions being too liberal.Eelco Dolstra
2010-02-03* Revert r19797, and use a simpler solution: just don't monitor buildEelco Dolstra
hooks for silence. It's unnecessary because the remote nix-store command is already monitoring the real build.
2010-02-03* While waiting for a lock, print a sign of life every 5 minutes.Eelco Dolstra
This prevents remote builders from being killed by the `max-silent-time' inactivity monitor while they are waiting for a long garbage collection to finish. This happens fairly often in the Hydra build farm.
2010-02-02* Remove most Cygwin-specific code. Cygwin 1.7 implements advisoryEelco Dolstra
POSIX locks, and simulates Unix-style file deletion semantics sufficiently. Note that this means that Nix won't work on Cygwin 1.5 anymore.
2009-04-21* Use foreach in a lot of places.Eelco Dolstra
2009-03-23* No longer block while waiting for a lock on a store path. InsteadEelco Dolstra
poll for it (i.e. if we can't acquire the lock, then let the main select() loop wait for at most a few seconds and then try again). This improves parallelism: if two nix-store processes are both trying to build a path at the same time, the second one shouldn't block; it should first see if it can build other goals. Also, it prevents the deadlocks that have been occuring in Hydra lately, where a process waits for a lock held by another process that's waiting for a lock held by the first. The downside is that polling isn't really elegant, but POSIX doesn't provide a way to wait for locks in a select() loop. The only solution would be to spawn a thread for each lock to do a blocking fcntl() and then signal the main thread, but that would require pthreads.
2009-02-16* Release output locks as soon as possible, not when the destructor ofEelco Dolstra
the DerivationGoal runs. Otherwise, if a goal is a top-level goal, then the lock won't be released until nix-store finishes. With --keep-going and lots of top-level goals, it's possible to run out of file descriptors (this happened sometimes in the build farm for Nixpkgs). Also, for failed derivation, it won't be possible to build it again until the lock is released. * Idem for locks on build users: these weren't released in a timely manner for failed top-level derivation goals. So if there were more than (say) 10 such failed builds, you would get an error about having run out of build users.
2008-05-21* GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour andEelco Dolstra
Armijn Hemel.
2007-08-28* Fix a race condition with parallel builds where multipleEelco Dolstra
fixed-output derivations or substitutions try to build the same store path at the same time. Locking generally catches this, but not between multiple goals in the same process. This happened especially often (actually, only) in the build farm with fetchurl downloads of the same file being executed on multiple machines and then copied back to the main machine where they would clobber each other (NIXBF-13). Solution: if a goal notices that the output path is already locked, then go to sleep until another goal finishes (hopefully the one locking the path) and try again.
2007-08-28* PathLocks::lockPaths: don't allow reacquiring a lock we alreadyEelco Dolstra
hold.
2006-09-04* Use a proper namespace.Eelco Dolstra
* Optimise header file usage a bit. * Compile the parser as C++.
2006-06-20* Concurrent GC on Cygwin.Eelco Dolstra
2006-06-19* On Windows we cannot delete open (lock) files, so we delete lockEelco Dolstra
files after we've closed them. Since this only succeeds if the lock is no longer opened by any process, the token trick used on Unix is not necessary.
2006-06-15* In `nix-env -i|-u|-e', lock the profile to prevent races betweenEelco Dolstra
concurrent nix-env operations on the same profile. Fixes NIX-7.
2005-01-31* Don't delete active lock files.Eelco Dolstra
2005-01-27* Make lock removal safe by signalling to blocked processes that theEelco Dolstra
lock they are waiting on has become stale (we do this by writing a meaningless token to the unlinked file).
2004-05-11* True parallel builds. Nix can now run as many build jobs inEelco Dolstra
parallel as possible (similar to GNU Make's `-j' switch). This is useful on SMP systems, but it is especially useful for doing builds on multiple machines. The idea is that a large derivation is initiated on one master machine, which then distributes sub-derivations to any number of slave machines. This should not happen synchronously or in lock-step, so the master must be capable of dealing with multiple parallel build jobs. We now have the infrastructure to support this. TODO: substitutes are currently broken.
2004-01-15* Catch SIGINT to terminate cleanly when the user tries to interruptEelco Dolstra
Nix. This is to prevent Berkeley DB from becoming wedged. Unfortunately it is not possible to throw C++ exceptions from a signal handler. In fact, you can't do much of anything except change variables of type `volatile sig_atomic_t'. So we set an interrupt flag in the signal handler and check it at various strategic locations in the code (by calling checkInterrupt()). Since this is unlikely to cover all cases (e.g., (semi-)infinite loops), sometimes SIGTERM may now be required to kill Nix.
2003-12-21* Bug fix: parallel builds of the same derivation failed due to lock file ↵Eelco Dolstra
removal.
2003-11-21* Remove lock files after building. Eelco Dolstra
2003-11-18* libnix -> libstore.Eelco Dolstra