aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2022-09-19Remove a useless debug message in filetransfer.ccThéophane Hufschmitt
Remove the `verify TLS: Nix CA file = 'blah'` message that Nix used to print when fetching anything as it's both useless (`libcurl` prints the same info in its logs) and misleading (gives the impression that a new TLS connection is being established which might not be the case because of multiplexing. See #7011 )
2022-09-13Merge pull request #6994 from agbrooks/masterThéophane Hufschmitt
Prevent tempdir from being GC-ed before addToStoreFromDump completes
2022-09-12Address PR feedback on #6694Andrew Brooks
2022-09-06Keep created temp dirs inside store, but protect from GCAndrew Brooks
Implements the approach suggested by feedback on PR #6994, where tempdir paths are created in the store (now with an exclusive lock). As part of this work, the currently-broken and unused `createTempDirInStore` function is updated to create an exclusive lock on the temp directory in the store. The GC now makes a non-blocking attempt to lock any store directories that "look like" the temp directories created by this function, and if it can't acquire one, ignores the directory.
2022-09-02Prevent tempdir from being GC-ed before addToStoreFromDump has renamed itAndrew Brooks
This fixes issue 6823 by placing the tempdir used in LocalStore::addToStoreFromDump outside the Nix store, where automatic GC is no longer a concern.
2022-09-02Don’t readDerivation if impure derivations feature is disabledMatthew Bauer
readDerivation is pretty slow, and while it may not be significant for some use cases, on things like ghc-nix where we have thousands of derivations is really slows things down. So, this just doesn’t do the impure derivation check if the impure derivation experimental feature is disabled. Perhaps we could cache the result of isPure() and keep the check, but this is a quick fix to for the slowdown introduced with impure derivations features in 2.8.0.
2022-09-01linkify mention of other optionsAdam Joseph
2022-09-01move substituter signature-checking conditions to configuration file ↵Adam Joseph
documentation
2022-08-26Merge pull request #6954 from winterqt/darwin-sandbox-trustdEelco Dolstra
fix(libstore): allow access to trustd on macOS
2022-08-24fix(libstore): allow access to trustd on macOSWinter
2022-08-24Remove redundant FinallyEelco Dolstra
2022-08-24Fix macOS buildEelco Dolstra
2022-08-23Fix a hang in nix-copy-ssh.shEelco Dolstra
This hang for some reason didn't trigger in the Nix build, but did running 'make installcheck' interactively. What happened: * Store::addMultipleToStore() calls a SinkToSource object to copy a path, which in turn calls LegacySSHStore::narFromPath(), which acquires a connection. * The SinkToSource object is not destroyed after the last bytes has been read, so the coroutine's stack is still alive and its destructors are not run. So the connection is not released. * Then when the next path is copied, because max-connections = 1, LegacySSHStore::narFromPath() hangs forever waiting for a connection to be released. The fix is to make sure that the source object is destroyed when we're done with it.
2022-08-22IndentationEelco Dolstra
2022-08-22Merge remote-tracking branch 'origin/master' into parallel-nix-copyEelco Dolstra
2022-08-22Merge pull request #6926 from rapenne-s/download_limitEelco Dolstra
add a nix.conf option to set a download speed limit
2022-08-22SpellingEelco Dolstra
2022-08-20Better documentation wordingSolène Rapenne
Co-authored-by: Anderson Torres <torres.anderson.85@protonmail.com>
2022-08-19add a nix.conf option to set a download speed limitSolene Rapenne
2022-08-19don't read outputs into memory for output rewritingpennae
RewritingSink can handle being fed input where a reference crosses a chunk boundary. we don't need to load the whole source into memory, and in fact *not* loading the whole source lets nix build FODs that do not fit into memory (eg fetchurl'ing data files larger than system memory).
2022-08-17Remove warnLargeDump()Eelco Dolstra
This message was unhelpful (#1184) and probably misleading since memory is O(1) in most cases now.
2022-08-10Merge pull request #6850 from NinjaTrappeur/nin/build-checkThéophane Hufschmitt
Fix Nix build --check flag
2022-08-03Only use `renameFile` where neededThéophane Hufschmitt
In most places the fallback to copying isn’t needed and can actually be bad, so we’d rather not transparently fallback
2022-08-03moveFile -> renameFileThéophane Hufschmitt
`move` tends to have this `mv` connotation of “I will copy it for you if needs be”
2022-08-03Create a wrapper around stdlib’s `rename`Théophane Hufschmitt
Directly takes some c++ strings, and gently throws an exception on error (rather than having to inline this logic everywhere)
2022-08-01derivation-goal.cc: remove bmCheck custom return branch on buildDoneFélix Baylac-Jacqué
Once a derivation goal has been completed, we check whether or not this goal was meant to be repeated to check its output. An early return branch was preventing the worker to reach that repeat code branch, hence breaking the --check command (#2619). It seems like this early return branch is an artifact of a passed refactoring. As far as I can tell, buildDone's main branch also cleanup the tmp directory before returning.
2022-07-22Merge pull request #6814 from amjoseph-nixpkgs/pr/sandbox-error-messagesThéophane Hufschmitt
local-derivation-goal.cc: improve error messages when sandboxing fails
2022-07-20Merge branch 'master' into parallel-nix-copyThéophane Hufschmitt
2022-07-19libstore/globals.cc: Move cgroup detection to libutilAlex Wied
2022-07-19libstore/globals.cc: Automatically set cores based on cgroup CPU limitAlex Wied
By default, Nix sets the "cores" setting to the number of CPUs which are physically present on the machine. If cgroups are used to limit the CPU and memory consumption of a large Nix build, the OOM killer may be invoked. For example, consider a GitLab CI pipeline which builds a large software package. The GitLab runner spawns a container whose CPU is limited to 4 cores and whose memory is limited to 16 GiB. If the underlying machine has 64 cores, Nix will invoke the build with -j64. In many cases, that level of parallelism will invoke the OOM killer and the build will completely fail. This change sets the default value of "cores" to be ceil(cpu_quota / cpu_period), with a fallback to std::thread::hardware_concurrency() if cgroups v2 is not detected.
2022-07-19Don't ultimately trust the signed pathsThéophane Hufschmitt
Like the old implem did (and like you'd want it to be anyways)
2022-07-19local-derivation-goal.cc: save global errno to the stack before performing ↵Adam Joseph
tests which might clobber it
2022-07-19as requested by @thufschmitt ↵Adam Joseph
https://github.com/NixOS/nix/pull/6814#discussion_r924275777
2022-07-19change warn() to notice()Adam Joseph
2022-07-17local-derivation-goal.cc: detect unprivileged_userns_clone failure modeAdam Joseph
The workaround for "Some distros patch Linux" mentioned in local-derivation-goal.cc will not help in the `--option sandbox-fallback false` case. To provide the user more helpful guidance on how to get the sandbox working, let's check to see if the `/proc` node created by the aforementioned patch is present and configured in a way that will cause us problems. If so, give the user a suggestion for how to troubleshoot the problem.
2022-07-17local-derivation-goal.cc: add comment re: CLONE_NEWUSERAdam Joseph
local-derivation-goal.cc contains a comment stating that "Some distros patch Linux to not allow unprivileged user namespaces." Let's give a pointer to a common version of this patch for those who want more details about this failure mode.
2022-07-16local-derivation-goal.cc: warn if failing and /proc/self/ns/user missingAdam Joseph
This commit causes nix to `warn()` if sandbox setup has failed and `/proc/self/ns/user` does not exist. This is usually a sign that the kernel was compiled without `CONFIG_USER_NS=y`, which is required for sandboxing.
2022-07-16local-derivation-goal.cc: warn if failing due to max_user_namespaces==0Adam Joseph
This commit uses `warn()` to notify the user if sandbox setup fails with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or zero, since that is at least part of the reason why sandbox setup failed. Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or equivalent at boot time has been the recommended mitigation for several Linux LPE vulnerabilities over the past few years. Many users have applied this mitigation and then forgotten that they have done so.
2022-07-16local-derivation-goal.cc: improve error messages when sandboxing failsAdam Joseph
The failure modes for nix's sandboxing setup are pretty complicated. When nix is unable to set up the sandbox, let's provide more detail about what went wrong. Specifically: * Make sure the error message includes the word "sandbox" so the user knows that the failure was related to sandboxing. * If `--option sandbox-fallback false` was provided, and removing it would have allowed further attempts to make progress, let the user know.
2022-07-15fix(libstore): allow Nix to access all Rosetta 2 paths on MacOSAlex Wied
Fixes: #5884
2022-07-15Disable auto-chroot if $NIX_STATE_DIR is setEelco Dolstra
Issue #6732.
2022-07-14Fix bug, test more, document moreJohn Ericson
2022-07-14Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputsJohn Ericson
2022-07-14Disable auto-chroot if $NIX_STORE_DIR is setEelco Dolstra
Fixes #6732.
2022-06-29Don't fail if we can't create ~/.local/share/nix/rootEelco Dolstra
https://hydra.nixos.org/build/182135943
2022-06-29Merge pull request #6706 from lheckemann/cache-info-cache-invalidationThéophane Hufschmitt
nar-info-disk-cache: refresh nix-cache-info weekly
2022-06-25Merge pull request #6714 from edolstra/auto-chroot-storeEelco Dolstra
Automatically use a chroot store if /nix doesn't exist
2022-06-24Only do the auto chroot store on LinuxEelco Dolstra
2022-06-23[fixup] handle cache expiration in sqlite queryLinus Heckemann
2022-06-23libstore/nar-info: drop unused system fieldCole Helbling
This was unused everywhere (and even the official NixOS binary cache did not produce .narinfo files containing a "System:" field).