aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2023-03-16Merge pull request #8049 from edolstra/unexpected-eofEelco Dolstra
Fix "unexpected EOF" errors on macOS
2023-03-16LocalDerivationGoal: set NIX_ATTRS_*_FILE correctly for sandboxed buildsLinus Heckemann
2023-03-15Remove a variable in LocalDerivationGoalEelco Dolstra
2023-03-15Change builderOut from Pipe to AutoCloseFDEelco Dolstra
2023-03-15Move pseudoterminal slave open to childEelco Dolstra
Hopefully this fixes "unexpected EOF" failures on macOS (#3137, #3605, #7242, #7702). The problem appears to be that under some circumstances, macOS discards the output written to the slave side of the pseudoterminal. Hence the parent never sees the "sandbox initialized" message from the child, even though it succeeded. The conditions are: * The child finishes very quickly. That's why this bug is likely to trigger in nix-env tests, since that uses a builtin builder. Adding a short sleep before the child exits makes the problem go away. * The parent has closed its duplicate of the slave file descriptor. This shouldn't matter, since the child has a duplicate as well, but it does. E.g. moving the close to the bottom of startBuilder() makes the problem go away. However, that's not a solution because it would make Nix hang if the child dies before sending the "sandbox initialized" message. * The system is under high load. E.g. "make installcheck -j16" makes the issue pretty reproducible, while it's very rare under "make installcheck -j1". As a fix/workaround, we now open the pseudoterminal slave in the child, rather than the parent. This removes the second condition (i.e. the parent no longer needs to close the slave fd) and I haven't been able to reproduce the "unexpected EOF" with this.
2023-03-14Simplify commonChildInit()Eelco Dolstra
2023-03-10Merge pull request #8015 from tweag/progress-during-nix-copyThéophane Hufschmitt
Display progress when running copyPaths (nix copy)
2023-03-09Make findRuntimeRoots() more resilient to disappearing processesEelco Dolstra
I saw this random failure in https://hydra.nixos.org/build/211811692: error: opening /proc/15307/fd: No such process while running nix-collect-garbage in a readfile-context.sh. This is because we're not handling ESRCH errors reading /proc/<pid>/fd. So just move the read inside the try/catch where we do handle it.
2023-03-09Display progress when running copyPaths (nix copy)Alexander Bantyev
`nix copy` operations did not show progress. This is quite confusing. Add a `progressSink` which displays the progress during `copyPaths`, pretty much copied from `copyStorePath`. Fixes https://github.com/NixOS/nix/issues/8000
2023-03-08Fix crash/hang with CA derivationsEelco Dolstra
The curl download can outlive DrvOutputSubstitutionGoal (if some other error occurs), so at shutdown setting the promise to an exception will fail because 'this' is no longer valid in the callback. This can manifest itself as a segfault, "corrupted double-linked list" or hang.
2023-03-03Merge pull request #7918 from zimbatm/fix-empty-nix-store-envEelco Dolstra
treat empty NIX_STORE_DIR env vars as unset
2023-03-03Merge pull request #7942 from edolstra/remove-formatEelco Dolstra
Remove FormatOrString and remaining uses of format()
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2023-03-02Log the decompressed body in case of http errorsThéophane Hufschmitt
Don't show the users the raw (possibly compressed) error message as everyone isn't able to decompress brotli in their brain.
2023-03-02Merge pull request #7924 from mkenigs/validEelco Dolstra
Always set valid in path-info --json output
2023-03-01Treat empty env var paths as unsetFélix Baylac Jacqué
We make sure the env var paths are actually set (ie. not "") before sending them to the canonicalization function. If we forget to do so, the user will end up facing a puzzled failed assertion internal error. We issue a non-failing warning as a stop-gap measure. We could want to revisit this to issue a detailed failing error message in the future.
2023-03-01Remove dead code `RemoteStore::sameMachine`John Ericson
It has been dead code since 9747ea84b43c75f719d719673b9d0a7cb50d34e5.
2023-03-01Merge pull request #7788 from ↵Valentin Gagarin
bobvanderlinden/pr-improve-nix-profile-install-error Improve error on conflict for nix profile install
2023-02-28Always set valid in path-info --json outputMatthew Kenigsberg
Currently the valid key is only present when the path is invalid, which makes checking path validity more complex than it should be. With this change, the valid key can always be used to check if a path is valid
2023-02-28Merge pull request #7793 from layus/interrupt_downloadsEelco Dolstra
Check interrupts even when download stalled
2023-02-27buildenv: throw BuildEnvFileConflictError with more contextBob van der Linden
At the moment an Error is thrown that only holds an error message regarding `nix-env` and `nix profile`. These tools make use of builtins.buildEnv, but buildEnv is also used in other places. These places are unrelated to Nix profiles, so the error shouldn't mention these tools. This generic error is now BuildEnvFileConflictError, which holds more contextual information about the files that were conflicting while building the environment.
2023-02-27add information on the `build-hook` settingValentin Gagarin
add a warning that you probably never want to change this.
2023-02-21Merge pull request #7856 from yorickvP/fix-nsswitchThéophane Hufschmitt
Wait with making /etc unwritable until after build env setup
2023-02-20`Derivation::toJSON`: fix bug!John Ericson
When I moved this code from the binary to libnixstore #7863, I forgot to display the environment variables!
2023-02-19Test `toJSON` of `DerivationOutput` and `Derivation`John Ericson
2023-02-19Move `Derivation` toJSON logic to libnixstoreJohn Ericson
2023-02-17Wait with making /etc unwritable until after build env setupYorick van Pelt
This fixes /etc/nsswitch.conf
2023-02-14Make /etc writability conditional on uid-range featureYorick van Pelt
2023-02-14Don't allow writing to /etcYorick van Pelt
2023-02-13Merge pull request #7616 from hercules-ci/fix-3898Eelco Dolstra
Fix foreign key error inserting into NARs #3898
2023-02-10Merge pull request #7805 from edolstra/c++2aEelco Dolstra
Fix building with GCC 9
2023-02-10Merge pull request #7802 from edolstra/fix-7783Eelco Dolstra
Fix PID namespace support check
2023-02-10Fix building with GCC 9Eelco Dolstra
Nixpkgs on aarch64-linux is currently stuck on GCC 9 (https://github.com/NixOS/nixpkgs/issues/208412) and using gcc11Stdenv doesn't work either. So use c++2a instead of c++20 for now. Unfortunately this means we can't use some C++20 features for now (like std::span).
2023-02-10Merge pull request #5588 from tweag/balsoft/xdgThéophane Hufschmitt
Follow XDG Base Directory standard
2023-02-10A setting to follow XDG Base Directory standardAlexander Bantyev
XDG Base Directory is a standard for locations for storing various files. Nix has a few files which seem to fit in the standard, but currently use a custom location directly in the user's ~, polluting it: - ~/.nix-profile - ~/.nix-defexpr - ~/.nix-channels This commit adds a config option (use-xdg-base-directories) to follow the XDG spec and instead use the following locations: - $XDG_STATE_HOME/nix/profile - $XDG_STATE_HOME/nix/defexpr - $XDG_STATE_HOME/nix/channels If $XDG_STATE_HOME is not set, it is assumed to be ~/.local/state. Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com> Co-authored-by: Tim Fenney <kodekata@gmail.com> Co-authored-by: pasqui23 <pasqui23@users.noreply.github.com> Co-authored-by: Artturin <Artturin@artturin.com> Co-authored-by: John Ericson <Ericson2314@Yahoo.com>
2023-02-10LocalDerivationGoal::startBuilder(): Use startProcess() to cloneEelco Dolstra
2023-02-10Simplify the PID namespace check: just try to mount /procEelco Dolstra
Fixes #7783.
2023-02-10nit: cleaner diffGuillaume Maudoux
2023-02-10Flush data when download endsGuillaume Maudoux
2023-02-09Merge pull request #7712 from Mic92/advertise-compressionsEelco Dolstra
advertise transport encoding in http transfers to
2023-02-09Check interupts even when download stalledGuillaume Maudoux
2023-02-07NarInfoDiskCache: Also test id consistency with updated fieldsRobert Hensing
And clarify test
2023-02-07NarInfoDiskCache: Keep BinaryCache.id stable and improve testRobert Hensing
Fixes #3898 The entire `BinaryCaches` row used to get replaced after it became stale according to the `timestamp` column. In a concurrent scenario, this leads to foreign key conflicts as different instances of the in-process `state.caches` cache now differ, with the consequence that the older process still tries to use the `id` number of the old record. Furthermore, this phenomenon appears to have caused the cache for actual narinfos to be erased about every week, while the default ttl for narinfos was supposed to be 30 days.
2023-02-07NarInfoDiskCache: Prepare reproducer for #3898Robert Hensing
2023-02-07NarInfoDiskCacheImpl: Make dbPath a parameterRobert Hensing
This allows testing with a clean database.
2023-02-07NarInfoDiskCache: Rename cacheExists -> upToDateCacheExistsRobert Hensing
This is slightly more accurate considering that an outdated record may exist in the persistent cache. Possibly-outdated records are quite relevant as they may be foreign keys to more recent information that we want to keep, but we will not return them here.
2023-02-07sqlite.cc: Add SQL tracingRobert Hensing
Set environment variable NIX_DEBUG_SQLITE_TRACES=1 to log all sql statements.
2023-02-07Mention --no-sandbox if sandboxing is unsupportedEelco Dolstra
2023-02-07Check whether we can use PID namespacesEelco Dolstra
In unprivileged podman containers, /proc is not fully visible (there are other filesystems mounted on subdirectories of /proc). Therefore we can't mount a new /proc in the sandbox that matches the PID namespace of the sandbox. So this commit automatically disables sandboxing if /proc is not fully visible.
2023-02-07Fix auto-uid-allocation in Docker containersEelco Dolstra
This didn't work because sandboxing doesn't work in Docker. However, the sandboxing check is done lazily - after clone(CLONE_NEWNS) fails, we retry with sandboxing disabled. But at that point, we've already done UID allocation under the assumption that user namespaces are enabled. So let's get rid of the "goto fallback" logic and just detect early whether user / mount namespaces are enabled. This commit also gets rid of a compatibility hack for some ancient Linux kernels (<2.13).