aboutsummaryrefslogtreecommitdiff
path: root/src/libutil/util.cc
AgeCommit message (Collapse)Author
2022-01-18Merge pull request #5906 from pennae/primops-optimizationEelco Dolstra
optimize primops and utils by caching more and copying less
2022-01-18Merge pull request #5932 from edolstra/remove-shared-stringsEelco Dolstra
Remove shared strings
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2022-01-17Merge pull request #5812 from pennae/small-perf-improvementsEelco Dolstra
improve parser performance a bit
2022-01-13avoid copies of parser input datapennae
when given a string yacc will copy the entire input to a newly allocated location so that it can add a second terminating NUL byte. since the parser is a very internal thing to EvalState we can ensure that having two terminating NUL bytes is always possible without copying, and have the parser itself merely check that the expected NULs are present. # before Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms] Range (min … max): 566.9 ms … 579.1 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms] Range (min … max): 380.2 ms … 387.7 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s] Range (min … max): 2.923 s … 2.946 s 50 runs # after Benchmark 1: nix search --offline nixpkgs hello Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms] Range (min … max): 566.7 ms … 579.7 ms 50 runs Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms] Range (min … max): 374.5 ms … 379.1 ms 50 runs Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s] Range (min … max): 2.906 s … 2.934 s 50 runs
2022-01-13use more string_view in utilspennae
there's a couple places that can be easily converted from using strings to using string_views instead. gives a slight (~1%) boost to system eval. # before nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' Time (mean ± σ): 2.946 s ± 0.026 s [User: 2.655 s, System: 0.209 s] Range (min … max): 2.905 s … 2.995 s 20 runs # after Time (mean ± σ): 2.928 s ± 0.024 s [User: 2.638 s, System: 0.211 s] Range (min … max): 2.893 s … 2.970 s 20 runs
2022-01-07isInDir() / isDirOrInDir(): Use std::string_viewEelco Dolstra
2021-12-22Remove CPU lockingEelco Dolstra
This was already accidentally disabled in ba87b08. It also no longer appears to be beneficial, and in fact slow things down, e.g. when evaluating a NixOS system configuration: elapsed time: median = 3.8170 mean = 3.8202 stddev = 0.0195 min = 3.7894 max = 3.8600 [rejected, p=0.00000, Δ=0.36929±0.02513]
2021-12-16Ignore EPERM when unsharing FS stateEelco Dolstra
On Docker (but not podman), unshare(CLONE_FS) fails with EPERM. So let's ignore it and hope nothing bad happens. Attempted fix for #5777.
2021-11-29createTempFile(): Mark file as CLOEEXECEelco Dolstra
Fixes #5674.
2021-11-24Fix a minor data race with _isInterruptedAlex Shabalin
2021-11-17Fix XDG_CONFIG_DIRS fallbackJan Tojnar
According to XDG Base Directory Specification, it should fall back to /etc/xdg when the env var is not present.
2021-11-16Ignore errors unsharing/restoring the mount namespaceEelco Dolstra
This prevents Nix from barfing when run in a container where it doesn't have the appropriate privileges.
2021-11-08Merge branch 'fix-writable-shell' of https://github.com/yorickvP/nixEelco Dolstra
2021-10-29Merge branch 'master' of https://github.com/alekswn/nixEelco Dolstra
2021-10-17Fix error detection in 'base64Decode()'Alexey Novikov
Fixed a bug in initialization of 'base64DecodeChars' variable. Currently decoder do not fail on invalid Base64 strings. Added test-case to verify the fix. Also have made 'base64DecodeChars' to be computed at compile time. And added a test case to encode/decode string with non-printable charactes.
2021-10-15Restore parent mount namespace in restoreProcessContextYorick van Pelt
This ensures any started processes can't write to /nix/store (except during builds). This partially reverts 01d07b1e, which happened because of #2646. The problem was only happening after nix downloads anything, causing me to suspect the download thread. The problem turns out to be: "A process can't join a new mount namespace if it is sharing filesystem-related attributes with another process", in this case this process is the curl thread. Ideally, we might kill it before spawning the shell process, but it's inside a static variable in the getFileTransfer() function. So instead, stop it from sharing FS state using unshare(). A strategy such as the one from #5057 (single-threaded chroot helper binary) is also very much on the table. Fixes #4337.
2021-10-13strcpy -> memcpyEelco Dolstra
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2021-10-13Non-blocking garbage collectorEelco Dolstra
The garbage collector no longer blocks other processes from adding/building store paths or adding GC roots. To prevent the collector from deleting store paths just added by another process, processes need to connect to the garbage collector via a Unix domain socket to register new temporary roots.
2021-10-06Don't reset the logger in a vforkEelco Dolstra
9c766a40cbbd3a350a9582d0fd8201e3361a63b2 broke logging from the daemon, because commonChildInit is called when starting the build hook in a vfork, so it ends up resetting the parent's logger. So don't vfork. It might be best to get rid of vfork altogether, but that may cause problems, e.g. when we call an external program like git from the evaluator.
2021-10-05Show failing PIDEelco Dolstra
2021-10-05TypoEelco Dolstra
2021-10-05Connect/bind Unix domain sockets in a child processEelco Dolstra
In the child process, we can do a chdir() and avoid the problem of the path not fitting into sockaddr_un.
2021-09-27Fix 'error: reading a line: Input/output error' in startBuilder()Eelco Dolstra
With -vvvv, the ProgressBar was polluting the stderr of the child, messing up its \2 message to the parent.
2021-09-14Fix macOS buildEelco Dolstra
2021-09-13RunOptions: Use designated initializersEelco Dolstra
Also get rid of _killStderr because it wasn't actually checked anywhere.
2021-09-08Add missing include in util.ccNaïm Favier
2021-07-30libutil: initialize the base64 decode array only onceAndreas Rammhold
Previously, despite having a boolean that tracked initialization, the decode characters have been "calculated" every single time a base64 string was being decoded. With this change we only initialize the decode array once in a thread-safe manner.
2021-07-24libutil: use uniform initialization in _deletePathAlyssa Ross
Otherwise I get a compiler error when building for NetBSD: src/libutil/util.cc: In function 'void nix::_deletePath(const Path&, uint64_t&)': src/libutil/util.cc:438:17: error: base operand of '->' is not a pointer 438 | AutoCloseFD dirfd(open(dir.c_str(), O_RDONLY)); | ^~~~~ src/libutil/util.cc:439:10: error: 'dirfd' was not declared in this scope 439 | if (!dirfd) { | ^~~~~ src/libutil/util.cc:444:17: error: 'dirfd' was not declared in this scope 444 | _deletePath(dirfd.get(), path, bytesFreed); | ^~~~~
2021-07-20deletePath(): Return ENFILE instead of EBADF when out of file descriptorsEelco Dolstra
Also remove an erroneous comment.
2021-07-02Add $NO_COLOR check to ANSI escape conditionsYestin L. Harrison
2021-07-01Respect TERM=dumb more consistentlyYestin L. Harrison
2021-05-18Fix extra slash in canonPath outputMatthew Bauer
When you have a symlink like: /tmp -> ./private/tmp you need to resolve ./private/tmp relative to /tmp’s dir: ‘/’. Unlike any other path output by dirOf, / ends with a slash. We don’t want trailing slashes here since we will append another slash in the next comoponent, so clear s like we would if it was a symlink to an absoute path. This should fix at least part of the issue in https://github.com/NixOS/nix/issues/4822, will need confirmation that it actually fixes the problem to close though. Introduced in f3f228700a52857fe6e8632df4e935551ea219ff.
2021-04-15Merge pull request #4679 from ony/feature/one-pass-canon-pathEelco Dolstra
Optimize canonPath to resolve relative symlinks in one pass
2021-04-07Restore stack size in child processesEelco Dolstra
Fixes #4673.
2021-04-07restoreSignals() + restoreAffinity() -> restoreProcessContext()Eelco Dolstra
2021-04-07PathSubstitutionGoal: Clean up pipeEelco Dolstra
If there were many top-level goals (which are not destroyed until the very end), commands like $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' \ /run/current-system --no-check-sigs --substitute-on-destination could fail with "Too many open files". So now we do some explicit cleanup from amDone(). It would be cleaner to separate goals from their temporary internal state, but that would be a bigger refactor.
2021-03-31canonPath in one passMykola Orliuk
2021-03-26Fix some typosEelco Dolstra
Fixes #4671.
2021-02-07libutil: EPERM from kill(-1, ...) is fineAlyssa Ross
I tested a trivial program that called kill(-1, SIGKILL), which was run as the only process for an unpriveleged user, on Linux and FreeBSD. On Linux, kill reported success, while on FreeBSD it failed with EPERM. POSIX says: > If pid is -1, sig shall be sent to all processes (excluding an > unspecified set of system processes) for which the process has > permission to send that signal. and > The kill() function is successful if the process has permission to > send sig to any of the processes specified by pid. If kill() fails, > no signal shall be sent. and > [EPERM] > The process does not have permission to send the signal to any > receiving process. My reading of this is that kill(-1, ...) may fail with EPERM when there are no other processes to kill (since the current process is ignored). Since kill(-1, ...) only attempts to kill processes the user has permission to kill, it can't mean that we tried to do something we didn't have permission to kill, so it should be fine to interpret EPERM the same as success here for any POSIX-compliant system. This fixes an issue that Mic92 encountered[1] when he tried to review a Nixpkgs PR on FreeBSD. [1]: https://github.com/NixOS/nixpkgs/pull/81459#issuecomment-606073668
2021-01-21Remove trailing whitespaceEelco Dolstra
2020-12-02read(): Use char * instead of unsigned char *Eelco Dolstra
This gets rid of some pointless casts.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-12-02writeFull/writeFile: Use std::string_viewEelco Dolstra
2020-12-01replaceStrings(): Use std::string_viewEelco Dolstra
2020-11-16filterANSIEscapes(): Handle UTF-8 charactersEelco Dolstra
2020-10-11Split out `commonChildInit`John Ericson
2020-10-09Remove LazyEelco Dolstra
This fixes a crash during startup when compiling Nix as a single compilation unit.
2020-10-09writeFile(): Add error context to writeFull() failureEelco Dolstra
Issue #4092.
2020-10-06Factor out common showBytes()Eelco Dolstra
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 02:43:23 +0000