aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-07-02Merge "Revert "Revert "bump the extra --version info from info to notice, ↵Qyriad
-vv -> -v""" into main
2024-07-02Revert "Revert "bump the extra --version info from info to notice, -vv -> -v""Qyriad
This reverts commit 9f16a20f3d35dbc13ba6c8830745e9fc40d408eb. Since c55e93ca2, the original reason for reverting d003dcd7f no longer applies. Change-Id: If88f8555a6060d0530dcfecdf55afaa40afd1ad9
2024-07-02libmain: better fix for #424, #425eldritch horrors
not printing activities at all when no progress information is available hides *all* progress information from e.g. flake show. this is not ideal and needs to be fixed, but the fix *still* has problems with flake show: in multiline mode we will overwrite all useful flake show output as soon as the progress bar is redrawn. flake show output is also mangled in any number of other situations (like -v being set), so we should probably be not too worried about it and fix progress reporting properly another day Change-Id: I6d39d670e261bbae00560b6a8e15dec8e16b35c4
2024-07-02Merge "tree-wide: unify progress bar inactive and paused states" into mainalois31
2024-07-02Merge "Reject fully-qualified URLs in 'from' argument of `nix registry add`" ↵Delan Azabani
into main
2024-07-01tree-wide: unify progress bar inactive and paused statesAlois Wohlschlager
Previously, the progress bar had two subtly different states in which the bar would not actually render, both with their own shortcomings: inactive (which was irreversible) and paused (reversible, but swallowing logs). Furthermore, there was no way of resetting the statistics, so a very bad solution was implemented (243c0f18dae2a08ea0e46f7ff33277c63f7506d7) that would create a new logger for each line of the repl, leaking the previous one and discarding the value of printBuildLogs. Finally, if stderr was not attached to a TTY, the update thread was started even though the logger was not active, violating the invariant required by the destructor (which is not observed because the logger is leaked). In this commit, the two aforementioned states are unified into a single one, which can be exited again, correctly upholds the invariant that the update thread is only running while the progress bar is active, and does not swallow logs. The latter change in behavior is not expected to be a problems in the rare cases where the paused state was used before, since other loggers (like the simple one) don't exhibit it anyway. The startProgressBar/stopProgressBar API is removed due to being a footgun, and a new method for properly resetting the progress is added. Co-Authored-By: Qyriad <qyriad@qyriad.me> Change-Id: I2b7c3eb17d439cd0c16f7b896cfb61239ac7ff3a
2024-07-01Merge changes Ie29a8a89,I873eedcf into mainjade
* changes: store: delete obsolete lsof-disabling code store: guess the URL of failing fixed-output derivations
2024-06-30enable -Werror=unused-resulteldritch horrors
we do not have any of these warnings appearing at the moment, but it seems like a good idea to enable [[nodiscard]] checking anyway. once we start introducing more functions with must-use conditions we will need such checking, and the rust stdlib has proven them very useful. Change-Id: Ibb6b042ae1ec5f527f8dc2809a7816a4c1548ae2
2024-06-30libexpr/flake: allow automatic rejection of configuration options from flakesAlois Wohlschlager
The `allow-flake-configuration` option allows the user to control whether to accept configuration options supplied by flakes. Unfortunately, setting this to false really meant "ask each time" (with an option to remember the choice for each specific option encountered). Let no mean no, and introduce (and default to) a separate value for the "ask each time" behaviour. Co-Authored-By: Jade Lovelace <lix@jade.fyi> Change-Id: I7ccd67a95bfc92cffc1ebdc972d243f5191cc1b4
2024-06-29Reject fully-qualified URLs in 'from' argument of `nix registry add`Delan Azabani
We previously allowed you to map any flake URL to any other flake URL, including shorthand flakerefs, indirect flake URLs like `flake:nixpkgs`, direct flake URLs like `github:NixOS/nixpkgs`, or local paths. But flake registry entries mapping from direct flake URLs often come from swapping the 'from' and 'to' arguments by accident, and even when created intentionally, they may not actually work correctly. This patch rejects those URLs (and fully-qualified flake: URLs), making it harder to swap the arguments by accident. Fixes #181. Change-Id: I24713643a534166c052719b8770a4edfcfdb8cf3
2024-06-27store: delete obsolete lsof-disabling codeJade Lovelace
Since Ifa0adda7984e, we don't use this code anymore on macOS, so we have no reason to have a knob to disable it anymore. Change-Id: Ie29a8a8978d9aefd4551895f4f9b3cc0827496df
2024-06-27store: guess the URL of failing fixed-output derivationsJade Lovelace
This is a shameless layering violation in favour of UX. It falls back trivially to "unknown", so it's purely a UX feature. Diagnostic sample: ``` error: hash mismatch in fixed-output derivation '/nix/store/sjfw324j4533lwnpmr5z4icpb85r63ai-x1.drv': likely URL: https://meow.puppy.forge/puppy.tar.gz specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= got: sha256-a1Qvp3FOOkWpL9kFHgugU1ok5UtRPSu+NwCZKbbaEro= ``` Change-Id: I873eedcf7984ab23f57a6754be00232b5cb5b02c
2024-06-26Merge "packaging: make pegtl use the __forDefaults mechanism" into mainjade
2024-06-26Merge "doc/hacking: fix up some outdated info about cross, hydra links" into ↵jade
main
2024-06-26libmain: don't print empty lineseldritch horrors
this most notably affects `nix eval`: if there is no progress bar to be shown and no activities going on we should not print anything at all. a progress bar with no activities would print a bunch of terminal escapes *and a space*, which is not helpful in simple cases like nix eval -E 1. notably this does *not* affect nix eval called on non-terminal outputs, but it is slightly confusing nevertheless (and not difficult to avoid). fixes https://git.lix.systems/lix-project/lix/issues/424 Change-Id: Iee793c79ba5a485d6606e0d292ed2eae6dfb7216
2024-06-26Merge changes I476a2516,I8a274227 into mainjade
* changes: doc/hacking: fix internal api docs section to say to enable it doc: Add more about the release note generator
2024-06-26Merge "libmain/progress-bar: move implementation out of the header" into mainalois31
2024-06-26packaging: make pegtl use the __forDefaults mechanismJade Lovelace
This avoids needing to pass it in when callPackage'ing Lix from external code. Change-Id: Ie07e84a151e38614064609a2f6dbff165e193be7
2024-06-25doc/hacking: fix up some outdated info about cross, hydra linksJade Lovelace
We would like to build these with Hydra but we do not currently have a Hydra to build them with conveniently. Change-Id: I0832a33881138dd1caab3805df7ad097db347e62
2024-06-25doc/hacking: fix internal api docs section to say to enable itJade Lovelace
I filed a bug to build these in releng in the future: https://git.lix.systems/lix-project/lix/issues/422 Change-Id: I476a2516cc2be382d4b7c8529a02f9212a78fdb2
2024-06-25doc: Add more about the release note generatorJade Lovelace
Change-Id: I8a274227cb1b05d442d3f644603dd2844ecc9d05
2024-06-25Merge "change shebangs of all .sh scripts to bash" into mainjade
2024-06-25Merge changes If0ddec6b,Iaa63ed18 into mainjade
* changes: Add some release notes for things we did packaging: don't build internal api docs by default in dev shells
2024-06-25Merge ".envrc: remove MAKEFLAGS and use clang environment by default" into mainlunaphied
2024-06-25.envrc: remove MAKEFLAGS and use clang environment by defaultLunaphied
MAKEFLAGS hasn't been relevant since we switched off the Make buildsystem and using the clang environment by default gives you clangd by default which most developers will want. Change-Id: I9c11d0613577047e6c908f049c1ffaca5fb5ff67
2024-06-25libexpr: rewrite the parser with pegtl instead of flex/bisoneldritch horrors
this gives about 20% performance improvements on pure parsing. obviously it will be less on full eval, but depending on how much parsing is to be done (e.g. including hackage-packages.nix or not) it's more like 4%-10%. this has been tested (with thousands of core hours of fuzzing) to ensure that the ASTs produced by the new parser are exactly the same as the old one would have produced. error messages will change (sometimes by a lot) and are not yet perfect, but we would rather leave this as is for later. test results for running only the parser (excluding the variable binding code) in a tight loop with inputs and parameters as given are promising: - 40% faster on lix's package.nix at 10000 iterations - 1.3% faster on nixpkgs all-packages.nix at 1000 iterations - equivalent on all of nixpkgs concatenated at 100 iterations (excluding invalid files, each file surrounded with parens) more realistic benchmarks are somewhere in between the extremes, parsing once again getting the largest uplift. other realistic workloads improve by a few percentage points as well, notably system builds are 4% faster. Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json) old/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix mean: 0.408s ± 0.025s user: 0.355s | system: 0.033s median: 0.389s range: 0.388s ... 0.442s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix mean: 0.332s ± 0.024s user: 0.279s | system: 0.033s median: 0.314s range: 0.313s ... 0.361s relative: 0.814 --- old/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 6.133s ± 0.022s user: 5.395s | system: 0.437s median: 6.128s range: 6.099s ... 6.183s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 5.925s ± 0.025s user: 5.176s | system: 0.456s median: 5.934s range: 5.861s ... 5.943s relative: 0.966 --- GC_INITIAL_HEAP_SIZE=10g old/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 4.503s ± 0.027s user: 3.731s | system: 0.547s median: 4.499s range: 4.478s ... 4.541s relative: 1 GC_INITIAL_HEAP_SIZE=10g new/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 4.285s ± 0.031s user: 3.504s | system: 0.571s median: 4.281s range: 4.221s ... 4.328s relative: 0.951 --- old/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello mean: 16.475s ± 0.07s user: 14.088s | system: 1.572s median: 16.495s range: 16.351s ... 16.536s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello mean: 15.973s ± 0.013s user: 13.558s | system: 1.615s median: 15.973s range: 15.946s ... 15.99s relative: 0.97 --- Change-Id: Ie66ec2d045dec964632c6541e25f8f0797319ee2
2024-06-25Merge "Revert "libfetchers: make attribute / URL query handling consistent"" ↵jade
into main
2024-06-24Add some release notes for things we didJade Lovelace
Change-Id: If0ddec6b64a43c3d8f6cae39e0292863f3b49401
2024-06-24packaging: don't build internal api docs by default in dev shellsJade Lovelace
These are totally available and you can just turn them on, but they have very bad dependency tracking and thus bloat incremental change times, which is not really ok. Change-Id: Iaa63ed18a789e74fcb757248cd24c3b194afcc80
2024-06-24Revert "libfetchers: make attribute / URL query handling consistent"jade
This reverts commit 35eec921af1043fc6322edc0ad88c872d41623b8. Reason for revert: Regressed nix-eval-jobs, and it appears to be this change is buggy/missing a case. It just needs another pass. Code causing the problem in n-e-j, when invoked with `nix-eval-jobs --flake '.#hydraJobs'`: ``` n-e-j/tests/assets » ../../build/src/nix-eval-jobs --meta --workers 1 --flake .#hydraJobs warning: unknown setting 'trusted-users' warning: `--gc-roots-dir' not specified error: unsupported Git input attribute 'dir' error: worker error: error: unsupported Git input attribute 'dir' ``` ``` nix::Value *vRoot = [&]() { if (args.flake) { auto [flakeRef, fragment, outputSpec] = nix::parseFlakeRefWithFragmentAndExtendedOutputsSpec( args.releaseExpr, nix::absPath(".")); nix::InstallableFlake flake{ {}, state, std::move(flakeRef), fragment, outputSpec, {}, {}, args.lockFlags}; return flake.toValue(*state).first; } else { return releaseExprTopLevelValue(*state, autoArgs, args); } }(); ``` Inspecting the program behaviour reveals that `dir` was in fact set in the URL going into the fetcher. This is in turn because unlike in the case changed in this commit, it was not erased before handing it to libfetchers, which is probably just a mistake. ``` (rr) up 3 0x00007ffff60262ae in nix::fetchers::Input::fromURL (url=..., requireTree=requireTree@entry=true) at src/libfetchers/fetchers.cc:39 warning: Source file is more recent than executable. 39 auto res = inputScheme->inputFromURL(url, requireTree); (rr) p url $1 = (const nix::ParsedURL &) @0x7fffdc874190: {url = "git+file:///home/jade/lix/nix-eval-jobs", base = "git+file:///home/jade/lix/nix-eval-jobs", scheme = "git+file", authority = std::optional<std::string> = {[contained value] = ""}, path = "/home/jade/lix/nix-eval-jobs", query = std::map with 1 element = {["dir"] = "tests/assets"}, fragment = ""} (rr) up 4 0x00007ffff789d904 in nix::parseFlakeRefWithFragment (url=".#hydraJobs", baseDir=std::optional<std::string> = {...}, allowMissing=allowMissing@entry=false, isFlake=isFlake@entry=true) at src/libexpr/flake/flakeref.cc:179 warning: Source file is more recent than executable. 179 FlakeRef(Input::fromURL(parsedURL, isFlake), getOr(parsedURL.query, "dir", "")), (rr) p parsedURL $2 = {url = "git+file:///home/jade/lix/nix-eval-jobs", base = "git+file:///home/jade/lix/nix-eval-jobs", scheme = "git+file", authority = std::optional<std::string> = {[contained value] = ""}, path = "/home/jade/lix/nix-eval-jobs", query = std::map with 1 element = { ["dir"] = "tests/assets"}, fragment = ""} (rr) list 174 175 if (pathExists(flakeRoot + "/.git/shallow")) 176 parsedURL.query.insert_or_assign("shallow", "1"); 177 178 return std::make_pair( 179 FlakeRef(Input::fromURL(parsedURL, isFlake), getOr(parsedURL.query, "dir", "")), 180 fragment); 181 } ``` Change-Id: Ib55a882eaeb3e59228857761dc1e3b2e366b0f5e
2024-06-24change shebangs of all .sh scripts to bashvigress8
On operating systems where /bin/sh is not Bash, some scripts are invalid because of bashisms, and building Lix fails with errors like this: `render-manpage.sh: 3: set: Illegal option -o pipefail` This modifies all scripts that use a `/bin/sh` shebang to `/usr/bin/env bash`, including currently POSIX-compliant ones, to prevent any future confusion. Change-Id: Ia074cc6db42d40fc59a63726f6194ea0149ea5e0
2024-06-24devShell: guard against running from another directoryJade Lovelace
I was working on nix-eval-jobs with a dev shell with some shenanigans to run against a locally built Lix and it was getting really annoying when `nix develop ../lix#` was messing up my other git repo's hooks. This is a fix via blunt force, but it is at least obvious how it works. Change-Id: Ia29eeb5be57ab6a2c88451c00ea18a51e4dfe65e
2024-06-24justfile: accept extra options to just setup and pass them to mesonJade Lovelace
This lets you get the default options and still be able to add more. Change-Id: Ife32c348b1498ff2ccdddf051a5bba520cfa36f0
2024-06-24libmain/progress-bar: move implementation out of the headerAlois Wohlschlager
Change-Id: Ib4b42ebea290ee575294df6b2f17a38a5d850b80
2024-06-24Add build-dir setting, clean up default TMPDIR handlingRobert Hensing
This is a squash of upstream PRs #10303, #10312 and #10883. fix: Treat empty TMPDIR as unset Fixes an instance of nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed. ... which I've been getting in one of my shells for some reason. I have yet to find out why TMPDIR was empty, but it's no reason for Nix to break. (cherry picked from commit c3fb2aa1f9d1fa756dac38d3588c836c5a5395dc) fix: Treat empty XDG_RUNTIME_DIR as unset See preceding commit. Not observed in the wild, but is sensible and consistent with TMPDIR behavior. (cherry picked from commit b9e7f5aa2df3f0e223f5c44b8089cbf9b81be691) local-derivation-goal.cc: Reuse defaultTempDir() (cherry picked from commit fd31945742710984de22805ee8d97fbd83c3f8eb) fix: remove usage of XDG_RUNTIME_DIR for TMP (cherry picked from commit 1363f51bcb24ab9948b7b5093490a009947f7453) tests/functional: Add count() (cherry picked from commit 6221770c9de4d28137206bdcd1a67eea12e1e499) Remove uncalled for message (cherry picked from commit b1fe388d33530f0157dcf9f461348b61eda13228) Add build-dir setting (cherry picked from commit 8b16cced18925aa612049d08d5e78eccbf0530e4) Change-Id: Ic7b75ff0b6a3b19e50a4ac8ff2d70f15c683c16a
2024-06-24Merge "Fix build instructions in hacking.md and justfile" into mainDelan Azabani
2024-06-24Merge "Fix compile error under gcc with -Denable-pch-std=false" into mainDelan Azabani
2024-06-24Fix build instructions in hacking.md and justfileDelan Azabani
The stdenv phases don’t actually do anything (at least not anymore), and our justfile doesn’t behave the same as our docs. This patch removes the stdenv phases from the docs, documents our usage of just, and makes `just setup` heed `$mesonFlags`. Fixes #413. Fixes #414. Change-Id: Ieb0b2a8ae420526238b5f9a73d7849ec6919995d
2024-06-24Fix compile error under gcc with -Denable-pch-std=falseDelan Azabani
Following the latest hacking.md currently fails because of a missing include in upstream editline. This patch fixes the build by adding that missing include. Fixes #410. Change-Id: Iefd4cb687ed3da71ccda9fe9624f38e6ef4623e5
2024-06-23libutil: remove runProgram2 stdin functionalityeldritch horrors
this was only used in one place, and that place has been rewritten to use a temporary file instead. keeping this around is not very helpful at this time, and in any case we'd be better off rewriting subprocess handling in rust where we not only have a much safer library for such things but also async frameworks necessary for this easily available. Change-Id: I6f8641b756857c84ae2602cdf41f74ee7a1fda02
2024-06-23libfetchers: write git commit message to tempfileeldritch horrors
we want to remove runProgram's ability to provide stdin to a process because the concurrency issues of handling both stdin and stdout are much more pronounced once runProgram returns not is collected output but a source. this is possible in the current c++ framework, however it isn't necessary in almost all cases (as demonstrated by only this single user existing) and in much better handled with a proper async concurrency model that lets the caller handle both at the same time. Change-Id: I29da1e1ad898d45e2e33a7320b246d5003e7712b
2024-06-23flakes: add --commit-lock-file message testeldritch horrors
we had no test to ensure that we generated a commit message at all? Change-Id: Ic9aa8fde92b83e1ea6f61cd2a21867aa73d4e885
2024-06-23Merge changes I697f4f39,I9f25235d into mainQyriad
* changes: doc-comment Fields for Activity and Result types mildly refactor the renderActivity if hell-chain
2024-06-23Merge "libstore: reduce loglevel of `waiting for a machine to build`" into mainMaximilian Bosch
2024-06-23doc-comment Fields for Activity and Result typesQyriad
The ones we were able to figure out, at least. Change-Id: I697f4f3942e35a7adf1b2a6cc28b3168d1de111c
2024-06-23mildly refactor the renderActivity if hell-chainQyriad
This is primarily for readability, but iwrc chaining std::string's operator+ is also pretty scuffed performance-wise, and this was doing a lot of operator+ chainging. Change-Id: I9f25235df153eb2bbb491f1ff7ebbeed9a8ec985
2024-06-23Merge "libfetchers: make attribute / URL query handling consistent" into mainMaximilian Bosch
2024-06-23libutil: return Pid from startProcess, not pid_teldritch horrors
Change-Id: Icc8a15090c77f54ea7d9220aadedcd4a19922814
2024-06-23libutil: give Pid proper resource semanticseldritch horrors
copy-constructing or assigning from pid_t can easily lead to duplicate Pid instances for the same process if a pid_t was used carelessly, and Pid itself was copy-constructible. both could cause surprising results such as killing processes twice (which could become very problemantic, but luckily modern systems don't reuse PIDs all that quickly), or more than one piece of the code believing it owns a process when neither do Change-Id: Ifea7445f84200b34c1a1d0acc2cdffe0f01e20c6
2024-06-23libutil: make Pid -> pid_t operations expliciteldritch horrors
Change-Id: I3137cc140590001fe7ba542844e735944a0a9255