aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-07-18libexpr/print: pretty-print idempotentlyAlois Wohlschlager
When pretty-printing is enabled, previously an unforced thunk would trigger indentation, even when it subsequently does not evaluate to a nested structure. The resulting output looked inconsistent, and furthermore pretty-printing was not idempotent (since pretty-printing the same value again, which is now fully evaluated, will not trigger indentation). When strict evaluation is enabled, force the item before inspecting its type, so that it is properly known whether it contains a nested structure. Furthermore, there is no need to cause indentation for unforced thunks, since the very next operation will be printing them as `«thunk»`. This is mostly a port of https://github.com/NixOS/nix/pull/11100 , but we only force the item when it's going to be forced anyway due to strict pretty-printing, and a new test was written since the REPL testing framework in Lix is different. Co-Authored-By: Robert Hensing <robert@roberthensing.nl> Change-Id: Ib7560fe531d09e05ca6b2037a523fe21a26d9d58
2024-07-18Merge "doc/manual: clarify documentation related to the `$$` parser bug" ↵alois31
into main
2024-07-17doc/release-notes: link the upcoming release notes againAlois Wohlschlager
The insertion marker comment broke the list into two parts, the first containing only the link to the upcoming release notes and the second the past releases. This confused the generator, leading to the first part being discarded. Indent the marker comment so that it's syntactically part of the preceding item, and in particular doesn't split the list any more. Change-Id: I357c51bb03e4e0d79a76d30158615fd9eda95ea8
2024-07-17doc/release-notes: add date for major releaseRaito Bezarius
Change-Id: I93aab93c069bb3989c3f8d17e0862899e6f76865 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-07-17doc/manual: clarify documentation related to the `$$` parser bugAlois Wohlschlager
Due to a mistake in the grammar, a dollar character implicitly escapes a second dollar character that immediately follows, so that it cannot start an interpolation. Unfortunately, this behaviour has since come to be relied upon, so it cannot be fixed. Furthermore, the documentation on regular strings did not mention this behaviour at all, while in the case of indented strings it was rather implicit. Mention it explicitly in both cases, and describe how an interpolation can follow a dollar character (namely, by escaping that). Since we have to touch that section anyway, state that any character (other than n, r, and t; but notably including `$` even if not succeeded by `{`) can be escaped using a backslash in regular strings. Change-Id: I7e5d68a9a4130eec98ce8218b485168f4b31a677
2024-07-17Merge "tests/functional/repl.sh: actually fail test on wrong stdout" into mainMax Siling
2024-07-17tests/functional/repl.sh: actually fail test on wrong stdoutGoldstein
Previous test implementation assumed that grep supports newlines in patterns. It doesn't, so tests spuriously passed, even though some tests outputs were broken. This patches output (and expected output) before grepping, so there're no newlines in pattern. Change-Id: Ie6561f9f2e18b83d976f162269d20136e2595141
2024-07-16remove boost coroutine referenceseldritch horrors
we no longer need these since sinkToSource and sourceToSink are gone. Change-Id: Ibbf440e2cf71bf3e9f3b833af2d78a21fb1b3193
2024-07-16remove sourceToSink, sinkToSource, and boehm patcheldritch horrors
Change-Id: I1379841299713175d0225b82a67f50660f9eb5e2
2024-07-16libstore: remove remaining sinkToSource useseldritch horrors
Change-Id: Id1ee0d2ad4a3774f4bbb960d76f0f76ac4f3eff9
2024-07-16libstore: remove WriteConn::sink fieldseldritch horrors
we no longer need these since we're no longer using sinks to serialize things. Change-Id: Iffb1a3eab33c83f611c88fa4e8beaa8d5ffa079b
2024-07-16libstore: generatorize protocol serializerseldritch horrors
this is cursed. deeply and profoundly cursed. under NO CIRCUMSTANCES must protocol serializer helpers be applied to temporaries! doing so will inevitably cause dangling references and cause the entire thing to crash. we need to do this even so to get rid of boost coroutines, and likewise to encapsulate the serializers we suffer today at least a little bit to allow a gradual migration to an actual IPC protocol. (this isn't a problem that's unique to generators. c++ coroutines in general cannot safely take references to arbitrary temporaries since c++ does not have a lifetime system that can make this safe. -sigh-) Change-Id: I2921ba451e04d86798752d140885d3c5cc08e146
2024-07-16libstore: remove a sinkToSouce from old daemon protocoleldritch horrors
this doesn't have a test because this code path is only reached by clients that predate 2.4, and we really should not be caring about those any more right now. even the test suite doesn't, and the few tests that might care are disabled because they will not even work Change-Id: Id9eb190065138fedb2c7d90c328ff9eb9d97385b
2024-07-16libstore: rewrite the nar parser as a contents generatoreldritch horrors
this is not completely necessary at this point because the parser right now already returns a generator to pass through all input data it read, but the nar parser *was* very lax and would accept nars that weren't in canonical form (defined as the form dumpPath would return). nar hashing depends on these things, and as such rewriting the parser now allows us to reject non-canonical nars that extract to the same store contents as their canonical counterpart but have different nar hashes despite that. Change-Id: Iccd319e3bd5912d8297014c84c495edc59019bb7
2024-07-15Merge changes I8d87c0e9,I25937702 into mainQyriad
* changes: nix3-upgrade-nix: always use the /new/ nix-env to perform the installation libutil: implement a realPath() utility
2024-07-15libstore: rewrite narFromPath as generatoreldritch horrors
Change-Id: Ifa783c2c65c06ddd1d0212016d5bfd07666ea91c
2024-07-15Merge "use clangStdenv for the default devShell, so we get clangd by ↵lunaphied
default" into main
2024-07-15nix3-upgrade-nix: always use the /new/ nix-env to perform the installationQyriad
Fixes #411. Change-Id: I8d87c0e9295deea26ff33234e15ee33cc68ab303
2024-07-15libutil: implement a realPath() utilityQyriad
Just a wrapper around POSIX realpath(). Change-Id: I2593770285dbae573eace490efce5b272b00b001
2024-07-14Merge "libexpr/eval.cc: remove unnecessary C string conversion" into mainnan-git
2024-07-14libexpr/eval.cc: remove unnecessary C string conversionNaN
Change-Id: I5b7c21df84ff8ff64cf6a1e261fc3729a06bd4f6
2024-07-13nix-support/binary-tarball.nix: Pass through root pathsArtemis Tosini
Passing through root paths allows external programs to see which nix and cacert are in a binary tarball, e.g. to recreate it from substituters Change-Id: I27431134df53bbc6623484f8a0822004b51f7c87
2024-07-13docs: update to define integer overflowJade Lovelace
Change-Id: Ie8a1b31035f2d27a220e5df2e9e178ec3b39ee68
2024-07-13language: cleanly ban integer overflowsJade Lovelace
This also bans various sneaking of negative numbers from the language into unsuspecting builtins as was exposed while auditing the consequences of changing the Nix language integer type to a newtype. It's unlikely that this change comprehensively ensures correctness when passing integers out of the Nix language and we should probably add a checked-narrowing function or something similar, but that's out of scope for the immediate change. During the development of this I found a few fun facts about the language: - You could overflow integers by converting from unsigned JSON values. - You could overflow unsigned integers by converting negative numbers into them when going into Nix config, into fetchTree, and into flake inputs. The flake inputs and Nix config cannot actually be tested properly since they both ban thunks, however, we put in checks anyway because it's possible these could somehow be used to do such shenanigans some other way. Note that Lix has banned Nix language integer overflows since the very first public beta, but threw a SIGILL about them because we run with -fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in production builds. Since the Nix language uses signed integers, overflow was simply undefined behaviour, and since we defined that to trap, it did. Trapping on it was a bad UX, but we didn't even entirely notice that we had done this at all until it was reported as a bug a couple of months later (which is, to be fair, that flag working as intended), and it's got enough production time that, aside from code that is IMHO buggy (and which is, in any case, not in nixpkgs) such as https://git.lix.systems/lix-project/lix/issues/445, we don't think anyone doing anything reasonable actually depends on wrapping overflow. Even for weird use cases such as doing funny bit crimes, it doesn't make sense IMO to have wrapping behaviour, since two's complement arithmetic overflow behaviour is so *aggressively* not what you want for *any* kind of mathematics/algorithms. The Nix language exists for package management, a domain where bit crimes are already only dubiously in scope to begin with, and it makes a lot more sense for that domain for the integers to never lose precision, either by throwing errors if they would, or by being arbitrary-precision. This change will be ported to CppNix as well, to maintain language consistency. Fixes: https://git.lix.systems/lix-project/lix/issues/423 Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
2024-07-13libutil: add checked arithmetic toolsJade Lovelace
This is in preparation for adding checked arithmetic to the evaluator. Change-Id: I6e115ce8f5411feda1706624977a4dcd5efd4d13
2024-07-12use clangStdenv for the default devShell, so we get clangd by defaultLunaphied
The default-stdenv-devShell can always be used with `.#native-stdenvPackages`. Change-Id: I9b3e72210ba5219b6b65c71a2818110769623904
2024-07-12Use std::strong_ordering for version comparisonJade Lovelace
The actual motive here is the avoidance of integer overflow if we were to make these use checked NixInts and retain the subtraction. However, the actual *intent* of this code is a three-way comparison, which can be done with operator<=>, so we should just do *that* instead. Change-Id: I7f9a7da1f3176424b528af6d1b4f1591e4ab26bf
2024-07-11libstore: remove upcast_goaleldritch horrors
upcast_goal was only ever needed to break circular includes, but the same solution that gave us upcast_goal also lets us fully remove it: just upcast goals without a wrapper function, but only in .cc files. Change-Id: I9c71654b2535121459ba7dcfd6c5da5606904032
2024-07-11libstore: turn copyNAR into a generatoreldritch horrors
Change-Id: Id452f6a03faa1037ff13af0f63e32883966ff40d
2024-07-11libstore: turn the NAR parser into a passthrough generatoreldritch horrors
this will let us turn copyNAR into a generator as well, which in turn is necessary to turn the users of copyNAR into generators without resorting to sinkToSource coroutines. currently this uses the SerializingTransform in all cases, even for copyNAR where it is not necessary. should this be a performance problem we can easily swap out the transform for one which does not produce any bytes of its own, but that should not be necessary. Change-Id: I7e685879318fcbb78d8b88abfddd7752360eb0ce
2024-07-11libutil: remove makeDecompressionSinkeldritch horrors
the sole remaining user of this function can use makeDecompressionSource instead, while making the sinkToSource in the caller unnecessary as well Change-Id: I4258227b5dbbb735a75b477d8a57007bfca305e9
2024-07-11libstore: make BinaryCacheStore::getFile return a sourceeldritch horrors
this lets us remove the last true remaining uses of makeDecompressionSink. Change-Id: I146ca2bbe1a9ae9a367117a7b8a304b23a63e5e2
2024-07-11libutil: rewrite RewritingSink as sourceeldritch horrors
the rewriting sink was just broken. when given a rewrite set that contained a key that is also a proper infix of another key it was possible to produce an incorrectly rewritten result if the writer used the wrong block size. fixing this duplicates rewriteStrings, to avoid this we'll rewrite rewriteStrings to use RewritingSource in a new mode that'll allow rewrites we had previously forbidden. Change-Id: I57fa0a9a994e654e11d07172b8e31d15f0b7e8c0
2024-07-10Merge "lix-doc: update dependencies and refactor" into mainlunaphied
2024-07-10Merge changes I81552018,Ieb65c133 into maineldritch horrors
* changes: releng: add releaseTests flake output, test script add aarch64-linux as a cross-build target
2024-07-10devShell: Fix errors with environment variablespiegames
Previous code assumed that $MANPATH always exists Change-Id: I5a4d012045ba6ff9086373b3f46a75d82285d393
2024-07-10Merge "libmain: clear display attributes in the multiline progress bar" into ↵alois31
main
2024-07-10lix-doc: update dependencies and refactorLunaphied
This updates the version of rnix used and refactors the code generally to be more precise and capable in it's identification of both lambdas and determining which documentation comments are attached. Change-Id: Ib0dddabd71f772c95077f9d7654023b37a7a1fd2
2024-07-09releng: add releaseTests flake output, test scripteldritch horrors
this is supposed to be a set of outputs we want to always succeed for releases. sadly we can't add nixos installer tests using lix to these because the nixos test framework does not allow overriding nix in the installer test suites due to unfortunate oversights in the framework. Change-Id: I815520181ccca70a47205d38ba27e73529347f04
2024-07-09add aarch64-linux as a cross-build targeteldritch horrors
we want to be sure we can cross-build to aarch64 for releases, add a target to our crossSystems list to make those cheacks easier to run. Change-Id: Ieb65c1333a5232641ace0ba4d122fc7d528ebc04
2024-07-09Fix dry-run flag for nix-collect-garbageQuantum Jump
`nix-collect-garbage --dry-run` previously elided the entire garbage collection check, meaning that it would just exit the script without printing anything. This change makes the dry run flag instead set the GC action to `gcReturnDead` rather than `gcDeleteDead`, and then continue with the script. So if you set `--dry-run`, it will print the paths it *would* have garbage collected, but not actually delete them. I filed a bug for this: https://git.lix.systems/lix-project/lix/issues/432 but then realised I could give fixing it a go myself. Change-Id: I062dbf1a80bbab192b5fd0b3a453a0b555ad16f2
2024-07-08libexpr: stop lying about DrvInfo's constnessQyriad
DrvInfo's query methods all use mutable fields to cache, but like. that's basically the entire interface for DrvInfo. Not only that, but these formerly-const-marked functions can even throw due to eval errors! Changing this only required removing some `const` markers in nix-env, and changing a single inline `queryInstalled()` call to be an lvalue instead. Change-Id: I796807118f3b35b0e93668b5e28210d9e521b2ae
2024-07-08libmain: clear display attributes in the multiline progress barAlois Wohlschlager
Activities can set display attributes in their log output using the "Select Graphics Rendition" functionality. To prevent interfering with subsequent text displayed, these should be reset after writing the log line. The multiline progress bar neglected to do this, resulting for example in a colorised "building …" header in the next line. Reset the attributes properly, like the standard progress bar already does. Change-Id: I1dc69f4a1d747a76b83e8721a72d9bb0e5554488
2024-07-07libstore: make LocalDerivationGoal::needsHashRewrite virtualArtemis Tosini
This rather simple function existed just to check some flags, but the response varies by platform. This is a perfect case for our subclasses. Change-Id: Ieb1732a8d024019236e0d0028ad843a24ec3dc59
2024-07-06libutil: remove RewritingSink match/size trackingeldritch horrors
size tracking can be done with a LengthSink and a tee. match tracking was defeated by never having done any match tracking, all users would see the same (empty) set of matches at all times. match tracking with bytes offsets alone would not be sufficient in the general case, only because computeHashModulo uses a single rewrite could it have worked. Change-Id: Idb214b5222e0ea24f450f5505712a342b63d7570
2024-07-06libutil: turn HashModuloSink into a free functioneldritch horrors
Change-Id: I5878007502fa68c2816a0f4c61f7d0e60bdde702
2024-07-06libutil: return sources from runProgram2eldritch horrors
this much more closely mimics what is actually happening: we're reading data from somewhere else, actively, rather than passively waiting. with the data flow matching the underlying system interactions better we can remove a few sinkToSource calls that merely exists to undo the mismatch caused by not treating subprocess output as a data source to begin with Change-Id: If4abfc2f8398fb5e88c9b91a8bdefd5504bb2d11
2024-07-05libutil: return a program handle from runProgram2eldritch horrors
this will let us also return a source for the program output later, which will in turn make sinkToSource unnecessary for program output processing. this may also reopen a path for provigin program input, but that still needs a proper async io framework to avoid problems. Change-Id: Iaf93f47db99c38cfaf134bd60ed6a804d7ddf688
2024-07-05libutil: convert readFileSource to a generatoreldritch horrors
Change-Id: I5f92b15fd367d46eb047d74ab6e317b4f51a46d3
2024-07-05libstore: convert dumpPath to a generatoreldritch horrors
Change-Id: Ic4cf5562504aa29130304469936f958c0426e5ef