aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-06-24devShell: guard against running from another directoryJade Lovelace
I was working on nix-eval-jobs with a dev shell with some shenanigans to run against a locally built Lix and it was getting really annoying when `nix develop ../lix#` was messing up my other git repo's hooks. This is a fix via blunt force, but it is at least obvious how it works. Change-Id: Ia29eeb5be57ab6a2c88451c00ea18a51e4dfe65e
2024-06-24justfile: accept extra options to just setup and pass them to mesonJade Lovelace
This lets you get the default options and still be able to add more. Change-Id: Ife32c348b1498ff2ccdddf051a5bba520cfa36f0
2024-06-24Add build-dir setting, clean up default TMPDIR handlingRobert Hensing
This is a squash of upstream PRs #10303, #10312 and #10883. fix: Treat empty TMPDIR as unset Fixes an instance of nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed. ... which I've been getting in one of my shells for some reason. I have yet to find out why TMPDIR was empty, but it's no reason for Nix to break. (cherry picked from commit c3fb2aa1f9d1fa756dac38d3588c836c5a5395dc) fix: Treat empty XDG_RUNTIME_DIR as unset See preceding commit. Not observed in the wild, but is sensible and consistent with TMPDIR behavior. (cherry picked from commit b9e7f5aa2df3f0e223f5c44b8089cbf9b81be691) local-derivation-goal.cc: Reuse defaultTempDir() (cherry picked from commit fd31945742710984de22805ee8d97fbd83c3f8eb) fix: remove usage of XDG_RUNTIME_DIR for TMP (cherry picked from commit 1363f51bcb24ab9948b7b5093490a009947f7453) tests/functional: Add count() (cherry picked from commit 6221770c9de4d28137206bdcd1a67eea12e1e499) Remove uncalled for message (cherry picked from commit b1fe388d33530f0157dcf9f461348b61eda13228) Add build-dir setting (cherry picked from commit 8b16cced18925aa612049d08d5e78eccbf0530e4) Change-Id: Ic7b75ff0b6a3b19e50a4ac8ff2d70f15c683c16a
2024-06-24Merge "Fix build instructions in hacking.md and justfile" into mainDelan Azabani
2024-06-24Merge "Fix compile error under gcc with -Denable-pch-std=false" into mainDelan Azabani
2024-06-24Fix build instructions in hacking.md and justfileDelan Azabani
The stdenv phases don’t actually do anything (at least not anymore), and our justfile doesn’t behave the same as our docs. This patch removes the stdenv phases from the docs, documents our usage of just, and makes `just setup` heed `$mesonFlags`. Fixes #413. Fixes #414. Change-Id: Ieb0b2a8ae420526238b5f9a73d7849ec6919995d
2024-06-24Fix compile error under gcc with -Denable-pch-std=falseDelan Azabani
Following the latest hacking.md currently fails because of a missing include in upstream editline. This patch fixes the build by adding that missing include. Fixes #410. Change-Id: Iefd4cb687ed3da71ccda9fe9624f38e6ef4623e5
2024-06-23libutil: remove runProgram2 stdin functionalityeldritch horrors
this was only used in one place, and that place has been rewritten to use a temporary file instead. keeping this around is not very helpful at this time, and in any case we'd be better off rewriting subprocess handling in rust where we not only have a much safer library for such things but also async frameworks necessary for this easily available. Change-Id: I6f8641b756857c84ae2602cdf41f74ee7a1fda02
2024-06-23libfetchers: write git commit message to tempfileeldritch horrors
we want to remove runProgram's ability to provide stdin to a process because the concurrency issues of handling both stdin and stdout are much more pronounced once runProgram returns not is collected output but a source. this is possible in the current c++ framework, however it isn't necessary in almost all cases (as demonstrated by only this single user existing) and in much better handled with a proper async concurrency model that lets the caller handle both at the same time. Change-Id: I29da1e1ad898d45e2e33a7320b246d5003e7712b
2024-06-23flakes: add --commit-lock-file message testeldritch horrors
we had no test to ensure that we generated a commit message at all? Change-Id: Ic9aa8fde92b83e1ea6f61cd2a21867aa73d4e885
2024-06-23Merge changes I697f4f39,I9f25235d into mainQyriad
* changes: doc-comment Fields for Activity and Result types mildly refactor the renderActivity if hell-chain
2024-06-23Merge "libstore: reduce loglevel of `waiting for a machine to build`" into mainMaximilian Bosch
2024-06-23doc-comment Fields for Activity and Result typesQyriad
The ones we were able to figure out, at least. Change-Id: I697f4f3942e35a7adf1b2a6cc28b3168d1de111c
2024-06-23mildly refactor the renderActivity if hell-chainQyriad
This is primarily for readability, but iwrc chaining std::string's operator+ is also pretty scuffed performance-wise, and this was doing a lot of operator+ chainging. Change-Id: I9f25235df153eb2bbb491f1ff7ebbeed9a8ec985
2024-06-23Merge "libfetchers: make attribute / URL query handling consistent" into mainMaximilian Bosch
2024-06-23libutil: return Pid from startProcess, not pid_teldritch horrors
Change-Id: Icc8a15090c77f54ea7d9220aadedcd4a19922814
2024-06-23libutil: give Pid proper resource semanticseldritch horrors
copy-constructing or assigning from pid_t can easily lead to duplicate Pid instances for the same process if a pid_t was used carelessly, and Pid itself was copy-constructible. both could cause surprising results such as killing processes twice (which could become very problemantic, but luckily modern systems don't reuse PIDs all that quickly), or more than one piece of the code believing it owns a process when neither do Change-Id: Ifea7445f84200b34c1a1d0acc2cdffe0f01e20c6
2024-06-23libutil: make Pid -> pid_t operations expliciteldritch horrors
Change-Id: I3137cc140590001fe7ba542844e735944a0a9255
2024-06-23libutil: add some serialize.hh serializer testseldritch horrors
Change-Id: I0116265a18bc44bba16c07bf419af70d5195f07d
2024-06-23libutil: remove sinkToSource eof callbackeldritch horrors
this is only used in one place, and only to set a nicer error message on EndOfFile. the only caller that actually *catches* this exception should provide an error message in that catch block rather than forcing support for setting error message so deep into the stack. copyStorePath is never called outside of PathSubstitutionGoal anyway, which catches everything. Change-Id: Ifbae8706d781c388737706faf4c8a8b7917ca278
2024-06-23Merge "libstore: Start creating LocalDerivationGoal subclasses" into mainArtemis Tosini
2024-06-23libstore: Start creating LocalDerivationGoal subclassesArtemis Tosini
LocalDerivationGoal includes a large number of low-level sandboxing primitives for Darwin and Linux, intermingled with ifdefs. Start creating platform-specific classes to make it easier to add new platforms and review platform-specific code. This change only creates support infrastructure and moves two function, more functions will be moved in future changes. Change-Id: I9fc29fa2a7345107d4fc96c46fa90b4eabf6bb89
2024-06-22mildly cleanup ExprSelect::evalQyriad
Better variable names, some comments, and a slight logic rearrange. Change-Id: I9685ae252f83217aa85f06432234159c9ad19d1c
2024-06-22doc-comment ExprSelect's fieldsQyriad
Change-Id: I63e79991a4bab93421266785e9258e0f5bb89b8f
2024-06-22libstore: reduce loglevel of `waiting for a machine to build`Maximilian Bosch
This comes quite often when the available job slots on all remote builders are exhausted and this is pretty spammy. This isn't really an issue, but expected behavior. A better way to display this is a nom-like approach where all scheduled builds are shown in a tree and pending builds are being marked as such IMHO. Change-Id: I6bc14e6054f84e3eb0768127b490e263d8cdcf89
2024-06-22libfetchers: make attribute / URL query handling consistentMaximilian Bosch
The original idea was to fix lix#174, but for a user friendly solution, I figured that we'd need more consistency: * Invalid query params will cause an error, just like invalid attributes. This has the following two consequences: * The `?dir=`-param from flakes will be removed before the URL to be fetched is passed to libfetchers. * The tarball fetcher doesn't allow URLs with custom query params anymore. I think this was questionable anyways given that an arbitrary set of query params was silently removed from the URL you wanted to fetch. The correct way is to use an attribute-set with a key `url` that contains the tarball URL to fetch. * Same for the git & mercurial fetchers: in that case it doesn't even matter though: both fetchers added unused query params to the URL that's passed from the input scheme to the fetcher (`url2` in the code). It turns out that this was never used since the query parameters were erased again in `getActualUrl`. * Validation happens for both attributes and URLs. Previously, a lot of fetchers validated e.g. refs/revs only when specified in a URL and the validity of attribute names only in `inputFromAttrs`. Now, all the validation is done in `inputFromAttrs` and `inputFromURL` constructs attributes that will be passed to `inputFromAttrs`. * Accept all attributes as URL query parameters. That also includes lesser used ones such as `narHash`. And "output" attributes like `lastModified`: these could be declared already when declaring inputs as attribute rather than URL. Now the behavior is at least consistent. Personally, I think we should differentiate in the future between "fetched input" (basically the attr-set that ends up in the lock-file) and "unfetched input" earlier: both inputFrom{Attrs,URL} entrypoints are probably OK for unfetched inputs, but for locked/fetched inputs a custom entrypoint should be used. Then, the current entrypoints wouldn't have to allow these attributes anymore. Change-Id: I1be1992249f7af8287cfc37891ab505ddaa2e8cd
2024-06-22libmain: add progress bar with multiple status linesFinn Behrens
Add the log-formats `multiline` and `multiline-with-logs` which offer multiple current active building status lines. Change-Id: Idd8afe62f8591b5d8b70e258c5cefa09be4cab03
2024-06-22Merge "add a basic libmain test for the progress bar rendering" into mainQyriad
2024-06-21Merge "libstore: remove operations that are never called by supported ↵jade
clients" into main
2024-06-20add a basic libmain test for the progress bar renderingQyriad
Hooray for leaky abstraction allowing us to test this particular part of the render pipeline. Change-Id: Ie0f251ff874f63324e6a9c6388b84ec6507eeae2
2024-06-20remove InstallableValueCommand classQyriad
Change-Id: Id12383f4741cba07159712700ebcfbe37e61560c
2024-06-20de-inheritance CmdSearch for InstallableValueCommandQyriad
Change-Id: I125c8cac05c8e924b55e4eb1060496e35ea4e941
2024-06-20de-inheritance CmdEdit for InstallableValueCommandQyriad
Change-Id: If85ea78954a45470b0b25c08dc7d40bfebd53610
2024-06-20de-inheritance CmdRun for InstallableValueCommandQyriad
Change-Id: Ief858c1488197211e2ee8b70aa40ed6c65743558
2024-06-20de-inheritance CmdBundle for InstallableValueCommandQyriad
Change-Id: Icbac4ef927ddcaf0d2a74b376e5a77299529cd34
2024-06-20de-inheritance CmdEval for InstallableValueCommandQyriad
Change-Id: I08b1702310e863d15de26dc838eb0bcb62417c10
2024-06-20use a type alias for ProgressBar's chosen time point typeQyriad
Change-Id: I621a455b1daba806fc498958aee7931fbfc55445
2024-06-20extract ProgressBar declaration into its header fileQyriad
Change-Id: Ica9e2ec41d99eaa196a0d535501edf45c589b2b6
2024-06-20slight cleanup to ProgressBar::getStatus()Qyriad
Binaries were identical before and after this commit on our machine Change-Id: I6f8bfbe3298d6c5f43d5702c7a1e05cb180226cc
2024-06-20Merge "BrotliDecompressionSource: don't bail out too early" into mainIlya K
2024-06-20BrotliDecompressionSource: don't bail out too earlyK900
If we've consumed the entire input, that doesn't actually mean we're done decompressing - there might be more output left. This worked (?) in most cases because the input and output sizes are pretty comparable, but sometimes they're not and then things get very funny. Change-Id: I73435a654a911b8ce25119f713b80706c5783c1b
2024-06-20Merge "libstore: fix queryValidPaths concurrency" into mainjade
2024-06-19libstore: remove operations that are never called by supported clientsJade Lovelace
I did a whole bunch of `git log -S` to find out exactly when all these things were obsoleted and found the commit in which their usage was removed, which I have added in either the error message or a comment. I've also made *some* of the version checks into static asserts for when we update the minimum supported protocol version. In the end this is not a lot of code we are deleting, but it's code that we will never have to support into the future when we build a protocol bridge, which is why I did it. It is not in the support baseline. Change-Id: Iea3c80795c75ea74f328cf7ede7cbedf8c41926b
2024-06-19Merge pull request #10570 from layus/shared_cachesEelco Dolstra
Share evaluation caches across installables Before: $ rm -rf ~/.cache/nix && time -f '%E' nix build --dry-run \ 'nixpkgs#hello' \ 'nixpkgs#clang' \ 'nixpkgs#cargo' \ 'nixpkgs#rustup' \ 'nixpkgs#bear' \ 'nixpkgs#firefox' \ 'nixpkgs#git-revise' \ 'nixpkgs#hyperfine' \ 'nixpkgs#curlie' \ 'nixpkgs#xz' \ 'nixpkgs#ripgrep' 0:03.61 After: $ rm -rf ~/.cache/nix && time -f '%E' nix build --dry-run \ 'nixpkgs#hello' \ 'nixpkgs#clang' \ 'nixpkgs#cargo' \ 'nixpkgs#rustup' \ 'nixpkgs#bear' \ 'nixpkgs#firefox' \ 'nixpkgs#git-revise' \ 'nixpkgs#hyperfine' \ 'nixpkgs#curlie' \ 'nixpkgs#xz' \ 'nixpkgs#ripgrep' 0:01.46 This could probably use a more proper benchmark... Fixes #313 (cherry picked from commit de51e5c335865e3e0a8cccd283fec1a52cce243f) Change-Id: I9350bebd462b6af12c51db5bf432321abfe84a16
2024-06-19filetransfer: return a Source from download()eldritch horrors
without this we will not be able to get rid of makeDecompressionSink, which in turn will be necessary to get rid of sourceToSink (since the libarchive archive wrapper *must* be a Source due to api limitations) Change-Id: Iccd3d333ba2cbcab49cb5a1d3125624de16bce27
2024-06-19libutil: return a source from readFileeldritch horrors
don't consume a sink, return a source instead. the only reason to not do this is a very slight reduction in dynamic allocations, but since we are going to *at least* do disk io that will not be a lot of overhead anyway Change-Id: Iae2f879ec64c3c3ac1d5310eeb6a85e696d4614a
2024-06-19libutil: add makeDecompressionSourceeldritch horrors
Change-Id: Iac7f24d79e24417436b9b5cbefd6af051aeea0a6
2024-06-19libstore: BinaryCacheStore::getFile{ -> Contents}eldritch horrors
if we want have getFile return a source instead of consuming a sink we'll have to disambiguate this overload another way, eg like this. Change-Id: Ia26de2020c309a37e7ccc3775c1ad1f32e0a778b
2024-06-19Merge "libfetchers: represent unfetched submodules consistently" into mainalois31
2024-06-19Merge changes Id0e651e4,I0ed20da8,I76bd6d22,I5d8ffb7b into mainjade
* changes: store: fix null reference from DerivationGoal::waiteeDone libmain: fix UB in verbosity assignment build: make UBSan work :) libexpr: fix accessing uninitialized values and fix pure-eval docs