aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-05-30build-time: remove 20% more by PCH'ing C++ stdlibJade Lovelace
It seems like someone implemented precompiled headers a long time ago and then it never got ported to meson or maybe didn't work at all. This is, however, blessedly easy to simply implement. I went looking for `#define` that could affect the result of precompiling the headers, and as far as I can tell we aren't doing any of that, so this should truly just be free build time savings. Previous state: Compilation (551 times): Parsing (frontend): 1302.1 s Codegen & opts (backend): 956.3 s New state: **** Time summary: Compilation (567 times): Parsing (frontend): 1123.0 s Codegen & opts (backend): 1078.1 s I wonder if the "regression" in codegen time is just doing the PCH operation a few times, because meson does it per-target. Change-Id: I664366b8069bab4851308b3a7571bea97ac64022
2024-05-30shellHook: make it actually runJade Lovelace
When we changed this in I91cb6eb6668f3a8eace36ecbdb01eb367861d77b to not run in nested shells, we didn't predict that `nix develop` would do something ridiculous and append -env to things silently. `nix-shell` of course does not do this, so we need to tolerate both. Change-Id: Ibe7cf546823d7358ebc0414ecbe154e3e3194f45
2024-05-30Merge "Revert "tests/filetransfer: reënable on Darwin"" into mainjade
2024-05-30Revert "tests/filetransfer: reënable on Darwin"jade
This reverts commit 285bc67318e2ee4b69b13eb0b8e7b202fc287c51. Reason for revert: https://git.lix.systems/lix-project/lix/issues/364 For some reason this broke `main` even though the change we are reverting passed CI! Mysterious, haunted, etc. Needs more debugging, let's turn it off for now. Change-Id: Ica4819d61cd35b83eb52985bfcb657e858f025a9
2024-05-30Merge "build: fix static aws-cpp-sdk" into mainQyriad
2024-05-30Merge "tests/filetransfer: reënable on Darwin" into mainjade
2024-05-30Merge "Remove 100s of CPU time (10%) from build times (1465s -> 1302s)" into ↵jade
main
2024-05-30Merge "unix-domain-socket.cc: add comment explaining why ↵jade
bindConnectProcHelper" into main
2024-05-30Merge "release-notes: add missing credits/category to consistent-nix-build ↵Maximilian Bosch
entry" into main
2024-05-30release-notes: add missing credits/category to consistent-nix-build entryMaximilian Bosch
Change-Id: I737422a2ff9d66be30cc432f8c1ddba9b1e71f4f
2024-05-30libstore/filetransfer: remove debug printK900
foo. Change-Id: I7d7db22f68046d2ecf3b594b4ee6fd9c9dac4be1
2024-05-30Merge "libutil/args: warn on unknown settings after parsing all flags" into mainMaximilian Bosch
2024-05-30build: fix static aws-cpp-sdkQyriad
Change-Id: I310830951106f194f6960a6b2d52b5081a7f6156
2024-05-29Remove 100s of CPU time (10%) from build times (1465s -> 1302s)Jade Lovelace
I saw that boost/lexical_cast was costing about 100s in CPU time on our compiles. We can fix this trivially by doing explicit template instantiation in exactly one place and eliminating all other includes of it, which is a code improvement anyway by hiding the boost. Before: ``` lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin Analyzing build trace from 'buildtimeold.bin'... **** Time summary: Compilation (551 times): Parsing (frontend): 1465.3 s Codegen & opts (backend): 1110.9 s <snip> **** Expensive headers: 178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via: 40x: command.hh 5x: command-installable-value.hh 3x: installable-flake.hh 2x: <direct include> 2x: installable-attr-path.hh 176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via: 36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh 12x: globals.hh config.hh experimental-features.hh 11x: file-system.hh file-descriptor.hh 6x: serialise.hh strings.hh 6x: <direct include> 6x: archive.hh serialise.hh strings.hh ... 173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via: 55x: <direct include> 39x: command.hh installable-value.hh installables.hh 7x: libexpr.hh 4x: local-store.hh 4x: command-installable-value.hh installable-value.hh installables.hh 3x: binary-cache-store.hh ... 170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via: 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh 14x: store-api.hh nar-info.hh hash.hh 11x: <direct include> 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh 7x: libexpr.hh value.hh source-path.hh archive.hh 6x: fetchers.hh hash.hh ... 169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via: 40x: command.hh installable-value.hh 5x: command-installable-value.hh installable-value.hh 3x: installable-flake.hh installable-value.hh 2x: <direct include> 1x: installable-derived-path.hh 1x: installable-value.hh ... 159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via: 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh 19x: <direct include> 14x: store-api.hh nar-info.hh hash.hh serialise.hh 11x: serialise.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh 7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh ... 156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via: 42x: <direct include> 7x: command-installable-value.hh 2x: installable-attr-path.hh 150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh 11x: file-system.hh 10x: globals.hh 6x: fetchers.hh 6x: serialise.hh strings.hh error.hh 5x: archive.hh ... 133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via : 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh 19x: file-system.hh 11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh 7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh 6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh ... 132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh 16x: gtest.h 11x: file-system.hh types.hh ref.hh 10x: globals.hh types.hh ref.hh 10x: json.hpp 6x: serialise.hh ... done in 0.6s. ``` After: ``` lix/lix2 » maintainers/buildtime_report.sh build Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'... done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it. Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'... **** Time summary: Compilation (551 times): Parsing (frontend): 1302.1 s Codegen & opts (backend): 956.3 s <snip> **** Expensive headers: 178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via: 36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh 12x: globals.hh config.hh experimental-features.hh 11x: file-system.hh file-descriptor.hh 6x: <direct include> 6x: serialise.hh strings.hh 6x: fetchers.hh hash.hh serialise.hh strings.hh ... 154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via: 40x: command.hh 5x: command-installable-value.hh 3x: installable-flake.hh 2x: <direct include> 2x: installable-attr-path.hh 153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via: 55x: <direct include> 39x: command.hh installable-value.hh installables.hh 7x: libexpr.hh 4x: local-store.hh 4x: command-installable-value.hh installable-value.hh installables.hh 3x: binary-cache-store.hh ... 149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh 11x: file-system.hh 10x: globals.hh 6x: fetchers.hh 6x: serialise.hh strings.hh error.hh 5x: archive.hh ... 144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via: 40x: command.hh installable-value.hh 5x: command-installable-value.hh installable-value.hh 3x: installable-flake.hh installable-value.hh 2x: <direct include> 1x: installable-value.hh 1x: installable-derived-path.hh ... 136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via: 42x: <direct include> 7x: command-installable-value.hh 2x: installable-attr-path.hh 133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh 16x: gtest.h 11x: file-system.hh types.hh ref.hh 10x: globals.hh types.hh ref.hh 10x: json.hpp 6x: serialise.hh ... 89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via: 37x: command.hh installable-value.hh installables.hh 25x: store-api.hh realisation.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh 6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh 6x: libexpr.hh value.hh context.hh 6x: shared.hh ... 87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via: 35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h 12x: regex sstream istream 10x: file-system.hh types.hh ref.hh memory unique_ptr.h 10x: gtest.h memory unique_ptr.h 10x: globals.hh types.hh ref.hh memory unique_ptr.h 6x: fetchers.hh types.hh ref.hh memory unique_ptr.h ... 85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via: 37x: command.hh installable-value.hh installables.hh derived-path.hh 20x: globals.hh 20x: logging.hh 16x: store-api.hh logging.hh 6x: <direct include> 6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh ... done in 0.5s. ``` Change-Id: I27f0a2d566db17832cd9be935f12efe7f95b92d0
2024-05-30package: fix derivation correctness when staticQyriad
Change-Id: I394bb72d9f378cd78acc6cf67a9bb15e342d57c4
2024-05-30package: return from shellHook correctlyQyriad
If our shellHook is being run from a nested nix-shell (see 7a12bc200¹), then (I think) it is run from a bash function due to the nesting, then `return` is correct. If its `eval`'d though, then there isn't really a correct way to early exit. So we can just unconditionally be executed in a function. Basically, we have IIFE at home. [1]: 7a12bc2007accb5022037b5a04b0e5475a8bb409 Change-Id: Iacad25cbbf66cde2911604e6061e56ad6212af7e
2024-05-30package: dequalify {host,build}Platform for convenienceQyriad
Change-Id: I5ffeac894a5bff101683cf3d566c63b478779962
2024-05-29tests/filetransfer: reënable on DarwinJade Lovelace
Since we put __darwinAllowLocalNetworking in our derivation in I752b81c85ebeaab4e582ac01c239d69d65580f37, this stuff will just work fine. I checked our derivation works on the darwin community builder. Change-Id: I40e3a801d6bb38efede79af4aded65c1e1f57cec
2024-05-30libutil/args: warn on unknown settings after parsing all flagsCole Helbling
Upstream change: https://github.com/NixOS/nix/pull/10701 Change-Id: Icf271df57ec529dd8c64667d1ef9f6dbf02d33d3
2024-05-29unix-domain-socket.cc: add comment explaining why bindConnectProcHelperJade Lovelace
We reviewed this code a while ago, and we neglected to get a comment in saying why it's Like This at the time. Let's fix that, since it is code that looks very absurd at first glance. Change-Id: Ib67b49605ef9ef1c84ecda1db16be74fc9105398
2024-05-30Merge "docs: enable non-default TOC folding" into mainQyriad
2024-05-30Merge changes from topic "libutil-split" into mainjade
* changes: util.hh: Delete remaining file and clean up headers util.hh: Move nativeSystem to local-derivation-goal.cc util.hh: Move stuff to types.hh util.cc: Delete remaining file util.{hh,cc}: Move ignoreException to error.{hh,cc} util.{hh,cc}: Split out namespaces.{hh,cc} util.{hh,cc}: Split out users.{hh,cc} util.{hh,cc}: Split out strings.{hh,cc} util.{hh,cc}: Split out unix-domain-socket.{hh,cc} util.{hh,cc}: Split out child.{hh,cc} util.{hh,cc}: Split out current-process.{hh,cc} util.{hh,cc}: Split out processes.{hh,cc} util.{hh,cc}: Split out file-descriptor.{hh,cc} util.{hh,cc}: Split out file-system.{hh,cc} util.{hh,cc}: Split out terminal.{hh,cc} util.{hh,cc}: Split out environment-variables.{hh,cc}
2024-05-30docs: enable non-default TOC foldingQyriad
Folding by default would prevent things like "Ctrl+F for nix-env" from working trivially, but the user should be able to fold if they want to. Change-Id: I5273272289f0f24e1f040c691580acfe33f66bd4
2024-05-29Merge "tests: fix functional-timeout" into mainMario Rodas
2024-05-29libstore: fix http abuses no longer workingeldritch horrors
while refactoring the curl wrapper we inadvertently broken the immutable flake protocol, because the immutable flake protocol accumulates headers across the entire redirect chain instead of using only the headers given in the final response of the chain. this is a problem because Some Known Providers Of Flake Infrastructure set rel=immutable link headers only in the penultimate entry of the redirect chain, and curl does not regard it as worth returning to us via its response header enumeration mechanisms. fixes https://git.lix.systems/lix-project/lix/issues/358 Change-Id: I645c3932b465cde848bd6a3565925a1e3cbcdda0
2024-05-29util.hh: Delete remaining file and clean up headersTom Hubrecht
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29util.hh: Move nativeSystem to local-derivation-goal.ccTom Hubrecht
Change-Id: I74565fbfd3aeedef8f50465808fac712b84e47ad
2024-05-29util.hh: Move stuff to types.hhTom Hubrecht
Change-Id: Ia852306a4b8aac6856dc42bc69e4b58b53a0d67c
2024-05-29util.cc: Delete remaining fileTom Hubrecht
Change-Id: I2b47848904f2ce7bd78b83738e99a4c9da627751
2024-05-29util.{hh,cc}: Move ignoreException to error.{hh,cc}Tom Hubrecht
Change-Id: Iae6464217a55c313a983e5c651b26a4a1e446706
2024-05-29util.{hh,cc}: Split out namespaces.{hh,cc}Tom Hubrecht
Change-Id: I8fd3f3b50c15ede29d489066b4e8d99c2c4636a6
2024-05-29util.{hh,cc}: Split out users.{hh,cc}Tom Hubrecht
Change-Id: I1bd92479a2cb7e5c2c2e1541b80474adb05ea0df
2024-05-29util.{hh,cc}: Split out strings.{hh,cc}Tom Hubrecht
Change-Id: I4f642d1046d56b5db26f1b0296ee16a0e02d444a
2024-05-29util.{hh,cc}: Split out unix-domain-socket.{hh,cc}Tom Hubrecht
Change-Id: I3f9a628e0f8998b6146f5caa8ae9842361a66b8b
2024-05-29util.{hh,cc}: Split out child.{hh,cc}Tom Hubrecht
Change-Id: Iec4824e071f537b17dd62dbb8c01b8eec14e9783
2024-05-29util.{hh,cc}: Split out current-process.{hh,cc}Tom Hubrecht
Change-Id: I77095b9d37e85310075bada7a076ccd482c28e47
2024-05-29util.{hh,cc}: Split out processes.{hh,cc}Tom Hubrecht
Change-Id: I39280dc40ca3f7f9007bc6c898ffcf760e2238b7
2024-05-29util.{hh,cc}: Split out file-descriptor.{hh,cc}Tom Hubrecht
Change-Id: I0dd0f9a9c2003fb887e076127e7f825fd3289c76
2024-05-29util.{hh,cc}: Split out file-system.{hh,cc}Tom Hubrecht
Change-Id: Ifa89a529e7e34e7291eca87d802d2f569cf2493e
2024-05-29util.{hh,cc}: Split out terminal.{hh,cc}Tom Hubrecht
Change-Id: I9de2296b4012d50f540124001d54d6ca3be4c6da
2024-05-28Merge "package: don't run shellHook in nested nix-shells" into mainterru
2024-05-28util.{hh,cc}: Split out environment-variables.{hh,cc}Tom Hubrecht
Change-Id: Icff0aa33fda5147bd5dbe256a0b9d6a6c8a2c3f6
2024-05-28tests: fix functional-timeoutMario Rodas
grepQuietInvert is a typo introduced by c11836126b5. The test functional-timeout was failing silently because Bash considered the command-not-found error as truthy. Change-Id: Ic13829d02ec55d6ecd63a0f4d34ec0d32379609f
2024-05-27package: don't run shellHook in nested nix-shellsstuebinm
If a nested nix-shell is run inside a nix-shell, then the outer shell's shellHook will be passed through and run again, unless the nested shell defines its own. With lix's hook, this can be annoying: forgetting to exit its nix-shell, cd'ing to another repository & entering a nested nix-shell will happily install lix's pre-commit hook in it. This change makes lix's hook return early in such cases. Change-Id: I91cb6eb6668f3a8eace36ecbdb01eb367861d77b
2024-05-27shell.nix: conform to documentationstuebinm
According to doc/manual/src/contributing/hacking.md, all development shells should also be available in classic nix using the -A argument to nix-shell, e.g. nix-shell -A native-clangStdenvPackages This was not actually the case; flake-compat generates attrsets like nix-shell -A devShells.x86_64-linux.native-clangStdenvPackages instead. These are unwieldy to use, so rather than changing the docs, this changes the shell.nix file to result in the documented behavior. Change-Id: I0920ccbdfddacf371f0aeaae7e290db65cf76ee7
2024-05-27Merge "libfetchers: fix URL logging" into mainQyriad
2024-05-26Merge "Revert "bump the extra --version info from info to notice, -vv -> ↵Qyriad
-v"" into main
2024-05-26worker: process timeouts first, and exclusivelyeldritch horrors
check goals for timeouts first, and their activity fds only if no timeout has occurred. checking for timeouts *after* activity sets us up for assertion failures by running multiple build completion notifiers, the first of which will kill/reap the the goal process and consuming the Pid instance. when the second notifier attempts to do the same it will core dump with an assertion failure in Pid and take down not only the single goal, but the entire daemon and all goals it was building. luckily this is rare in practice since it requires a build to both finish and time out at the same time. writing a test for this is not feasible due to how much it relies on scheduling to actually trigger the underlying bug, but on idle machines it can usually be triggered by running multiple sleeping builds with timeout set to the sleep duration and `--keep-going`: nix-build --timeout 10 --builders '' --keep-going -E ' with import <nixpkgs> {}; builtins.genList (i: runCommand "foo-${toString i}" {} "sleep 10") 100 ' Change-Id: I394d36b2e5ffb909cf8a19977d569bbdb71cb67b
2024-05-26docstrings: NixRepl::getDerivationPath: exceptions directly thrownQyriad
getDerivationPath() directly throws nix::Error for invalid derivations Change-Id: I81ead950060b789794fa683b61c6349fece1690d
2024-05-26Revert "bump the extra --version info from info to notice, -vv -> -v"Qyriad
This reverts commit d003dcd7f4ce6b16c3494398a4edd36e668a1b46. d0390b5cf¹ bumped the default verbosity of nix3 commands, making this change obsolete. [1]: d0390b5cf2d232febaa89aa6d8b07c547513a460 Change-Id: I1ddc36ea4514cd8d2855ff334accf6e98c210879