aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-05-31Merge "truncate WAL files on exit" into mainjade
2024-05-31Merge changes Ifcb0d310,I664366b8,Ibe7cf546 into mainjade
* changes: gitignore: delete 90% of it build-time: remove 20% more by PCH'ing C++ stdlib shellHook: make it actually run
2024-05-31Merge "libutil: fix args assert being thrown on Darwin in nix-eval-jobs" ↵jade
into main
2024-05-31Merge "libstore/build: copy ca-certificates too" into mainalois31
2024-05-31truncate WAL files on exitannalee
Fix for https://github.com/NixOS/nix/issues/10300 https://github.com/lix-project/lix/commit/18a26202737a74f216d285d92bd4a84761788026 enabled persistent WAL files that will never get truncated. to fix this, journal_size_limit is set to 2^40, which results in the WAL files being truncated to 0 on exit, as well as limiting the WAL files to 2^40 bytes following a checkpoint. this aligns lix with the nix change: https://github.com/NixOS/nix/pull/10301 https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal https://www.sqlite.org/pragma.html#pragma_journal_size_limit https://github.com/sqlite/sqlite/blob/ed517a708284b6e00b6ae5f1e3f702bbfcbd32ed/src/wal.c#L2518 PR-Link: https://github.com/lix-project/lix/pull/9 Co-Authored-By: paparodeo <170618376+paparodeo@users.noreply.github.com> Change-Id: I90ec1a467c92c582ff8c07dd363a4cf789782214
2024-05-31libutil: fix args assert being thrown on Darwin in nix-eval-jobsJade Lovelace
This is because a dynamic_cast<nix::RootArgs *> of a (n-e-j) MyArgs returns nullptr even though MyArgs has virtual nix::RootArgs as a parent. class MyArgs : virtual public nix::MixEvalArgs, virtual public nix::MixCommonArgs, virtual nix::RootArgs { ... }; So this should work right?? But it does not. We found out that it's caused by -fvisibility=hidden in n-e-j, but honestly this code was bad anyway. The trivial solution is to simply stop relying on RTTI working properly here, which is probably better OO architecture anyway. However, I am not 100% confident *this* is sound, since we have this horrible hierarchy: Args (defines getRoot) / | \ RootArgs MixCommonArgs MixEvalArgs (overrides) I am not confident that this is guaranteed to resolve from Args always in the case of this override. Assertion failed: (res), function getRoot, file src/libutil/args.cc, line 67. 6MyArgsProcess 60503 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert frame #4: 0x0000000100b1a41c liblixutil.dylib`nix::Args::processArgs(std::__1::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, bool) [inlined] nix::Args::getRoot(this=0x00000001000d0688) at args.cc:67:5 [opt] 64 std::cout << typeid(*p).name(); 65 66 auto * res = dynamic_cast<RootArgs *>(p); -> 67 assert(res); 68 return *res; 69 } 70 Target 0: (nix-eval-jobs) stopped. (lldb) p this (MyArgs *) 0x00000001000d0688 (lldb) p *this (nix::Args) { longFlags = size=180 { ... } shortFlags = size=4 { ... } expectedArgs = size=1 { ... } processedArgs = size=0 {} hiddenCategories = size=1 { [0] = "Options to override configuration settings" } parent = nullptr } We also found that if we did this: class [[gnu::visibility("default")]] RootArgs : virtual public Args it would work properly (???!). This is of course, very strange, because objdump -Ct output on liblixexpr.dylib is identical both with and without it. Possibly related: https://www.qt.io/blog/quality-assurance/one-way-dynamic_cast-across-library-boundaries-can-fail-and-how-to-fix-it Fixes: https://git.lix.systems/lix-project/nix-eval-jobs/issues/2 Change-Id: I6b9ed968ed56420a9c4d2dffd18999d78c2761bd
2024-05-31Merge "document context-dependent keywords" into mainterru
2024-05-31libstore/build: copy ca-certificates tooAlois Wohlschlager
In b469c6509ba616da6df8a27e4ccb205a877c66c9, the ca-certificates file was missed. It should be copied too so that we don't end up bind-mounting a broken symlink. Change-Id: Ic9b292d602eb94b0e78f77f2a27a19d24665783c
2024-05-30gitignore: delete 90% of itJade Lovelace
*laughs in meson putting it all in build/* Change-Id: Ifcb0d3104cf9e64c4de91c3a92828899a209d00d
2024-05-30build-time: remove 20% more by PCH'ing C++ stdlibJade Lovelace
It seems like someone implemented precompiled headers a long time ago and then it never got ported to meson or maybe didn't work at all. This is, however, blessedly easy to simply implement. I went looking for `#define` that could affect the result of precompiling the headers, and as far as I can tell we aren't doing any of that, so this should truly just be free build time savings. Previous state: Compilation (551 times): Parsing (frontend): 1302.1 s Codegen & opts (backend): 956.3 s New state: **** Time summary: Compilation (567 times): Parsing (frontend): 1123.0 s Codegen & opts (backend): 1078.1 s I wonder if the "regression" in codegen time is just doing the PCH operation a few times, because meson does it per-target. Change-Id: I664366b8069bab4851308b3a7571bea97ac64022
2024-05-30shellHook: make it actually runJade Lovelace
When we changed this in I91cb6eb6668f3a8eace36ecbdb01eb367861d77b to not run in nested shells, we didn't predict that `nix develop` would do something ridiculous and append -env to things silently. `nix-shell` of course does not do this, so we need to tolerate both. Change-Id: Ibe7cf546823d7358ebc0414ecbe154e3e3194f45
2024-05-30libfetchers: handle nonexistent refs in GitLab repos more gracefullyLinus Heckemann
Before: $ nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent fetching git input 'git+file:///home/linus/projects/lix' fetching gitlab input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent' error: [json.exception.type_error.302] type must be string, but is null After: $ outputs/out/bin/nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent fetching git input 'git+file:///home/linus/projects/lix' fetching gitlab input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent' error: … while updating the lock file of flake 'git+file:///home/linus/projects/lix?ref=refs/heads/fix-gitlab-nonexistent&rev=915f16a619a36237a099b9aa9afed6d14ff613b4' … while updating the flake input 'nixpkgs' … while fetching the input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent' error: No commits returned by GitLab API -- does the ref really exist? Change-Id: Id9bc79d98348500e152ed519bb3ac79a3d15c38d
2024-05-30Merge "Revert "tests/filetransfer: reënable on Darwin"" into mainjade
2024-05-30Revert "tests/filetransfer: reënable on Darwin"jade
This reverts commit 285bc67318e2ee4b69b13eb0b8e7b202fc287c51. Reason for revert: https://git.lix.systems/lix-project/lix/issues/364 For some reason this broke `main` even though the change we are reverting passed CI! Mysterious, haunted, etc. Needs more debugging, let's turn it off for now. Change-Id: Ica4819d61cd35b83eb52985bfcb657e858f025a9
2024-05-30Merge "build: fix static aws-cpp-sdk" into mainQyriad
2024-05-30Merge "tests/filetransfer: reënable on Darwin" into mainjade
2024-05-30Merge "Remove 100s of CPU time (10%) from build times (1465s -> 1302s)" into ↵jade
main
2024-05-30Merge "unix-domain-socket.cc: add comment explaining why ↵jade
bindConnectProcHelper" into main
2024-05-30Merge "release-notes: add missing credits/category to consistent-nix-build ↵Maximilian Bosch
entry" into main
2024-05-30release-notes: add missing credits/category to consistent-nix-build entryMaximilian Bosch
Change-Id: I737422a2ff9d66be30cc432f8c1ddba9b1e71f4f
2024-05-30libstore/filetransfer: remove debug printK900
foo. Change-Id: I7d7db22f68046d2ecf3b594b4ee6fd9c9dac4be1
2024-05-30Merge "libutil/args: warn on unknown settings after parsing all flags" into mainMaximilian Bosch
2024-05-30build: fix static aws-cpp-sdkQyriad
Change-Id: I310830951106f194f6960a6b2d52b5081a7f6156
2024-05-29Remove 100s of CPU time (10%) from build times (1465s -> 1302s)Jade Lovelace
I saw that boost/lexical_cast was costing about 100s in CPU time on our compiles. We can fix this trivially by doing explicit template instantiation in exactly one place and eliminating all other includes of it, which is a code improvement anyway by hiding the boost. Before: ``` lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin Analyzing build trace from 'buildtimeold.bin'... **** Time summary: Compilation (551 times): Parsing (frontend): 1465.3 s Codegen & opts (backend): 1110.9 s <snip> **** Expensive headers: 178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via: 40x: command.hh 5x: command-installable-value.hh 3x: installable-flake.hh 2x: <direct include> 2x: installable-attr-path.hh 176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via: 36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh 12x: globals.hh config.hh experimental-features.hh 11x: file-system.hh file-descriptor.hh 6x: serialise.hh strings.hh 6x: <direct include> 6x: archive.hh serialise.hh strings.hh ... 173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via: 55x: <direct include> 39x: command.hh installable-value.hh installables.hh 7x: libexpr.hh 4x: local-store.hh 4x: command-installable-value.hh installable-value.hh installables.hh 3x: binary-cache-store.hh ... 170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via: 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh 14x: store-api.hh nar-info.hh hash.hh 11x: <direct include> 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh 7x: libexpr.hh value.hh source-path.hh archive.hh 6x: fetchers.hh hash.hh ... 169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via: 40x: command.hh installable-value.hh 5x: command-installable-value.hh installable-value.hh 3x: installable-flake.hh installable-value.hh 2x: <direct include> 1x: installable-derived-path.hh 1x: installable-value.hh ... 159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via: 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh 19x: <direct include> 14x: store-api.hh nar-info.hh hash.hh serialise.hh 11x: serialise.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh 7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh ... 156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via: 42x: <direct include> 7x: command-installable-value.hh 2x: installable-attr-path.hh 150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh 11x: file-system.hh 10x: globals.hh 6x: fetchers.hh 6x: serialise.hh strings.hh error.hh 5x: archive.hh ... 133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via : 37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh 19x: file-system.hh 11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh 7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh 6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh ... 132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh 16x: gtest.h 11x: file-system.hh types.hh ref.hh 10x: globals.hh types.hh ref.hh 10x: json.hpp 6x: serialise.hh ... done in 0.6s. ``` After: ``` lix/lix2 » maintainers/buildtime_report.sh build Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'... done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it. Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'... **** Time summary: Compilation (551 times): Parsing (frontend): 1302.1 s Codegen & opts (backend): 956.3 s <snip> **** Expensive headers: 178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via: 36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh 12x: globals.hh config.hh experimental-features.hh 11x: file-system.hh file-descriptor.hh 6x: <direct include> 6x: serialise.hh strings.hh 6x: fetchers.hh hash.hh serialise.hh strings.hh ... 154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via: 40x: command.hh 5x: command-installable-value.hh 3x: installable-flake.hh 2x: <direct include> 2x: installable-attr-path.hh 153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via: 55x: <direct include> 39x: command.hh installable-value.hh installables.hh 7x: libexpr.hh 4x: local-store.hh 4x: command-installable-value.hh installable-value.hh installables.hh 3x: binary-cache-store.hh ... 149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh 11x: file-system.hh 10x: globals.hh 6x: fetchers.hh 6x: serialise.hh strings.hh error.hh 5x: archive.hh ... 144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via: 40x: command.hh installable-value.hh 5x: command-installable-value.hh installable-value.hh 3x: installable-flake.hh installable-value.hh 2x: <direct include> 1x: installable-value.hh 1x: installable-derived-path.hh ... 136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via: 42x: <direct include> 7x: command-installable-value.hh 2x: installable-attr-path.hh 133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via: 36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh 16x: gtest.h 11x: file-system.hh types.hh ref.hh 10x: globals.hh types.hh ref.hh 10x: json.hpp 6x: serialise.hh ... 89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via: 37x: command.hh installable-value.hh installables.hh 25x: store-api.hh realisation.hh 7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh 6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh 6x: libexpr.hh value.hh context.hh 6x: shared.hh ... 87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via: 35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h 12x: regex sstream istream 10x: file-system.hh types.hh ref.hh memory unique_ptr.h 10x: gtest.h memory unique_ptr.h 10x: globals.hh types.hh ref.hh memory unique_ptr.h 6x: fetchers.hh types.hh ref.hh memory unique_ptr.h ... 85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via: 37x: command.hh installable-value.hh installables.hh derived-path.hh 20x: globals.hh 20x: logging.hh 16x: store-api.hh logging.hh 6x: <direct include> 6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh ... done in 0.5s. ``` Change-Id: I27f0a2d566db17832cd9be935f12efe7f95b92d0
2024-05-30package: fix derivation correctness when staticQyriad
Change-Id: I394bb72d9f378cd78acc6cf67a9bb15e342d57c4
2024-05-30package: return from shellHook correctlyQyriad
If our shellHook is being run from a nested nix-shell (see 7a12bc200¹), then (I think) it is run from a bash function due to the nesting, then `return` is correct. If its `eval`'d though, then there isn't really a correct way to early exit. So we can just unconditionally be executed in a function. Basically, we have IIFE at home. [1]: 7a12bc2007accb5022037b5a04b0e5475a8bb409 Change-Id: Iacad25cbbf66cde2911604e6061e56ad6212af7e
2024-05-30package: dequalify {host,build}Platform for convenienceQyriad
Change-Id: I5ffeac894a5bff101683cf3d566c63b478779962
2024-05-29tests/filetransfer: reënable on DarwinJade Lovelace
Since we put __darwinAllowLocalNetworking in our derivation in I752b81c85ebeaab4e582ac01c239d69d65580f37, this stuff will just work fine. I checked our derivation works on the darwin community builder. Change-Id: I40e3a801d6bb38efede79af4aded65c1e1f57cec
2024-05-30libutil/args: warn on unknown settings after parsing all flagsCole Helbling
Upstream change: https://github.com/NixOS/nix/pull/10701 Change-Id: Icf271df57ec529dd8c64667d1ef9f6dbf02d33d3
2024-05-29unix-domain-socket.cc: add comment explaining why bindConnectProcHelperJade Lovelace
We reviewed this code a while ago, and we neglected to get a comment in saying why it's Like This at the time. Let's fix that, since it is code that looks very absurd at first glance. Change-Id: Ib67b49605ef9ef1c84ecda1db16be74fc9105398
2024-05-30Merge "docs: enable non-default TOC folding" into mainQyriad
2024-05-30Merge changes from topic "libutil-split" into mainjade
* changes: util.hh: Delete remaining file and clean up headers util.hh: Move nativeSystem to local-derivation-goal.cc util.hh: Move stuff to types.hh util.cc: Delete remaining file util.{hh,cc}: Move ignoreException to error.{hh,cc} util.{hh,cc}: Split out namespaces.{hh,cc} util.{hh,cc}: Split out users.{hh,cc} util.{hh,cc}: Split out strings.{hh,cc} util.{hh,cc}: Split out unix-domain-socket.{hh,cc} util.{hh,cc}: Split out child.{hh,cc} util.{hh,cc}: Split out current-process.{hh,cc} util.{hh,cc}: Split out processes.{hh,cc} util.{hh,cc}: Split out file-descriptor.{hh,cc} util.{hh,cc}: Split out file-system.{hh,cc} util.{hh,cc}: Split out terminal.{hh,cc} util.{hh,cc}: Split out environment-variables.{hh,cc}
2024-05-30docs: enable non-default TOC foldingQyriad
Folding by default would prevent things like "Ctrl+F for nix-env" from working trivially, but the user should be able to fold if they want to. Change-Id: I5273272289f0f24e1f040c691580acfe33f66bd4
2024-05-29Merge "tests: fix functional-timeout" into mainMario Rodas
2024-05-29libstore: fix http abuses no longer workingeldritch horrors
while refactoring the curl wrapper we inadvertently broken the immutable flake protocol, because the immutable flake protocol accumulates headers across the entire redirect chain instead of using only the headers given in the final response of the chain. this is a problem because Some Known Providers Of Flake Infrastructure set rel=immutable link headers only in the penultimate entry of the redirect chain, and curl does not regard it as worth returning to us via its response header enumeration mechanisms. fixes https://git.lix.systems/lix-project/lix/issues/358 Change-Id: I645c3932b465cde848bd6a3565925a1e3cbcdda0
2024-05-29util.hh: Delete remaining file and clean up headersTom Hubrecht
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29util.hh: Move nativeSystem to local-derivation-goal.ccTom Hubrecht
Change-Id: I74565fbfd3aeedef8f50465808fac712b84e47ad
2024-05-29util.hh: Move stuff to types.hhTom Hubrecht
Change-Id: Ia852306a4b8aac6856dc42bc69e4b58b53a0d67c
2024-05-29util.cc: Delete remaining fileTom Hubrecht
Change-Id: I2b47848904f2ce7bd78b83738e99a4c9da627751
2024-05-29util.{hh,cc}: Move ignoreException to error.{hh,cc}Tom Hubrecht
Change-Id: Iae6464217a55c313a983e5c651b26a4a1e446706
2024-05-29util.{hh,cc}: Split out namespaces.{hh,cc}Tom Hubrecht
Change-Id: I8fd3f3b50c15ede29d489066b4e8d99c2c4636a6
2024-05-29util.{hh,cc}: Split out users.{hh,cc}Tom Hubrecht
Change-Id: I1bd92479a2cb7e5c2c2e1541b80474adb05ea0df
2024-05-29util.{hh,cc}: Split out strings.{hh,cc}Tom Hubrecht
Change-Id: I4f642d1046d56b5db26f1b0296ee16a0e02d444a
2024-05-29util.{hh,cc}: Split out unix-domain-socket.{hh,cc}Tom Hubrecht
Change-Id: I3f9a628e0f8998b6146f5caa8ae9842361a66b8b
2024-05-29util.{hh,cc}: Split out child.{hh,cc}Tom Hubrecht
Change-Id: Iec4824e071f537b17dd62dbb8c01b8eec14e9783
2024-05-29util.{hh,cc}: Split out current-process.{hh,cc}Tom Hubrecht
Change-Id: I77095b9d37e85310075bada7a076ccd482c28e47
2024-05-29util.{hh,cc}: Split out processes.{hh,cc}Tom Hubrecht
Change-Id: I39280dc40ca3f7f9007bc6c898ffcf760e2238b7
2024-05-29util.{hh,cc}: Split out file-descriptor.{hh,cc}Tom Hubrecht
Change-Id: I0dd0f9a9c2003fb887e076127e7f825fd3289c76
2024-05-29util.{hh,cc}: Split out file-system.{hh,cc}Tom Hubrecht
Change-Id: Ifa89a529e7e34e7291eca87d802d2f569cf2493e
2024-05-29util.{hh,cc}: Split out terminal.{hh,cc}Tom Hubrecht
Change-Id: I9de2296b4012d50f540124001d54d6ca3be4c6da