aboutsummaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-06-19libutil: BrotliDecompression{Sink -> Source}eldritch horrors
Change-Id: I9579dd08f7bd0f927bde9d3128515b0cee15f320
2024-06-18filetransfer: {up,down}load -> transfereldritch horrors
even the transfer function is not all that necessary since there aren't that many users, but we'll keep it for now. we could've kept both names but we also kind of want to use `download` for something else very soon Change-Id: I005e403ee59de433e139e37aa2045c26a523ccbf
2024-06-18libstore: fix queryValidPaths concurrencyJade Lovelace
The lock usage was obviously wrong so it was entirely serialized. This has the predicted speedups, the only question is whether it is sound because it's exposing a bunch of new code to actual concurrency. I did audit all the stores' queryPathInfoUncached implementations and they all look *intended* to be thread safe, but whether that is actually sound or not: lol lmao. I am highly confident in the s3 one because it is calling s3 sdk methods that are thread safe and has no actual state. Others are using Pool and look to be *supposed* to be thread safe, but unsure if they actually are. Change-Id: I0369152a510e878b5ac56c9ac956a98d48cd5fef
2024-06-18diff-closures: fix a use after freeJade Lovelace
Found by looking for interesting asan reports from the test suite. What happened here is that name got overwritten, but it was what actually held the backing memory for the thing it got overwritten by, which was a by-reference value coming out of std::regex. Due to absurd reasons I cannot seem to use a string_view iterator here, so I just copy the string with a longer lifetime instead. idk lol ==3796364==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000014c61 at pc 0x74843523bf1d bp 0x7ffc68351330 sp 0x7ffc68350af0 READ of size 3 at 0x503000014c61 thread T0 0 0x74843523bf1c in __asan_memcpy (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x159f1c) 1 0x6403cf6cbff4 in std::char_traits<char>::copy(char*, char const*, unsigned long) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/char_traits.h:445:33 <...> 7 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::str() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:966:6 8 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:955:16 9 0x6403cf6cbff4 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:37:26 10 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25 11 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17 <...> 0x503000014c61 is located 17 bytes inside of 21-byte region [0x503000014c50,0x503000014c65) freed by thread T0 here: 0 0x748435250470 in operator delete(void*) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16e470) <...> 6 0x6403cf6cbda2 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:792:9 7 0x6403cf6cbda2 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:36:13 8 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25 <...> previously allocated by thread T0 here: 0 0x74843524fa38 in operator new(unsigned long) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16da38) <...> 9 0x6403cf6cb68c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:784:4 10 0x6403cf6cb68c in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:33:21 11 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25 12 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17 <...> Change-Id: I9c408cf2a3d3155f9f7b2ad4848ee6c741331db0
2024-06-18store-api: fix/clarify capture lifetimes in copyPathsJade Lovelace
This seems to fix a use of stack after return. Change-Id: If690a6defb9a3225684685132cf78b227e271447
2024-06-18libstore: work around aws sdk log spam at debug levelJade Lovelace
aws-sdk-cpp spams logs about sending TLS data in the otherwise rather helpful debug logs. I've filed a PR upstream to stop it, but for now we can just fix their verbosity ourselves. Upstream-PR: https://github.com/aws/aws-sdk-cpp/pull/3003 Change-Id: I0c41a50d5f5958106836d6345843f4b05b9c8981
2024-06-18s3: delete obsolete ifdefsJade Lovelace
The versions checked for are so old that we can just drop support. Change-Id: Ib9cf136d1cb9a4a91a6613102c4fd15e1190363b
2024-06-18libutil: tidy Sync and fix its move constructorJade Lovelace
There was a previously-unused move constructor that just called abort, which makes no sense since it ought to just be `= delete` if you don't want one (commit history says it was Eelco doing an optimistic performance optimisation in 2016, so it probably would not pass review today). However, a Lock has some great reasons to be moved! You might need to unlock it early, for instance, or give it to someone else. So we change the move constructor to instead hollow out the moved-from object and make it unusable. Change-Id: Iff2a4c2f7ebd0a558c4866d4dfe526bc8558bed7
2024-06-18libfetchers: represent unfetched submodules consistentlyAlois Wohlschlager
Unfetched submodules are included as empty directories in archives, so they end up as such in the store when fetched in clean mode. Make sure the same happens in dirty mode too. Fortunately, they are already correctly represented in the ls-files output, so we just need to make sure to include the empty directory in our filter. Fixes: https://github.com/NixOS/nix/issues/6247 Change-Id: I60d06ff360cfa305d081b920838c893c06da801c
2024-06-17Merge changes from topic "protocol" into mainjade
* changes: libstore client: remove remaining dead code libstore: refuse to serialise ancient protocols libstore client: remove support for <2.3 clients libstore daemon: remove very old protocol support (<2.3) Delete old ValidPathInfo test, fix UnkeyedValidPathInfo Set up minimum protocol version
2024-06-17libexpr: add expr memory managementeldritch horrors
with the prepatory work done this mostly means turning plain pointers into unique_ptrs, with all the associated churn that necessitates. we might want to change some of these to box_ptrs at some point as well, but that would be a semantic change that isn't fully appropriate yet. Change-Id: I0c238c118617420650432f4ed45569baa3e3f413
2024-06-17libexpr: pass Exprs as references, not pointerseldritch horrors
almost all places where Exprs are passed as pointers expect the pointers to be non-null. pass them as references to encode this constraint in the type system as well (and also communicate that Exprs must not be freed). Change-Id: Ia98f166fec3c23151f906e13acb4a0954a5980a2
2024-06-17libexpr: store ExprConcatStrings elements as direct vectoreldritch horrors
storing a pointer only adds an unnecessary indirection at runtime. Change-Id: If06dd05effdf1ccb0df0873580f50c775608925d
2024-06-17libexpr: don't immediately throw parser errorseldritch horrors
now that destructors are hooked up we want to give the C skeleton every real chance to actually run them. since bison does not call destructors on values that have been passed to semantic actions even when an action causes an abort we will also have to delete some things manually still. Change-Id: Ia22bdaa9e969b74e17a6c496e35e6c2d86b7d750
2024-06-17libexpr: hook up bison destructors for state objectseldritch horrors
this doesn't help much yet since the state objects themselves also leak all memory they are given, but it is a first necessary step to properly managing parser memory. notably we have to clear $$ when returning from the parser since even the start symbol is subject to automatic deletion by the bison-generated parser before returning control to the call site Change-Id: I80245b0c747308e80923e7f18ce4e1a4898f93b0
2024-06-17mini-refactor "lambda.name or anonymous lambda" logicQyriad
Change-Id: I08d39c4ad5b967de526c0d5c5e6299256c7967f3
2024-06-17repl: implement tab completing :colon commandsQyriad
This uses a minor hack in which we check the rl_line_buffer global variable to workaround editline not including the colon in its completion callback. Fixes #361 Change-Id: Id159d209c537443ef5e37a975982e8e12ce1f486
2024-06-16libstore client: remove remaining dead codeJade Lovelace
Change-Id: I1764b3878439ff7b20ff64bd4efcf03070bb0e5e
2024-06-16libstore: refuse to serialise ancient protocolsJade Lovelace
We don't want to deal with these at all, let's stop doing so. (marking this one as the fix commit since its immediate predecessors aren't the complete fix) Fixes: https://git.lix.systems/lix-project/lix/issues/325 Change-Id: Ieea1b0b8ac0f903d1e24e5b3e63cfe12eeec119d
2024-06-16libstore client: remove support for <2.3 clientsJade Lovelace
Change-Id: I71c2e8ca644b6187e0084f35e82f3316c9d425b0
2024-06-16libstore daemon: remove very old protocol support (<2.3)Jade Lovelace
Change-Id: Ic05f478a659c199a66fe78ae05d357d317ac41b0
2024-06-16Set up minimum protocol versionJade Lovelace
Change-Id: Ibb931109a8328cfb22964542ab53644cc4181f9e
2024-06-17Merge changes I9cf007c8,I9b9ba058 into mainjade
* changes: releng: fix broken manifest from 2.90-rc1 Change the sqlite missing valid path message to say it's the db
2024-06-16Merge changes I81e76796,Iba319126 into mainArtemis Tosini
* changes: libutil: Set boost defines for FreeBSD meson.build: Allow undefined symbols on FreeBSD
2024-06-15Change the sqlite missing valid path message to say it's the dbJade Lovelace
I meant to edit https://gerrit.lix.systems/c/lix/+/1161 but apparently clicked the wrong button somehow. Oops. Change-Id: I9b9ba058ec9206d3c8abe125d91dc554cced52fe
2024-06-16Merge changes I07d2da41,I864d7340,I86612c64 into mainjulia
* changes: Change error messages about 'invalid paths' to 'path does not exist'. Add a clearer error message for InvalidPathError during evaluation Harmonise the Store::queryPathInfoUncached interface
2024-06-16Merge "docs: expand importNative/exec example (#10803)" into mainjade
2024-06-16Change error messages about 'invalid paths' to 'path does not exist'.julia
Fixes #270. Change-Id: I07d2da41498cfdf324a03af40533044d58c97c7e
2024-06-16Add a clearer error message for InvalidPathError during evaluationjulia
Part of #270, #271 Change-Id: I864d7340f26d3c0f9c45db7b6b545face38d8294
2024-06-16Harmonise the Store::queryPathInfoUncached interfacejulia
This: - Consistently returns `nullptr` for a non-existent store path, instead of a mix of `nullptr` and throwing exceptions. - If a store returns "bad" store paths in response to a request (e.g. incorrect hash or name), don't cache this result. This removes some duplication of code at the cache-access layer of queryPathInfo() checking this, and ­allows us to provide more specific errors. Part of #270. Change-Id: I86612c6499b1a37ab872c712c2304d6a3ff19edb
2024-06-16libutil: Set boost defines for FreeBSDArtemis Tosini
FreeBSD uses libunwind unwind.h, which does not require `_GNU_SOURCE` to expose `_Unwind_Backtrace`. Tell Boost that. Change-Id: I81e767967b1458118b86d212b5552d4d0a1200d9
2024-06-12doc: rewrite the multi-user documentation to actually talk about securityJade Lovelace
It's in the security section, and it was totally outdated anyway. I took the opportunity to write down the stuff we already believed. Change-Id: I73e62ae85a82dad13ef846e31f377c3efce13cb0
2024-06-12Misc workaround removals since 24.05 upgradeJade Lovelace
Change-Id: I9491b103333cb0e25c245199e88365ded7800d2e
2024-06-06build: expose option to enable or disable precompiled std headersQyriad
They are enabled by default, and Meson will also prints whether or not they're enabled at the bottom at the end of configuration. Change-Id: I48db238510bf9e74340b86f243f4bbe360794281
2024-06-06Merge "libstore/filetransfer: fix no-s3 build" into mainjade
2024-06-05libstore/filetransfer: fix no-s3 buildLinus Heckemann
Fixes a compiler error that looks like: error: could not convert '[...]' from 'future<void>' to 'future<nix::FileTransferResult>' Change-Id: I4aeadfeba0dadfdf133f25e6abce90ede7a86ca6
2024-06-03Show message about `--update-input` being replaced by `nix flake update`Nikodem Rabuliński
Fixes: https://git.lix.systems/lix-project/lix/issues/283 Change-Id: I6ee23874cb09f51d788521273076a25ba8764859
2024-06-03Accept multiple arguments to `nix flake update`Nikodem Rabuliński
Fixes: https://git.lix.systems/lix-project/lix/issues/194 Change-Id: Ia7bd4f7640384be9827dbb7e2c594f0aa5f1aff8
2024-06-01Revert "nix3: always use the same verbosity default (info)"Qyriad
This reverts commit d0390b5cf2d232febaa89aa6d8b07c547513a460. Other parts of the codebase will need to be adjusted in response to a default verbosity change. Let's just push this to after 2.90. Fixes #362. Fixes #367. Change-Id: I04648473579146851bda41d764adc1ef954c355d
2024-06-01Merge "build: fix static linking with a hack" into mainQyriad
2024-06-01chore: rebrand Nix to Lix when it makes senseRaito Bezarius
Here's my guide so far: $ rg '((?!(recursive).*) Nix (?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))' -g '!doc/' --pcre2 All items from this query have been tackled. For the documentation side: that's for https://git.lix.systems/lix-project/lix/issues/162. Additionally, all remaining references to github.com/NixOS/nix which were not relevant were also replaced. Fixes: https://git.lix.systems/lix-project/lix/issues/148. Fixes: https://git.lix.systems/lix-project/lix/issues/162. Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-06-01Merge "libfetchers: allow fetching gitlab refs with >1 commit" into mainLinus Heckemann
2024-05-31build: fix static linking with a hackQyriad
This causes libstore, libexpr, libfetchers, and libutil to be linked with -Wl,--whole-archive to executables, when building statically. libstore for the store backends, libexpr for the primops, libfetchers for the fetcher backends I assume(?), and libutil for the nix::logger initializer (which notably shows in pre-main constructors when HOME is not owned by the user. cursed.). This workaround should be removed when #359 is fixed. Fixes #306. Change-Id: Ie9ef0154e09a6ed97920ee8ab23810ca5e2de84c
2024-05-31Merge "build-remote: truncate+hash store URI used in lockfile paths" into mainjade
2024-05-31Merge "truncate WAL files on exit" into mainjade
2024-05-31Merge changes Ifcb0d310,I664366b8,Ibe7cf546 into mainjade
* changes: gitignore: delete 90% of it build-time: remove 20% more by PCH'ing C++ stdlib shellHook: make it actually run
2024-05-31libfetchers: allow fetching gitlab refs with >1 commitLinus Heckemann
Change-Id: I945c4c5512def9eff728bb67fe3c03ae17f99d6d
2024-05-31Merge "libutil: fix args assert being thrown on Darwin in nix-eval-jobs" ↵jade
into main
2024-05-31truncate WAL files on exitannalee
Fix for https://github.com/NixOS/nix/issues/10300 https://github.com/lix-project/lix/commit/18a26202737a74f216d285d92bd4a84761788026 enabled persistent WAL files that will never get truncated. to fix this, journal_size_limit is set to 2^40, which results in the WAL files being truncated to 0 on exit, as well as limiting the WAL files to 2^40 bytes following a checkpoint. this aligns lix with the nix change: https://github.com/NixOS/nix/pull/10301 https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal https://www.sqlite.org/pragma.html#pragma_journal_size_limit https://github.com/sqlite/sqlite/blob/ed517a708284b6e00b6ae5f1e3f702bbfcbd32ed/src/wal.c#L2518 PR-Link: https://github.com/lix-project/lix/pull/9 Co-Authored-By: paparodeo <170618376+paparodeo@users.noreply.github.com> Change-Id: I90ec1a467c92c582ff8c07dd363a4cf789782214
2024-05-31build-remote: truncate+hash store URI used in lockfile pathsLunaphied
Fixes: https://git.lix.systems/lix-project/lix/issues/157 Fixes: https://git.lix.systems/lix-project/lix/issues/221 Previously the entire escaped store URI was included. This would cause build failures if a very long or deeply nested path was being used in the store. Now, we use the first 48 characters of the URL (escaped), then 16 bytes of hash of the entire URL. This should never collide and limits the length of the file name to a bit over 64, which is fine. Change-Id: Ic1ba690a94e83749567c2c29460b8d1bcf2ac413
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 15:39:48 +0000