aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2024-10-01Revert "libstore: remove worker removeGoal"Raito Bezarius
Revert submission 1946 Reason for revert: regression in building (found via bisection) Reported by users: > error: path '/nix/store/04ca5xwvasz6s3jg0k7njz6rzi0d225w-jq-1.7.1-dev' does not exist in the store Reverted changes: /q/submissionid:1946 Change-Id: I6f1a4b2f7d7ef5ca430e477fc32bca62fd97036b
2024-09-29libstore: merge ContinueImmediately and StillAliveeldritch horrors
nothing needs to signal being still active but not actively pollable, only that immediate polling for the next goal work phase is in order. Change-Id: Ia43c1015e94ba4f5f6b9cb92943da608c4a01555
2024-09-29libstore: remove worker removeGoaleldritch horrors
this was immensely inefficient on large caches, as can exist when many derivations are buildable simultaneously. since we have smart pointers to goals we can do cache maintenance in goal deleters instead, and use the exact iterators instead of doing a linear search. this *does* rely on goals being deleted to remove them from the cache, which isn't true for toplevel goals. those would have previously been removed when done in all cases, removing the cache entry when keep-going is set. this is arguably incorrect since it might result in those goals being retried, although that could only happen with dynamic derivations or the likes. (luckily dynamic derivations not complete enough to allow this at all) Change-Id: I8e750b868393588c33e4829333d370f2c509ce99
2024-09-29libstore: extract a real makeGoalCommoneldritch horrors
makeDerivationGoalCommon had the right idea, but it didn't quite go far enough. let's do the rest and remove the remaining factory duplication. Change-Id: I1fe32446bdfb501e81df56226fd962f85720725b
2024-09-29libstore: remove Goal::keyeldritch horrors
this was a debugging aid from day one that should not have any impact on build semantics, and if it *does* have an impact on build semantics then build semantics are seriously broken. keeping the order imposed by these keys will be impossible once we let a real event loop schedule our jobs. Change-Id: I5c313324e1f213ab6453d82f41ae5e59de809a5b
2024-09-29libstore: make non-cache goal pointers strongeldritch horrors
without circular references we do not need weak goal pointers except for caches, which should not prevent goal destructors running. caches though cannot create circular references even when they keep strong references. if we removed goals from caches when their work() is fully finished, not when their destructors are run, we could keep strong pointers in caches. since we do not gain much from this we keep those pointers weak for now. Change-Id: I1d4a6850ff5e264443c90eb4531da89f5e97a3a0
2024-09-29libstore: remove Goal::WaitForWorldeldritch horrors
have DerivationGoal and its subclasses produce a wrapper promise for their intermediate results instead, and return this wrapper promise. Worker already handles promises that do not complete immediately, so we do not have to duplicate this into an entire result type variant. Change-Id: Iae8dbf63cfc742afda4d415922a29ac5a3f39348
2024-09-29libstore: fix build testseldritch horrors
the new event loop could very occasionally notice that a dependency of some goal has failed, process the failure, cause the depending goal to fail accordingly, and in the doing of the latter two steps let further dependencies that previously have not been reported as failed do their reporting anyway. in such cases a goal could fail with "1 dependencies failed", but more than one dependency failure message was shown. we'll now report the correct number of failed dependency goals in all cases. Change-Id: I5aa95dcb2db4de4fd5fee8acbf5db833531d81a8
2024-09-29libstore: have makeLocalDerivationGoal return unique_ptrseldritch horrors
these can be unique rather than shared because shared_ptr has a converting constructor. preparatory refactor for something else and not necessary on its own, and the extra allocations we must do for shared_ptr control blocks isn't usually relevant anyway. Change-Id: I5391715545240c6ec8e83a031206edafdfc6462f
2024-09-27libstore: turn Goal::WaitForGoals into a promiseeldritch horrors
also gets rid of explicit strong references to dependencies of any goal, and weak references to dependers as well. those are now only held within promises representing goal completion and thus independent of the goal's relation to each other. the weak references to dependers was only needed for notifications, and that's much better handled entirely by kj itself. Change-Id: I00d06df9090f8d6336ee4bb0c1313a7052fb016b
2024-09-27libstore: replace Goal::WaitForSlot with semaphoreseldritch horrors
now that we have an event loop in the worker we can use it and its magical execution suspending properties to replace the slot counts we managed explicitly with semaphores and raii tokens. technically this would not have needed an event loop base to be doable, but it is a whole lot easier to wait for a token to be available if there is a callback mechanism ready for use that doesn't require a whole damn dedicated abstract method in Goal to work, and specific calls to that dedicated method strewn all over the worker implementation Change-Id: I1da7cf386d94e2bbf2dba9b53ff51dbce6a0cff7
2024-09-27libstore: simplify Worker::waitForInputeldritch horrors
with waitForAWhile turned into promised the core functionality of waitForInput is now merely to let gc run every so often if needed Change-Id: I68da342bbc1d67653901cf4502dabfa5bc947628
2024-09-27libstore: make waiting for a while a promiseeldritch horrors
this simplifies waitForInput quite a lot, and at the same time makes polling less thundering-herd-y. it even fixes early polling wakeups! Change-Id: I6dfa62ce91729b8880342117d71af5ae33366414
2024-09-27libstore: turn builder output processing into event loopeldritch horrors
this removes the rather janky did-you-mean-async poll loop we had so far. sadly kj does not play well with pty file descriptors, so we do have to add our own async input stream that does not eat pty EIO and turns it into an exception. that's still a *lot* better than the old code, and using a real even loop makes everything else easier later. Change-Id: Idd7e0428c59758602cc530bcad224cd2fed4c15e
2024-09-26Merge changes Iaa2e0e9d,Ia973420f into mainjade
* changes: Fix passing custom CA files into the builtin:fetchurl sandbox [security] builtin:fetchurl: Enable TLS verification
2024-09-26Fix passing custom CA files into the builtin:fetchurl sandboxPuck Meerburg
Without this, verifying TLS certificates would fail on macOS, as well as any system that doesn't have a certificate file at /etc/ssl/certs/ca-certificates.crt, which includes e.g. Fedora. Change-Id: Iaa2e0e9db3747645b5482c82e3e0e4e8f229f5f9
2024-09-25[security] builtin:fetchurl: Enable TLS verificationEelco Dolstra
This is better for privacy and to avoid leaking netrc credentials in a MITM attack, but also the assumption that we check the hash no longer holds in some cases (in particular for impure derivations). Partially reverts https://github.com/NixOS/nix/commit/5db358d4d78aea7204a8f22c5bf2a309267ee038. (cherry picked from commit c04bc17a5a0fdcb725a11ef6541f94730112e7b6) (cherry picked from commit f2f47fa725fc87bfb536de171a2ea81f2789c9fb) (cherry picked from commit 7b39cd631e0d3c3d238015c6f450c59bbc9cbc5b) Upstream-PR: https://github.com/NixOS/nix/pull/11585 Change-Id: Ia973420f6098113da05a594d48394ce1fe41fbb9
2024-09-25main: log stack traces for std::terminateJade Lovelace
These stack traces kind of suck for the reasons mentioned on the CppTrace page here (no symbols for inline functions is a major one): https://github.com/jeremy-rifkin/cpptrace I would consider using CppTrace if it were packaged, but to be honest, I think that the more reasonable option is actually to move entirely to out-of-process crash handling and symbolization. The reason for this is that if you want to generate anything of substance on SIGSEGV or really any deadly signal, you are stuck in async-signal-safe land, which is not a place to be trying to run a symbolizer. LLVM does it anyway, probably carefully, and chromium *can* do it on debug builds but in general uses crashpad: https://source.chromium.org/chromium/chromium/src/+/main:base/debug/stack_trace_posix.cc;l=974;drc=82dff63dbf9db05e9274e11d9128af7b9f51ceaa;bpv=1;bpt=1 However, some stack traces are better than *no* stack traces when we get mystery exceptions falling out the bottom of the program. I've also promoted the path for "mystery exceptions falling out the bottom of the program" to hard crash and generate a core dump because although there's been some months since the last one of these, these are nonetheless always *atrociously* diagnosed. We can't improve the crash handling further until either we use Crashpad (which involves more C++ deps, no thanks) or we put in the ostensibly work in progress Rust minidump infrastructure, in which case we need to finish full support for Rust in libutil first. Sample report: Lix crashed. This is a bug. We would appreciate if you report it at https://git.lix.systems/lix-project/lix/issues with the following information included: Exception: std::runtime_error: lol Stack trace: 0# nix::printStackTrace() in /home/jade/lix/lix3/build/src/nix/../libutil/liblixutil.so 1# 0x000073C9862331F2 in /home/jade/lix/lix3/build/src/nix/../libmain/liblixmain.so 2# 0x000073C985F2E21A in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6 3# 0x000073C985F2E285 in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6 4# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/jade/lix/lix3/build/src/nix/../libmain/liblixmain.so 5# 0x00005CF65B6B048B in /home/jade/lix/lix3/build/src/nix/nix 6# 0x000073C985C8810E in /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6 7# __libc_start_main in /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6 8# 0x00005CF65B610335 in /home/jade/lix/lix3/build/src/nix/nix Change-Id: I1a9f6d349b617fd7145a37159b78ecb9382cb4e9
2024-09-21Merge "local-store: make extended attribute handling more robust" into mainalois31
2024-09-18util: fix brotli decompression of empty inputJade Lovelace
This caused an infinite loop before since it would just keep asking the underlying source for more data. In practice this happened because an HTTP server served a response to a HEAD request (for which curl will not retrieve any body or call our write callback function) with Content-Encoding: br, leading to decompressing nothing at all and going into an infinite loop. This adds a test to make sure none of our compression methods do that again, as well as just patching the HTTP client to never feed empty data into a compression algorithm (since they absolutely have the right to throw CompressionError on unexpectedly-short streams!). Reported on Matrix: https://matrix.to/#/!lymvtcwDJ7ZA9Npq:lix.systems/$8BWQR_zKxCQDJ40C5NnDo4bQPId3pZ_aoDj2ANP7Itc?via=lix.systems&via=matrix.org&via=tchncs.de Change-Id: I027566e280f0f569fdb8df40e5ecbf46c211dad1
2024-09-14Merge "store: add a hint on how to fix Lix installs broken by macOS Sequoia" ↵jade
into main
2024-09-14Merge changes Ia1481da4,Ifca1d74d into mainjade
* changes: archive: refactor bad mutable-state API in the NAR parse listener archive: rename ParseSink to NARParseVisitor
2024-09-14local-store: make extended attribute handling more robustAlois Wohlschlager
* Move the extended attribute deletion after the hardlink sanity check. We shouldn't be removing extended attributes on random files. * Make the entity owner-writable before attempting to remove extended attributes, since this operation usually requires write access on the file, and we shouldn't fail xattr deletion on a file that has been made unwritable by the builder or a previous canonicalisation pass. Fixes: https://git.lix.systems/lix-project/lix/issues/507 Change-Id: I7e6ccb71649185764cd5210f4a4794ee174afea6
2024-09-14store: add a hint on how to fix Lix installs broken by macOS SequoiaJade Lovelace
This is not a detailed diagnosis, and it's not worth writing one, tbh. This error basically never happens in normal operation, so diagnosing it by changing the error on macOS is good enough. Relevant: https://git.lix.systems/lix-project/lix-installer/issues/24 Relevant: https://git.lix.systems/lix-project/lix-installer/issues/18 Relevant: https://git.lix.systems/lix-project/lix/issues/521 Change-Id: I03701f917d116575c72a97502b8e1617679447f2
2024-09-13archive: refactor bad mutable-state API in the NAR parse listenerJade Lovelace
Remove the mutable state stuff that assumes that one file is being written a time. It's true that we don't write multiple files interleaved, but that mutable state is evil. Change-Id: Ia1481da48255d901e4b09a9b783e7af44fae8cff
2024-09-11archive: rename ParseSink to NARParseVisitorJade Lovelace
- Rename the listener to not be called a "sink". If it were a "sink" it would be eating bytes and conform with any of the Nix sink stuff (maybe FileHandle should be a Sink itself! but that's a later CL's problem). This is a parser listener. - Move the RetrieveRegularNARSink thing into store-api.cc, which is its only usage, and fix it to actually do what it is stated to do: crash if its invariants are violated. It's, of course, used to erm, unpack single-file NAR files, generated via a horrible contraption of sources and sinks that looks like a plumbing blueprint. Refactoring that is a future task. - Add a description of the invariants of NARParseVisitor in preparation of refactoring it. Change-Id: Ifca1d74d2947204a1f66349772e54dad0743e944
2024-09-09Merge changes If8ec210f,I6e2851b2 into mainalois31
* changes: libfetchers: serialise accept-flake-config properly libstore: declare SandboxMode JSON serialisation in the header
2024-09-08libstore: turn Worker in a kj event loop usereldritch horrors
using a proper event loop basis we no longer have to worry about most of the intricacies of poll(), or platform-dependent replacements for it. we may even be able to use the event loop and its promise system for all of our scheduling in the future. we don't do any real async processing yet, this is just preparation to separate the first such change from the huge api design difference with the async framework we chose (kj from capnp): kj::Promise, unlike std::future, doesn't return exceptions unmangled. it instead wraps any non-kj exception into a kj exception, erasing all type information and preserving mostly the what() string in the process. this makes sense in the capnp rpc use case where unrestricted exception types can't be transferred, and since it moves error handling styles closer to a world we'd actually like there's no harm in doing it only here for now Change-Id: I20f888de74d525fb2db36ca30ebba4bcfe9cc838
2024-09-02libstore: declare SandboxMode JSON serialisation in the headerAlois Wohlschlager
The JSON serialisation should be declared in the header so that all translation units can see it when needed, even though it seems that it has not been used anywhere else so far. Unfortunately, this means we cannot use the NLOHMANN_JSON_SERIALIZE_ENUM convenience macro, since it uses a slightly different signature, but the code is not too bad either. Change-Id: I6e2851b250e0b53114d2fecb8011ff1ea9379d0f
2024-09-01Clarify that `diff-hook` no longer needs to be an absolute pathRebecca Turner
See: https://gerrit.lix.systems/c/lix/+/1864 Change-Id: Ic70bfe42b261a83f2cb68b8f102833b739b8e03a
2024-09-01Merge changes I5566a985,I88cf53d3 into mainrebecca “wiggles” turner
* changes: Support relative and `~/` paths in config settings Thread `ApplyConfigOptions` through config parsing
2024-08-30libstore: add build result to Goal::Finishedeldritch horrors
it just makes sense to have it too, rather than just the pass/fail information we keep so far. once we turn goals into something more promise-shaped it'll also help detangle the current data flow mess Change-Id: I915cf04d177cad849ea7a5833215d795326f1946
2024-08-30libstore: rename Goal::Finished::result to exitCodeeldritch horrors
the more useful type for `result` is BuildResult. Change-Id: If93d9384e8d686eb63b33320f1d565f9b9afbf3a
2024-08-30libstore: remove queryMissing call from Workereldritch horrors
it doesn't have a purpose except cache priming, which is largely irrelevant by default (since another code path already runs this exact query). our store implementations do not benefit that much from this either, and the more bursty load may indeed harm them. Change-Id: I1cc12f8c21cede42524317736d5987f1e43fc9c9
2024-08-30libstore: use notifications for stats counterseldritch horrors
updating statistics *immediately* when any counter changes declutters things somewhat and makes useful status reports less dependent on the current worker main loop. using callbacks will make it easier to move the worker loop into kj entirely, using only promises for scheduling. Change-Id: I695dfa83111b1ec09b1a54cff268f3c1d7743ed6
2024-08-30libstore: don't ContinueImmediately where we can tail calleldritch horrors
there's no reason to go through the event loop in these cases. returning ContinueImmediately here is just a very convoluted way of jumping to the state we've just set after unwinding one frame of the stack, which never matters in the cases changed here because there are no live RAII guards. Change-Id: I7c00948c22e3caf35e934c1a14ffd2d40efc5547
2024-08-30libstore: print dependency errors from DerivationGoaleldritch horrors
this is not ideal, but it's better than having this stuck in the worker loop itself. setting ex on all failing goals is not problematic because only toplevel goals can ever be observable, all the others are ignored. notably only derivation goals ever set `ex`, substitution goals do not. Change-Id: I02e2164487b2955df053fef3c8e774d557aa638a
2024-08-30libstore: hide Worker goal factory methodseldritch horrors
this doesn't serve a great purpose yet except to confine construction of goals to the stack frame of Worker::run() and its child frames. we don't need this yet (and the goal constructors remain fully visible), but in a future change that fully removes the current worker loop we'll need some way of knowing which goals are top-level goals without passing the goals themselves around. once that's possible we can remove visible goals as a concept and rely on build result futures and a scheduler built upon them Change-Id: Ia73cdeffcfb9ba1ce9d69b702dc0bc637a4c4ce6
2024-08-30libstore: add "is dependency" info to goaleldritch horrors
whether goal errors are reported via the `ex` member or just printed to the log depends on whether the goal is a toplevel goal or a dependency. if goals are aware of this themselves we can move error printing out of the worker loop, and since a running worker can only be used by running goals it's totally sufficient to keep a `Worker::running` flag for this Change-Id: I6b5cbe6eccee1afa5fde80653c4b968554ddd16f
2024-08-28tree-wide: shuffle headers around for about 30s compile timeJade Lovelace
This didn't really feel so worth it afterwards, but I did untangle a bunch of stuff that should not have been tangled. The general gist of this change is that variant bullshit was causing a bunch of compile time, and it seems like the only way to deal with variant induced compile time is to keep variant types out of headers. Explicit template instantiation seems to do nothing for them. I also seem to have gotten some back-end time improvement from explicitly instantiating regex, but I don't know why. There is no corresponding front-end time improvement from it: regex is still at the top of the sinners list. **** Templates that took longest to instantiate: 15231 ms: std::basic_regex<char>::_M_compile (28 times, avg 543 ms) 15066 ms: std::__detail::_Compiler<std::regex_traits<char>>::_Compiler (28 times, avg 538 ms) 12571 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_disjunction (28 times, avg 448 ms) 12454 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_alternative (28 times, avg 444 ms) 12225 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_term (28 times, avg 436 ms) 11363 ms: nlohmann::basic_json<>::parse<const char *> (21 times, avg 541 ms) 10628 ms: nlohmann::basic_json<>::basic_json (109 times, avg 97 ms) 10134 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_atom (28 times, avg 361 ms) Back-end time before messing with the regex: **** Function sets that took longest to compile / optimize: 8076 ms: void boost::io::detail::put<$>(boost::io::detail::put_holder<$> cons... (177 times, avg 45 ms) 4382 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (1247 times, avg 3 ms) 3137 ms: boost::stacktrace::detail::to_string_impl_base<boost::stacktrace::de... (137 times, avg 22 ms) 2896 ms: void boost::io::detail::mk_str<$>(std::__cxx11::basic_string<$>&, ch... (177 times, avg 16 ms) 2304 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (210 times, avg 10 ms) 2116 ms: bool std::__detail::_Compiler<$>::_M_expression_term<$>(std::__detai... (112 times, avg 18 ms) 2051 ms: std::_Rb_tree_iterator<$> std::_Rb_tree<$>::_M_emplace_hint_unique<$... (244 times, avg 8 ms) 2037 ms: toml::result<$> toml::detail::sequence<$>::invoke<$>(toml::detail::l... (93 times, avg 21 ms) 1928 ms: std::__detail::_Compiler<$>::_M_quantifier() (28 times, avg 68 ms) 1859 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump(nlohmann::js... (41 times, avg 45 ms) 1824 ms: std::_Function_handler<$>::_M_manager(std::_Any_data&, std::_Any_dat... (973 times, avg 1 ms) 1810 ms: std::__detail::_BracketMatcher<$>::_BracketMatcher(std::__detail::_B... (112 times, avg 16 ms) 1793 ms: nix::fetchers::GitInputScheme::fetch(nix::ref<$>, nix::fetchers::Inp... (1 times, avg 1793 ms) 1759 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (281 times, avg 6 ms) 1722 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (19 times, avg 90 ms) 1677 ms: boost::io::basic_altstringbuf<$>::overflow(int) (194 times, avg 8 ms) 1674 ms: std::__cxx11::basic_string<$>::_M_mutate(unsigned long, unsigned lon... (249 times, avg 6 ms) 1660 ms: std::_Rb_tree_node<$>* std::_Rb_tree<$>::_M_copy<$>(std::_Rb_tree_no... (304 times, avg 5 ms) 1599 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (19 times, avg 84 ms) 1568 ms: void std::__detail::_Compiler<$>::_M_insert_bracket_matcher<$>(bool) (112 times, avg 14 ms) 1541 ms: std::__shared_ptr<$>::~__shared_ptr() (531 times, avg 2 ms) 1539 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump_escaped(std:... (41 times, avg 37 ms) 1471 ms: void std::__detail::_Compiler<$>::_M_insert_character_class_matcher<... (112 times, avg 13 ms) After messing with the regex (notice std::__detail::_Compiler vanishes here, but I don't know why): **** Function sets that took longest to compile / optimize: 8054 ms: void boost::io::detail::put<$>(boost::io::detail::put_holder<$> cons... (177 times, avg 45 ms) 4313 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (1217 times, avg 3 ms) 3259 ms: boost::stacktrace::detail::to_string_impl_base<boost::stacktrace::de... (137 times, avg 23 ms) 3045 ms: void boost::io::detail::mk_str<$>(std::__cxx11::basic_string<$>&, ch... (177 times, avg 17 ms) 2314 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (207 times, avg 11 ms) 1923 ms: std::_Rb_tree_iterator<$> std::_Rb_tree<$>::_M_emplace_hint_unique<$... (216 times, avg 8 ms) 1817 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (18 times, avg 100 ms) 1816 ms: toml::result<$> toml::detail::sequence<$>::invoke<$>(toml::detail::l... (93 times, avg 19 ms) 1788 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump(nlohmann::js... (40 times, avg 44 ms) 1749 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (278 times, avg 6 ms) 1724 ms: std::__cxx11::basic_string<$>::_M_mutate(unsigned long, unsigned lon... (248 times, avg 6 ms) 1697 ms: boost::io::basic_altstringbuf<$>::overflow(int) (194 times, avg 8 ms) 1684 ms: nix::fetchers::GitInputScheme::fetch(nix::ref<$>, nix::fetchers::Inp... (1 times, avg 1684 ms) 1680 ms: std::_Rb_tree_node<$>* std::_Rb_tree<$>::_M_copy<$>(std::_Rb_tree_no... (303 times, avg 5 ms) 1589 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (18 times, avg 88 ms) 1483 ms: non-virtual thunk to boost::wrapexcept<$>::~wrapexcept() (181 times, avg 8 ms) 1447 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump_escaped(std:... (40 times, avg 36 ms) 1441 ms: std::__shared_ptr<$>::~__shared_ptr() (496 times, avg 2 ms) 1420 ms: boost::stacktrace::basic_stacktrace<$>::init(unsigned long, unsigned... (137 times, avg 10 ms) 1396 ms: boost::basic_format<$>::~basic_format() (194 times, avg 7 ms) 1290 ms: std::__cxx11::basic_string<$>::_M_replace_cold(char*, unsigned long,... (231 times, avg 5 ms) 1258 ms: std::vector<$>::~vector() (354 times, avg 3 ms) 1222 ms: std::__cxx11::basic_string<$>::_M_replace(unsigned long, unsigned lo... (231 times, avg 5 ms) 1194 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (49 times, avg 24 ms) 1186 ms: bool tao::pegtl::internal::sor<$>::match<$>(std::integer_sequence<$>... (1 times, avg 1186 ms) 1149 ms: std::__detail::_Executor<$>::_M_dfs(std::__detail::_Executor<$>::_Ma... (70 times, avg 16 ms) 1123 ms: toml::detail::sequence<$>::invoke(toml::detail::location&) (69 times, avg 16 ms) 1110 ms: nlohmann::json_abi_v3_11_3::basic_json<$>::json_value::destroy(nlohm... (55 times, avg 20 ms) 1079 ms: std::_Function_handler<$>::_M_manager(std::_Any_data&, std::_Any_dat... (541 times, avg 1 ms) 1033 ms: nlohmann::json_abi_v3_11_3::detail::lexer<$>::scan_number() (20 times, avg 51 ms) Change-Id: I10af282bcd4fc39c2d3caae3453e599e4639c70b
2024-08-28Merge "libstore: remove static initializers for Store registrations" into mainrebecca “wiggles” turner
2024-08-26libstore: remove static initializers for Store registrationsPierre Bourdon
Ref #359. Change-Id: Ia45530ddee25fa9fc399ff10738bb0d8bbc8b221
2024-08-26treewide: fix a bunch of lintsJade Lovelace
Fixes: - Identifiers starting with _ are prohibited - Some driveby header dependency cleaning which wound up with doing some extra fixups. - Fucking C style casts, man. C++ made these 1000% worse by letting you also do memory corruption with them with references. - Remove casts to Expr * where ExprBlackHole is an incomplete type by introducing an explicitly-cast eBlackHoleAddr as Expr *. - An incredibly illegal cast of the text bytes of the StorePath hash into a size_t directly. You can't DO THAT. Replaced with actually parsing the hash so we get 100% of the bits being entropy, then memcpying the start of the hash. If this shows up in a profile we should just make the hash parser faster with a lookup table or something sensible like that. - This horrendous bit of UB which I thankfully slapped a deprecation warning on, built, and it didn't trigger anywhere so it was dead code and I just deleted it. But holy crap you *cannot* do that. inline void mkString(const Symbol & s) { mkString(((const std::string &) s).c_str()); } - Some wrong lints. Lots of wrong macro lints, one wrong suspicious-sizeof lint triggered by the template being instantiated with only pointers, but the calculation being correct for both pointers and not-pointers. - Exceptions in destructors strike again. I tried to catch the exceptions that might actually happen rather than all the exceptions imaginable. We can let the runtime hard-kill it on other exceptions imo. Change-Id: I71761620846cba64d66ee7ca231b20c061e69710
2024-08-25Support relative and `~/` paths in config settingsRebecca Turner
Change-Id: I5566a9858ba255f4ac5051d1368c7dfb24460f0a
2024-08-25Thread `ApplyConfigOptions` through config parsingRebecca Turner
This makes no changes to logic but makes the `ApplyConfigOptions` value available to consumers. Change-Id: I88cf53d38faac8472c556aee55c13d0acbd1e5db
2024-08-25libstore: make Goal::ex a shared_ptreldritch horrors
this makes WorkResult copyable, and just all around easier to deal with. in the future we'll need this to let Goal::work() return a promise for a WorkResult (or even just a Finished) that can be awaited by other goals. Change-Id: Ic5a1ce04c5a0f8e683bd00a2ed2b77a2e28989c1
2024-08-25libstore: remove unused Goal ctor parametereldritch horrors
Change-Id: I9345fe272d6df5bd592621ce2da369fc1cd36d6d
2024-08-25Merge "fix: good errors for failures caused by allowSubstitutes" into mainjade
2024-08-25Merge changes Ief8e8ebc,Id3135db0,If1e76169 into mainjade
* changes: libutil: delete unused boost context cruft build: remove approximately 400 seconds of CPU time (30%) fix: use http proxy for s3 access
2024-08-25libstore: diagnose local build failure in goaleldritch horrors
this should be done where we're actually trying to build something, not in the main worker loop that shouldn't have to be aware of such details Change-Id: I07276740c0e2e5591a8ce4828a4bfc705396527e