aboutsummaryrefslogtreecommitdiff
path: root/src/libutil/hash.cc
AgeCommit message (Collapse)Author
2024-08-28tree-wide: shuffle headers around for about 30s compile timeJade Lovelace
This didn't really feel so worth it afterwards, but I did untangle a bunch of stuff that should not have been tangled. The general gist of this change is that variant bullshit was causing a bunch of compile time, and it seems like the only way to deal with variant induced compile time is to keep variant types out of headers. Explicit template instantiation seems to do nothing for them. I also seem to have gotten some back-end time improvement from explicitly instantiating regex, but I don't know why. There is no corresponding front-end time improvement from it: regex is still at the top of the sinners list. **** Templates that took longest to instantiate: 15231 ms: std::basic_regex<char>::_M_compile (28 times, avg 543 ms) 15066 ms: std::__detail::_Compiler<std::regex_traits<char>>::_Compiler (28 times, avg 538 ms) 12571 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_disjunction (28 times, avg 448 ms) 12454 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_alternative (28 times, avg 444 ms) 12225 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_term (28 times, avg 436 ms) 11363 ms: nlohmann::basic_json<>::parse<const char *> (21 times, avg 541 ms) 10628 ms: nlohmann::basic_json<>::basic_json (109 times, avg 97 ms) 10134 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_atom (28 times, avg 361 ms) Back-end time before messing with the regex: **** Function sets that took longest to compile / optimize: 8076 ms: void boost::io::detail::put<$>(boost::io::detail::put_holder<$> cons... (177 times, avg 45 ms) 4382 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (1247 times, avg 3 ms) 3137 ms: boost::stacktrace::detail::to_string_impl_base<boost::stacktrace::de... (137 times, avg 22 ms) 2896 ms: void boost::io::detail::mk_str<$>(std::__cxx11::basic_string<$>&, ch... (177 times, avg 16 ms) 2304 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (210 times, avg 10 ms) 2116 ms: bool std::__detail::_Compiler<$>::_M_expression_term<$>(std::__detai... (112 times, avg 18 ms) 2051 ms: std::_Rb_tree_iterator<$> std::_Rb_tree<$>::_M_emplace_hint_unique<$... (244 times, avg 8 ms) 2037 ms: toml::result<$> toml::detail::sequence<$>::invoke<$>(toml::detail::l... (93 times, avg 21 ms) 1928 ms: std::__detail::_Compiler<$>::_M_quantifier() (28 times, avg 68 ms) 1859 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump(nlohmann::js... (41 times, avg 45 ms) 1824 ms: std::_Function_handler<$>::_M_manager(std::_Any_data&, std::_Any_dat... (973 times, avg 1 ms) 1810 ms: std::__detail::_BracketMatcher<$>::_BracketMatcher(std::__detail::_B... (112 times, avg 16 ms) 1793 ms: nix::fetchers::GitInputScheme::fetch(nix::ref<$>, nix::fetchers::Inp... (1 times, avg 1793 ms) 1759 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (281 times, avg 6 ms) 1722 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (19 times, avg 90 ms) 1677 ms: boost::io::basic_altstringbuf<$>::overflow(int) (194 times, avg 8 ms) 1674 ms: std::__cxx11::basic_string<$>::_M_mutate(unsigned long, unsigned lon... (249 times, avg 6 ms) 1660 ms: std::_Rb_tree_node<$>* std::_Rb_tree<$>::_M_copy<$>(std::_Rb_tree_no... (304 times, avg 5 ms) 1599 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (19 times, avg 84 ms) 1568 ms: void std::__detail::_Compiler<$>::_M_insert_bracket_matcher<$>(bool) (112 times, avg 14 ms) 1541 ms: std::__shared_ptr<$>::~__shared_ptr() (531 times, avg 2 ms) 1539 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump_escaped(std:... (41 times, avg 37 ms) 1471 ms: void std::__detail::_Compiler<$>::_M_insert_character_class_matcher<... (112 times, avg 13 ms) After messing with the regex (notice std::__detail::_Compiler vanishes here, but I don't know why): **** Function sets that took longest to compile / optimize: 8054 ms: void boost::io::detail::put<$>(boost::io::detail::put_holder<$> cons... (177 times, avg 45 ms) 4313 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (1217 times, avg 3 ms) 3259 ms: boost::stacktrace::detail::to_string_impl_base<boost::stacktrace::de... (137 times, avg 23 ms) 3045 ms: void boost::io::detail::mk_str<$>(std::__cxx11::basic_string<$>&, ch... (177 times, avg 17 ms) 2314 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (207 times, avg 11 ms) 1923 ms: std::_Rb_tree_iterator<$> std::_Rb_tree<$>::_M_emplace_hint_unique<$... (216 times, avg 8 ms) 1817 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (18 times, avg 100 ms) 1816 ms: toml::result<$> toml::detail::sequence<$>::invoke<$>(toml::detail::l... (93 times, avg 19 ms) 1788 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump(nlohmann::js... (40 times, avg 44 ms) 1749 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (278 times, avg 6 ms) 1724 ms: std::__cxx11::basic_string<$>::_M_mutate(unsigned long, unsigned lon... (248 times, avg 6 ms) 1697 ms: boost::io::basic_altstringbuf<$>::overflow(int) (194 times, avg 8 ms) 1684 ms: nix::fetchers::GitInputScheme::fetch(nix::ref<$>, nix::fetchers::Inp... (1 times, avg 1684 ms) 1680 ms: std::_Rb_tree_node<$>* std::_Rb_tree<$>::_M_copy<$>(std::_Rb_tree_no... (303 times, avg 5 ms) 1589 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (18 times, avg 88 ms) 1483 ms: non-virtual thunk to boost::wrapexcept<$>::~wrapexcept() (181 times, avg 8 ms) 1447 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump_escaped(std:... (40 times, avg 36 ms) 1441 ms: std::__shared_ptr<$>::~__shared_ptr() (496 times, avg 2 ms) 1420 ms: boost::stacktrace::basic_stacktrace<$>::init(unsigned long, unsigned... (137 times, avg 10 ms) 1396 ms: boost::basic_format<$>::~basic_format() (194 times, avg 7 ms) 1290 ms: std::__cxx11::basic_string<$>::_M_replace_cold(char*, unsigned long,... (231 times, avg 5 ms) 1258 ms: std::vector<$>::~vector() (354 times, avg 3 ms) 1222 ms: std::__cxx11::basic_string<$>::_M_replace(unsigned long, unsigned lo... (231 times, avg 5 ms) 1194 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (49 times, avg 24 ms) 1186 ms: bool tao::pegtl::internal::sor<$>::match<$>(std::integer_sequence<$>... (1 times, avg 1186 ms) 1149 ms: std::__detail::_Executor<$>::_M_dfs(std::__detail::_Executor<$>::_Ma... (70 times, avg 16 ms) 1123 ms: toml::detail::sequence<$>::invoke(toml::detail::location&) (69 times, avg 16 ms) 1110 ms: nlohmann::json_abi_v3_11_3::basic_json<$>::json_value::destroy(nlohm... (55 times, avg 20 ms) 1079 ms: std::_Function_handler<$>::_M_manager(std::_Any_data&, std::_Any_dat... (541 times, avg 1 ms) 1033 ms: nlohmann::json_abi_v3_11_3::detail::lexer<$>::scan_number() (20 times, avg 51 ms) Change-Id: I10af282bcd4fc39c2d3caae3453e599e4639c70b
2024-08-26treewide: fix a bunch of lintsJade Lovelace
Fixes: - Identifiers starting with _ are prohibited - Some driveby header dependency cleaning which wound up with doing some extra fixups. - Fucking C style casts, man. C++ made these 1000% worse by letting you also do memory corruption with them with references. - Remove casts to Expr * where ExprBlackHole is an incomplete type by introducing an explicitly-cast eBlackHoleAddr as Expr *. - An incredibly illegal cast of the text bytes of the StorePath hash into a size_t directly. You can't DO THAT. Replaced with actually parsing the hash so we get 100% of the bits being entropy, then memcpying the start of the hash. If this shows up in a profile we should just make the hash parser faster with a lookup table or something sensible like that. - This horrendous bit of UB which I thankfully slapped a deprecation warning on, built, and it didn't trigger anywhere so it was dead code and I just deleted it. But holy crap you *cannot* do that. inline void mkString(const Symbol & s) { mkString(((const std::string &) s).c_str()); } - Some wrong lints. Lots of wrong macro lints, one wrong suspicious-sizeof lint triggered by the template being instantiated with only pointers, but the calculation being correct for both pointers and not-pointers. - Exceptions in destructors strike again. I tried to catch the exceptions that might actually happen rather than all the exceptions imaginable. We can let the runtime hard-kill it on other exceptions imo. Change-Id: I71761620846cba64d66ee7ca231b20c061e69710
2024-08-08tree-wide: fix a pile of lintsJade Lovelace
This: - Converts a bunch of C style casts into C++ casts. - Removes some very silly pointer subtraction code (which is no more or less busted on i686 than it began) - Fixes some "technically UB" that never had to be UB in the first place. - Makes finally follow the noexcept status of the inner function. Maybe in the future we should ban the function from not being noexcept, but that is not today. - Makes various locally-used exceptions inherit from std::exception. Change-Id: I22e66972602604989b5e494fd940b93e0e6e9297
2024-08-08refactor: make HashType and Base enum classes for type safetyJade Lovelace
Change-Id: I9fbd55a9d50464a56fe11cb42a06a206914150d8
2024-08-08build: integrate clang-tidy into CIJade Lovelace
This still has utterly unacceptably bad output format design that I would not inflict on anyone I like, but it *does* now exist, and you *can* find the errors in the log. Future work would obviously be to fix that and integrate the actual errors into Gerrit using codechecker or so. Followup issue: https://git.lix.systems/lix-project/lix/issues/457 Fixes: https://git.lix.systems/lix-project/lix/issues/147 Change-Id: Ifca22e443d357762125f4ad6bc4f568af3a26c62
2024-08-04tree-wide: fix various lint warningsJade Lovelace
Change-Id: I0fc80718eb7e02d84cc4b5d5deec4c0f41116134
2024-07-05libutil: convert readFileSource to a generatoreldritch horrors
Change-Id: I5f92b15fd367d46eb047d74ab6e317b4f51a46d3
2024-07-05libstore: convert dumpPath to a generatoreldritch horrors
Change-Id: Ic4cf5562504aa29130304469936f958c0426e5ef
2024-06-19libutil: return a source from readFileeldritch horrors
don't consume a sink, return a source instead. the only reason to not do this is a very slight reduction in dynamic allocations, but since we are going to *at least* do disk io that will not be a lot of overhead anyway Change-Id: Iae2f879ec64c3c3ac1d5310eeb6a85e696d4614a
2024-05-29util.hh: Delete remaining file and clean up headersTom Hubrecht
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2023-04-17Merge pull request #7732 from hercules-ci/make-initLibStore-viable-alternativeJohn Ericson
Make `initLibStore` a viable alternative
2023-04-07Require openssl >= 1.1.1Robert Hensing
Versions older this are sufficiently old that we don't want to support them, and they require extra support code.
2023-04-07Move OpenSSL init to initLibUtilRobert Hensing
Part of an effort to make it easier to initialize the right things, by moving code into the appropriate libraries.
2023-04-07Rename and protect `BufferedSink::write`Théophane Hufschmitt
The `write` name is ambiguous and could lead to some funny bugs like https://github.com/NixOS/nix/pull/8173#issuecomment-1500009480. So rename it to the more explicit `writeUnbuffered`. Besides, this method shouldn't be (and isn't) used outside of the class implementation, so mark it `protected`. This makes it more symetrical to `BufferedSource` which uses a `protected readUnbuffered` method.
2023-03-29avoid a string copy in printHash16Vaci Koblizek
2023-03-29return string_view from printHashType rather than stringVaci Koblizek
2023-03-28Avoid a string copy during Hash::to_stringVaci Koblizek
2022-04-07don't assume that rev is a SHA1 hashKasper Gałkowski
This was a problem when writing a fetcher that uses e.g. sha256 hashes for revisions. This doesn't actually do anything new, but allows for creating such fetchers in the future (perhaps when support for Git's SHA256 object format gains more popularity).
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-01-27return string_views from forceString*pennae
once a string has been forced we already have dynamic storage allocated for it, so we can easily reuse that storage instead of copying.
2020-12-02Sink: Use std::string_viewEelco Dolstra
2020-10-28ConsistencyEelco Dolstra
2020-08-05Add Hash::dummy to signal default valueCarlo Nucera
We did this in the same spirit of the dummy value that's present in libstore/path.hh
2020-07-20Remove period at the end of the exception messageJohn Ericson
2020-07-20parser.hh -> split.hhJohn Ericson
2020-07-16Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into ↵John Ericson
better-ca-parse-errors
2020-07-16Merge remote-tracking branch 'upstream/master' into hash-always-has-typeJohn Ericson
2020-07-15Merge branch 'master' of github.com:NixOS/nix into hash-always-has-typeCarlo Nucera
2020-07-14Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2020-07-13Remove istringstream_nocopyEelco Dolstra
2020-07-06Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2020-07-03Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into ↵John Ericson
better-ca-parse-errors
2020-07-03Merge branch 'master' of github.com:NixOS/nix into hash-always-has-typeJohn Ericson
2020-07-02Have `splitPrefix` and `splitPrefixTo` parser helpersJohn Ericson
2020-07-02Move `getParsedTypeAndSRI` to a more suitable locationJohn Ericson
Also mark it static
2020-07-02Fix abort in 'nix develop'Eelco Dolstra
2020-07-02Write the implementation for parseNonSRIUnprefixedCarlo Nucera
2020-07-02Abstract common parsing functionalityCarlo Nucera
2020-07-02Inline and simplify in parseAnyPrefixedCarlo Nucera
2020-07-02Rename fromSRI to parseSRI for constistencyCarlo Nucera
2020-07-02Get rid of the std::pairCarlo Nucera
2020-07-02Inline newFunctionCarlo Nucera
2020-07-02Fix test suiteCarlo Nucera
2020-07-01Rename two hash constructors to proper functionsCarlo Nucera
2020-07-01Inline Hash::init()Carlo Nucera
2020-07-01Keep the previous name, for diffingCarlo Nucera
2020-07-01Fixed build, we still have test errorsCarlo Nucera
2020-06-30WIP initial designCarlo Nucera
2020-06-30Factor the prefix splitting in hashCarlo Nucera
2020-06-29Merge remote-tracking branch 'upstream/master' into hash-always-has-typeJohn Ericson