aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/local-store.cc
AgeCommit message (Collapse)Author
2023-03-21Support per-store Markdown documentationEelco Dolstra
2023-03-20Move enabled experimental feature to libutil structJohn Ericson
This is needed in subsequent commits to allow the settings and CLI args infrastructure itself to read this setting.
2023-03-02Remove FormatOrString and remaining uses of format()Eelco Dolstra
2023-02-28No inheritance for `TextInfo` and `FixedOutputInfo`John Ericson
2023-02-28Revert "Remove some designated initializers"John Ericson
This reverts commit ee9eb83a842eb97d0180fd9d349d30ff27fdb485.
2023-02-01Remove some designated initializersJohn Ericson
With the switch to C++20, the rules became more strict, and we can no longer initialize base classes. Make them comments instead. (BTW https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html this offers some new syntax for this use-case. Hopefully this will be adopted and we can eventually use it.)
2023-01-30Merge branch 'small-storePath-cleanups' into path-infoJohn Ericson
2023-01-23Don't add `StorePathDescriptor` for nowJohn Ericson
We don't need it yet, we can add it back later.
2023-01-17Move the default profiles to the user’s homeThéophane Hufschmitt
Rather than using `/nix/var/nix/{profiles,gcroots}/per-user/`, put the user profiles and gcroots under `$XDG_DATA_DIR/nix/{profiles,gcroots}`. This means that the daemon no longer needs to manage these paths itself (they are fully handled client-side). In particular, it doesn’t have to `chown` them anymore (removing one need for root). This does change the layout of the gc-roots created by nix-env, and is likely to break some stuff, so I’m not sure how to properly handle that.
2023-01-14Make `ValidPathInfo` have plain `StorePathSet` references like beforeJohn Ericson
This change can wait for another PR.
2023-01-06Do big rename to clean up codeJohn Ericson
- `PathReferences` -> `References` - `PathReferences<StorePath>` -> `StoreReference` - `references` -> `others` - `hasSelfReference` -> `self` And get rid of silly subclassing
2023-01-06Use named field initialization for referencesJohn Ericson
2023-01-06Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2023-01-03Move creation of the temp roots file into its own functionEelco Dolstra
This also moves the file handle into its own Sync object so we're not holding the _state while acquiring the file lock. There was no real deadlock risk here since locking a newly created file cannot block, but it's still a bit nicer.
2022-12-27Release shared lock before acquiring exclusive lockNaïm Favier
In principle, this should avoid deadlocks where two instances of Nix are holding a shared lock on big-lock and are both waiting to get an exclusive lock. However, it seems like `flock(2)` is supposed to do this automatically, so it's not clear whether this is actually where the problem comes from.
2022-11-21Fix indentationEelco Dolstra
2022-11-03Merge remote-tracking branch 'origin/master' into auto-uid-allocationEelco Dolstra
2022-09-22Dodge "trusted" vs "trustworthy" by being explicit John Ericson
Hopefully this is best!
2022-09-22"valid signature" -> "trustworthy signature"John Ericson
I just had a colleague get confused by the previous phrase for good reason. "valid" sounds like an *objective* criterion, e.g. and *invalid signature* would be one that would be trusted by no one, e.g. because it misformatted or something. What is actually going is that there might be a signature which is perfectly valid to *someone else*, but not to the user, because they don't trust the corresponding public key. This is a *subjective* criterion, because it depends on the arbitrary and personal choice of which public keys to trust. I therefore think "trustworthy" is a better adjective to use. Whether something is worthy of trust is clearly subjective, and then "trust" within that word nicely evokes `trusted-public-keys` and friends.
2022-09-19Improve durability of schema version file writessqualus
- call close explicitly in writeFile to prevent the close exception from being ignored - fsync after writing schema file to flush data to disk - fsync schema file parent to flush metadata to disk https://github.com/NixOS/nix/issues/7064
2022-09-12Address PR feedback on #6694Andrew Brooks
2022-09-06Keep created temp dirs inside store, but protect from GCAndrew Brooks
Implements the approach suggested by feedback on PR #6994, where tempdir paths are created in the store (now with an exclusive lock). As part of this work, the currently-broken and unused `createTempDirInStore` function is updated to create an exclusive lock on the temp directory in the store. The GC now makes a non-blocking attempt to lock any store directories that "look like" the temp directories created by this function, and if it can't acquire one, ignores the directory.
2022-09-02Prevent tempdir from being GC-ed before addToStoreFromDump has renamed itAndrew Brooks
This fixes issue 6823 by placing the tempdir used in LocalStore::addToStoreFromDump outside the Nix store, where automatic GC is no longer a concern.
2022-08-03Only use `renameFile` where neededThéophane Hufschmitt
In most places the fallback to copying isn’t needed and can actually be bad, so we’d rather not transparently fallback
2022-08-03moveFile -> renameFileThéophane Hufschmitt
`move` tends to have this `mv` connotation of “I will copy it for you if needs be”
2022-08-03Create a wrapper around stdlib’s `rename`Théophane Hufschmitt
Directly takes some c++ strings, and gently throws an exception on error (rather than having to inline this logic everywhere)
2022-05-04Get rid of most `.at` calls (#6393)Alain Zscheile
Use one of `get` or `getOr` instead which will either return a null-pointer (with a nicer error message) or a default value when the key is missing.
2022-04-21ca: add sqlite index on `RealisationsRefs(realisationReference)`Sergei Trofimovich
Without the change any CA deletion triggers linear scan on large RealisationsRefs table: sqlite>.eqp full sqlite> delete from RealisationsRefs where realisationReference IN ( select id from Realisations where outputPath = 1234567890 ); QUERY PLAN |--SCAN RealisationsRefs `--LIST SUBQUERY 1 `--SEARCH Realisations USING COVERING INDEX IndexRealisationsRefsOnOutputPath (outputPath=?) With the change it gets turned into a lookup: sqlite> CREATE INDEX IndexRealisationsRefsRealisationReference on RealisationsRefs(realisationReference); sqlite> delete from RealisationsRefs where realisationReference IN ( select id from Realisations where outputPath = 1234567890 ); QUERY PLAN |--SEARCH RealisationsRefs USING INDEX IndexRealisationsRefsRealisationReference (realisationReference=?) `--LIST SUBQUERY 1 `--SEARCH Realisations USING COVERING INDEX IndexRealisationsRefsOnOutputPath (outputPath=?)
2022-04-21Make sure to delete all the realisation refsregnat
Deleting just one will only work in the test cases where I didn’t bother creating too many of them :p
2022-04-21Fix the gc with indirect self-references via the realisationsregnat
If the derivation `foo` depends on `bar`, and they both have the same output path (because they are CA derivations), then this output path will depend both on the realisation of `foo` and of `bar`, which themselves depend on each other. This confuses SQLite which isn’t able to automatically solve this diamond dependency scheme. Help it by adding a trigger to delete all the references between the relevant realisations. Fix #5320
2022-04-20Actually, solve this in a lighter-weight wayJohn Ericson
The templating is very superficial
2022-04-20Move templated functions to `sqlite-impl.hh`John Ericson
This ensures that use-sites properly trigger new monomorphisations on one hand, and on the other hand keeps the main `sqlite.hh` clean and interface-only. I think that is good practice in general, but in this situation in particular we do indeed have `sqlite.hh` users that don't need the `throw_` function.
2022-04-19Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2022-04-19Avoid `fmt` when constructor already does itJohn Ericson
There is a correctnes issue here, but #3724 will fix that. This is just a cleanup for brevity's sake.
2022-03-31Add support for impure derivationsEelco Dolstra
Impure derivations are derivations that can produce a different result every time they're built. Example: stdenv.mkDerivation { name = "impure"; __impure = true; # marks this derivation as impure outputHashAlgo = "sha256"; outputHashMode = "recursive"; buildCommand = "date > $out"; }; Some important characteristics: * This requires the 'impure-derivations' experimental feature. * Impure derivations are not "cached". Thus, running "nix-build" on the example above multiple times will cause a rebuild every time. * They are implemented similar to CA derivations, i.e. the output is moved to a content-addressed path in the store. The difference is that we don't register a realisation in the Nix database. * Pure derivations are not allowed to depend on impure derivations. In the future fixed-output derivations will be allowed to depend on impure derivations, thus forming an "impurity barrier" in the dependency graph. * When sandboxing is enabled, impure derivations can access the network in the same way as fixed-output derivations. In relaxed sandboxing mode, they can access the local filesystem.
2022-03-29Simplify the handling of the hash moduloThéophane Hufschmitt
Rather than having four different but very similar types of hashes, make only one, with a tag indicating whether it corresponds to a regular of deferred derivation. This implies a slight logical change: The original Nix+multiple-outputs model assumed only one hash-modulo per derivation. Adding multiple-outputs CA derivations changed this as these have one hash-modulo per output. This change is now treating each derivation as having one hash modulo per output. This obviously means that we internally loose the guaranty that all the outputs of input-addressed derivations have the same hash modulo. But it turns out that it doesn’t matter because there’s nothing in the code taking advantage of that fact (and it probably shouldn’t anyways). The upside is that it is now much easier to work with these hashes, and we can get rid of a lot of useless `std::visit{ overloaded`. Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2022-03-25Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2022-03-18Generalize `DerivationType` in preparation for impure derivationsJohn Ericson
2022-03-17Clean up `DerivationOutput`, and headersJohn Ericson
1. `DerivationOutput` now as the `std::variant` as a base class. And the variants are given hierarchical names under `DerivationOutput`. In 8e0d0689be797f9e42f9b43b06f50c1af7f20b4a @matthewbauer and I didn't know a better idiom, and so we made it a field. But this sort of "newtype" is anoying for literals downstream. Since then we leaned the base class, inherit the constructors trick, e.g. used in `DerivedPath`. Switching to use that makes this more ergonomic, and consistent. 2. `store-api.hh` and `derivations.hh` are now independent. In bcde5456cc3295061a0726881c3e441444dd6680 I swapped the dependency, but I now know it is better to just keep on using incomplete types as much as possible for faster compilation and good separation of concerns.
2022-03-11Reduce variants for derivation hash moduloJohn Ericson
This changes was taken from dynamic derivation (#4628). It` somewhat undoes the refactors I first did for floating CA derivations, as the benefit of hindsight + requirements of dynamic derivations made me reconsider some things. They aren't to consequential, but I figured they might be good to land first, before the more profound changes @thufschmitt has in the works.
2022-03-10Merge remote-tracking branch 'upstream/master' into path-infoJohn Ericson
2022-02-28Merge remote-tracking branch 'upstream/master' into auto-uid-allocationJohn Ericson
2022-02-25Remove std::string alias (for real this time)Eelco Dolstra
Also use std::string_view in a few more places.
2022-02-16addToStoreFromDump(): Take std::string_viewEelco Dolstra
2022-01-25nix store ping: Report Nix daemon versionEelco Dolstra
Fixes #5952.
2022-01-18Add command 'nix store copy-log'Eelco Dolstra
Fixes #5222.
2022-01-18Get rid of std::shared_ptr<std::string> and ref<std::string>Eelco Dolstra
These were needed back in the pre-C++11 era because we didn't have move semantics. But now we do.
2022-01-11Make --repair-path also repair corrupt optimised linksGuillaume Maudoux
There already existed a smoke test for the link content length, but it appears that there exists some corruptions pernicious enough to replace the file content with zeros, and keeping the same length. --repair-path now goes as far as checking the content of the link, making it true to its name and actually repairing the path for such coruption cases.
2021-12-14Merge pull request #5366 from trofi/speedup-ca-queryThéophane Hufschmitt
ca-specific-schema.sql: add index on RealisationsRefs(referrer)
2021-12-10Merge branch 'master' of https://github.com/MagicRB/nixEelco Dolstra