aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2022-05-04Fix compiler warningEelco Dolstra
2022-05-04Move json stuff out of util.ccEelco Dolstra
2022-05-04Get rid of most `.at` calls (#6393)Alain Zscheile
Use one of `get` or `getOr` instead which will either return a null-pointer (with a nicer error message) or a default value when the key is missing.
2022-05-03nix profile: Support overriding outputsEelco Dolstra
2022-05-03Allow selecting derivation outputs using 'installable!outputs'Eelco Dolstra
E.g. 'nixpkgs#glibc^dev,static' or 'nixpkgs#glibc^*'.
2022-04-28Fix libcxx buildEelco Dolstra
Fixes #6458.
2022-04-28Fix passing $OUT_PATHS to the post-build hookEelco Dolstra
Fixes #6446.
2022-04-21Merge pull request #6431 from NixOS/unbreak-my-buildEelco Dolstra
Make the default SQLiteError constructor public
2022-04-21ca: add sqlite index on `RealisationsRefs(realisationReference)`Sergei Trofimovich
Without the change any CA deletion triggers linear scan on large RealisationsRefs table: sqlite>.eqp full sqlite> delete from RealisationsRefs where realisationReference IN ( select id from Realisations where outputPath = 1234567890 ); QUERY PLAN |--SCAN RealisationsRefs `--LIST SUBQUERY 1 `--SEARCH Realisations USING COVERING INDEX IndexRealisationsRefsOnOutputPath (outputPath=?) With the change it gets turned into a lookup: sqlite> CREATE INDEX IndexRealisationsRefsRealisationReference on RealisationsRefs(realisationReference); sqlite> delete from RealisationsRefs where realisationReference IN ( select id from Realisations where outputPath = 1234567890 ); QUERY PLAN |--SEARCH RealisationsRefs USING INDEX IndexRealisationsRefsRealisationReference (realisationReference=?) `--LIST SUBQUERY 1 `--SEARCH Realisations USING COVERING INDEX IndexRealisationsRefsOnOutputPath (outputPath=?)
2022-04-21Make sure to delete all the realisation refsregnat
Deleting just one will only work in the test cases where I didn’t bother creating too many of them :p
2022-04-21Fix the gc with indirect self-references via the realisationsregnat
If the derivation `foo` depends on `bar`, and they both have the same output path (because they are CA derivations), then this output path will depend both on the realisation of `foo` and of `bar`, which themselves depend on each other. This confuses SQLite which isn’t able to automatically solve this diamond dependency scheme. Help it by adding a trigger to delete all the references between the relevant realisations. Fix #5320
2022-04-21Make the default SQLiteError constructor publicThéophane Hufschmitt
Otherwise the clang builds fail because the constructor of `SQLiteBusy` inherits it, `SQLiteError::_throw` tries to call it, which fails. Strangely, gcc works fine with it. Not sure what the correct behavior is and who is buggy here, but either way, making it public is at the worst a reasonable workaround
2022-04-20Actually, solve this in a lighter-weight wayJohn Ericson
The templating is very superficial
2022-04-20Move templated functions to `sqlite-impl.hh`John Ericson
This ensures that use-sites properly trigger new monomorphisations on one hand, and on the other hand keeps the main `sqlite.hh` clean and interface-only. I think that is good practice in general, but in this situation in particular we do indeed have `sqlite.hh` users that don't need the `throw_` function.
2022-04-20Merge remote-tracking branch 'upstream/master' into fix-url-formatJohn Ericson
2022-04-19Avoid `fmt` when constructor already does itJohn Ericson
There is a correctnes issue here, but #3724 will fix that. This is just a cleanup for brevity's sake.
2022-04-17doc: rephrase connect-timeout help messageAlex Ameen
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-04-17doc: document nix.conf connect-timeout defaultAlex Ameen
2022-04-10Log builder args and environment variablesSebastian Blunt
Previously it only logged the builder's path, this changes it to log the arguments at the same log level, and the environment variables at the vomit level. This helped me debug https://github.com/svanderburg/node2nix/issues/75
2022-04-08Remove duplicate "error:"Eelco Dolstra
2022-04-08Error: Remove unused sname() methodEelco Dolstra
2022-04-08Remove unused Error.name fieldEelco Dolstra
2022-04-06Fix empty 'nix copy' error messageEelco Dolstra
This was caused by SubstitutionGoal not setting the errorMsg field in its BuildResult. We now get a more descriptive message than in 2.7.0, e.g. error: path '/nix/store/13mh...' is required, but there is no substituter that can build it instead of the misleading (since there was no build) error: build of '/nix/store/13mh...' failed Fixes #6295.
2022-04-04Add missing #includeEelco Dolstra
2022-03-31Fix macOS buildEelco Dolstra
2022-03-31Rename 'pure' -> 'sandboxed' for consistencyEelco Dolstra
2022-03-31needsNetworkAccess() -> isSandboxed()Eelco Dolstra
2022-03-31Document isPure()Eelco Dolstra
2022-03-31Fix macOS buildEelco Dolstra
2022-03-31Support fixed-output derivations depending on impure derivationsEelco Dolstra
2022-03-31Add support for impure derivationsEelco Dolstra
Impure derivations are derivations that can produce a different result every time they're built. Example: stdenv.mkDerivation { name = "impure"; __impure = true; # marks this derivation as impure outputHashAlgo = "sha256"; outputHashMode = "recursive"; buildCommand = "date > $out"; }; Some important characteristics: * This requires the 'impure-derivations' experimental feature. * Impure derivations are not "cached". Thus, running "nix-build" on the example above multiple times will cause a rebuild every time. * They are implemented similar to CA derivations, i.e. the output is moved to a content-addressed path in the store. The difference is that we don't register a realisation in the Nix database. * Pure derivations are not allowed to depend on impure derivations. In the future fixed-output derivations will be allowed to depend on impure derivations, thus forming an "impurity barrier" in the dependency graph. * When sandboxing is enabled, impure derivations can access the network in the same way as fixed-output derivations. In relaxed sandboxing mode, they can access the local filesystem.
2022-03-29Simplify the handling of the hash moduloThéophane Hufschmitt
Rather than having four different but very similar types of hashes, make only one, with a tag indicating whether it corresponds to a regular of deferred derivation. This implies a slight logical change: The original Nix+multiple-outputs model assumed only one hash-modulo per derivation. Adding multiple-outputs CA derivations changed this as these have one hash-modulo per output. This change is now treating each derivation as having one hash modulo per output. This obviously means that we internally loose the guaranty that all the outputs of input-addressed derivations have the same hash modulo. But it turns out that it doesn’t matter because there’s nothing in the code taking advantage of that fact (and it probably shouldn’t anyways). The upside is that it is now much easier to work with these hashes, and we can get rid of a lot of useless `std::visit{ overloaded`. Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2022-03-28nix profile install: Don't use queryDerivationOutputMap()Eelco Dolstra
Instead get the outputs from Installable::build(). This will also allow 'nix profile install' to support impure derivations. Fixes #6286.
2022-03-25Use wantOutputEelco Dolstra
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-24Retry substitution after an incomplete closure only onceEelco Dolstra
This avoids an infinite loop in the final test in tests/binary-cache.sh. I think this was only not triggered previously by accident (because we were clearing wantedOutputs in between).
2022-03-24Only return wanted outputsEelco Dolstra
2022-03-24Random cleanupEelco Dolstra
2022-03-24Random cleanupEelco Dolstra
2022-03-24Add a test for fetchClosure and 'nix store make-content-addressed'Eelco Dolstra
2022-03-24Fix makeContentAddressed() on self-referencesEelco Dolstra
LocalStore::addToStore() since 79ae9e4558cbefd743f28a5e73110c2303b03a85 expects a regular NAR hash, rather than a NAR hash modulo self-references. Fixes #6300. Also, makeContentAddressed() now rewrites the entire closure (so 'nix store make-content-addressable' no longer needs '-r'). See #6301.
2022-03-24fetchClosure: Allow a path to be rewritten to CA on the flyEelco Dolstra
The advantage is that the resulting closure doesn't need to be signed, so you don't need to configure any binary cache keys on the client.
2022-03-22buildProfile(): Ignore manifest.{nix,json}Eelco Dolstra
If a package installs a file named manifest.json, it caused nix-env to consider the profile a new-style profile created by 'nix profile'. Fixes #6032.
2022-03-18Generalize `DerivationType` in preparation for impure derivationsJohn Ericson
2022-03-17Clean up `DerivationOutput`, and headersJohn Ericson
1. `DerivationOutput` now as the `std::variant` as a base class. And the variants are given hierarchical names under `DerivationOutput`. In 8e0d0689be797f9e42f9b43b06f50c1af7f20b4a @matthewbauer and I didn't know a better idiom, and so we made it a field. But this sort of "newtype" is anoying for literals downstream. Since then we leaned the base class, inherit the constructors trick, e.g. used in `DerivedPath`. Switching to use that makes this more ergonomic, and consistent. 2. `store-api.hh` and `derivations.hh` are now independent. In bcde5456cc3295061a0726881c3e441444dd6680 I swapped the dependency, but I now know it is better to just keep on using incomplete types as much as possible for faster compilation and good separation of concerns.
2022-03-17Merge pull request #6277 from thufschmitt/ca/nix-build-dry-runEelco Dolstra
Fix `nix build --dry-run` with CA derivations
2022-03-17gc: don't visit implicit referrers on garbage collectionSergei Trofimovich
Before the change garbage collector was not considering `.drv` and outputs as alive even if configuration says otherwise. As a result `nix store gc --dry-run` could visit (and parse) `.drv` files multiple times (worst case it's quadratic). It happens because `alive` set was populating only runtime closure without regard for actual configuration. The change fixes it. Benchmark: my system has about 139MB, 40K `.drv` files. Performance before the change: $ time nix store gc --dry-run real 4m22,148s Performance after the change: $ time nix store gc --dry-run real 0m14,178s
2022-03-17Fix `nix build --dry-run` with CA derivationsThéophane Hufschmitt
Don’t try and assume that we know the output paths when we’ve just built with `--dry-run`. Instead make `--dry-run` follow a different code path that won’t assume the knowledge of the output paths at all. Fix #6275
2022-03-16Merge pull request #6245 from trofi/fix-nix-store-gc-limit-auto-optimise-storeEelco Dolstra
nix store gc: account for auto-optimised store
2022-03-15Merge pull request #6229 from obsidiansystems/refactor-hash-moduloThéophane Hufschmitt
Overhaul derivation hash modulo somewhat
2022-03-13nix store gc: account for auto-optimised storeSergei Trofimovich
Before the change on a system with `auto-optimise-store = true`: $ nix store gc --verbose --max 1 deleted all the paths instead of one path (we requested 1 byte limit). It happens because every file in `auto-optimise-store = true` has at least 2 links: file itself and a link in /nix/store/.links/ directory. The change conservatively assumes that any file that has one (as before) or two links (assume auto-potimise mode) will free space. Co-authored-by: Sandro <sandro.jaeckel@gmail.com>