aboutsummaryrefslogtreecommitdiff
path: root/src/libstore/nar-info-disk-cache.cc
AgeCommit message (Collapse)Author
2023-03-30Use "raw pattern" for content address typesJohn Ericson
We weren't because this ancient PR predated it! This is actually a new version of the pattern which addresses some issues identified in #7479.
2023-02-07NarInfoDiskCache: Keep BinaryCache.id stable and improve testRobert Hensing
Fixes #3898 The entire `BinaryCaches` row used to get replaced after it became stale according to the `timestamp` column. In a concurrent scenario, this leads to foreign key conflicts as different instances of the in-process `state.caches` cache now differ, with the consequence that the older process still tries to use the `id` number of the old record. Furthermore, this phenomenon appears to have caused the cache for actual narinfos to be erased about every week, while the default ttl for narinfos was supposed to be 30 days.
2023-02-07NarInfoDiskCache: Prepare reproducer for #3898Robert Hensing
2023-02-07NarInfoDiskCacheImpl: Make dbPath a parameterRobert Hensing
This allows testing with a clean database.
2023-02-07NarInfoDiskCache: Rename cacheExists -> upToDateCacheExistsRobert Hensing
This is slightly more accurate considering that an outdated record may exist in the persistent cache. Possibly-outdated records are quite relevant as they may be foreign keys to more recent information that we want to keep, but we will not return them here.
2022-12-21[PDT] TDE-3114: prevent a race-condition when creating the S3 cachemupdt
2022-06-23[fixup] handle cache expiration in sqlite queryLinus Heckemann
2022-06-22nar-info-disk-cache: refresh nix-cache-info weeklyLinus Heckemann
This allows changes to nix-cache-info to be picked up by existing clients. Previously, the only way for this to happen would be for clients to delete binary-cache-v6.sqlite, which is quite awkward for users. On the other hand, updates to nix-cache-info should be pretty rare, hence the choice of a fairly long TTL. Configurability is probably not useful enough to warrant implementing it.
2021-05-10Simplify the realisations disk cacheregnat
2021-05-06Add a realisations disk cacheregnat
Similar to the nar-info disk cache (and using the same db). This makes rebuilds muuch faster. - This works regardless of the ca-derivations experimental feature. I could modify the logic to not touch the db if the flag isn’t there, but given that this is a trash-able local cache, it doesn’t seem to be really worth it. - We could unify the `NARs` and `Realisation` tables to only have one generic kv table. This is left as an exercise to the reader. - I didn’t update the cache db version number as the new schema just adds a new table to the previous one, so the db will be transparently migrated and is backwards-compatible. Fix #4746
2021-01-18--refresh: Imply setting .narinfo disk cache TTL to 0Eelco Dolstra
2020-08-06Minimize the usage of `Hash::dummy`John Ericson
2020-08-05Remove optionality in ValidPathInfo::narInfoCarlo Nucera
2020-07-01Rename two hash constructors to proper functionsCarlo Nucera
2020-06-23Merge remote-tracking branch 'upstream/master' into hash-always-has-typeJohn Ericson
2020-06-19WIP: Make Hash always store a valid hash typeJohn Ericson
2020-06-18Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatypeJohn Ericson
2020-06-16StorePath: Rewrite in C++Eelco Dolstra
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB and runtime by 0.8%. This is mostly because we're not parsing the hash part as a hash anymore (just validating that it consists of base-32 characters). Also, replace storePathToHash() by StorePath::hashPart().
2020-06-03libutils/hash: remove default encodingzimbatm
This will make it easier to reason about the hash encoding and switch to SRI everywhere where possible.
2020-06-02Change parseCa(Opt) to parseContentAddress(Opt)Carlo Nucera
2020-06-02WIPJohn Ericson
2020-03-24Misc changes from the flakes branchEelco Dolstra
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-10-29queryPathInfoUncached(): Return const ValidPathInfoEelco Dolstra
2018-12-12Fix assertion failure in NarInfoDiskCacheEelco Dolstra
https://hydra.nixos.org/build/85827920
2018-12-12nar-info-disk-cache: include ca in the cache entriesDaiderd Jordan
Without this information the content addressable state and hashes are lost after the first request, this causes signatures to be required for everything even tho the path could be verified without signing.
2018-04-06rename the options to mention it's a narinfo TTL as disk cache is used all ↵AmineChikhaoui
over the place for other operations
2018-04-06Make the TTL for disk cache configurable, we can now completely disableAmineChikhaoui
disk cache lookup for example by doing: nix copy --from <binary-cahe> <store-path> --option \ positive-disk-cache-ttl 0 Issues: #1885 #2035
2018-03-20Slight simplificationEelco Dolstra
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-02-28NarInfoDiskCache: Handle SQLite busy errorsEelco Dolstra
2017-01-27Periodically purge binary-cache.sqliteEelco Dolstra
2017-01-27Implement TTL for binary cache lookupsEelco Dolstra
2017-01-27Remove unused NARExistence tableEelco Dolstra
2016-08-10SQLite:: Add some convenienceEelco Dolstra
2016-06-20Re-implement negative binary cache lookup cachingEelco Dolstra
2016-06-03Fix narrowing conversion from int64_t to boolEelco Dolstra
http://hydra.nixos.org/build/36613774
2016-06-01HttpBinaryCacheStore: Fix caching of WantMassQueryEelco Dolstra
Also, test HttpBinaryCacheStore in addition to LocalBinaryCacheStore.
2016-06-01Make the store directory a member variable of StoreEelco Dolstra
2016-05-30Re-implement the WantMassQuery property of binary cachesEelco Dolstra
2016-04-21Implement S3BinaryCacheStore::queryAllValidPaths()Eelco Dolstra
This allows commands like "nix verify --all" or "nix path-info --all" to work on S3 caches. Unfortunately, this requires some ugly hackery: when querying the contents of the bucket, we don't want to have to read every .narinfo file. But the S3 bucket keys only include the hash part of each store path, not the name part. So as a special exception queryAllValidPaths() can now return store paths *without* the name part, and queryPathInfo() accepts such store paths (returning a ValidPathInfo object containing the full name).
2016-04-20Cache path info lookups in SQLiteEelco Dolstra
This re-implements the binary cache database in C++, allowing it to be used by other Store backends, in particular the S3 backend.