aboutsummaryrefslogtreecommitdiff
path: root/scripts/download-using-manifests.pl.in
AgeCommit message (Collapse)Author
2016-01-05Verify TLS certificate before downloading binariesPhilip Potter
The --insecure flag to curl tells curl not to bother checking if the TLS certificate presented by the server actually matches the hostname requested, and actually is issued by a trusted CA chain. This almost entirely negates any benefit from using TLS in the first place. This removes the --insecure flag to ensure we actually have a secure connection to the intended hostname before downloading binaries. Manually tested locally within a dev-shell; was able to download binaries from https://cache.nixos.org without issue. [Note: --insecure was only used for fetching NARs, whose integrity is verified by Nix anyway using the hash from the .narinfo. But if we can fetch the .narinfo without --insecure, we can also fetch the .nar, so there is not much point to using --insecure. --Eelco]
2014-08-29Shut up "Wide character" warnings in Perl scriptsEelco Dolstra
2014-08-20Use proper quotes everywhereEelco Dolstra
2014-08-13Use $XDG_RUNTIME_DIR for temporary filesEelco Dolstra
2013-12-02Install bsdiff and bspatch in $(libexecdir)/nixEelco Dolstra
2013-07-01Add support for uncompressed NARs in binary cachesEelco Dolstra
Issue NixOS/hydra#102.
2013-06-20Don't keep "disabled" substituters runningEelco Dolstra
For instance, it's pointless to keep copy-from-other-stores running if there are no other stores, or download-using-manifests if there are no manifests. This also speeds things up because we don't send queries to those substituters.
2012-10-17nix-push: Only generate and copy a NAR if it doesn't already existEelco Dolstra
This prevents unnecessary and slow rebuilds of NARs that already exist in the binary cache.
2012-10-02Add operation ‘nix-store --repair-path’Eelco Dolstra
This operation allows fixing corrupted or accidentally deleted store paths by redownloading them using substituters, if available. Since the corrupted path cannot be replaced atomically, there is a very small time window (one system call) during which neither the old (corrupted) nor the new (repaired) contents are available. So repairing should be used with some care on critical packages like Glibc.
2012-09-21Style fixEelco Dolstra
2012-09-19Support xz compression in the download-using-manifests substituterEelco Dolstra
2012-07-30Do some validation of URLsEelco Dolstra
2012-07-27Let build.cc verify the expected hash of a substituter's outputEelco Dolstra
Since SubstitutionGoal::finished() in build.cc computes the hash anyway, we can prevent the inefficiency of computing the hash twice by letting the substituter tell Nix about the expected hash, which can then verify it.
2012-07-27download-using-manifests: Don't use nix-prefetch-urlEelco Dolstra
Instead call curl directly and pipe it into ‘nix-store --restore’. This saves I/O and prevents creating garbage in the Nix store.
2012-07-11Update the other substitutersEelco Dolstra
2011-11-29* download-using-manifests: use the Perl bindings.Eelco Dolstra
2011-10-10* Set the executable bit on scripts.Eelco Dolstra
2011-10-10* Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm underEelco Dolstra
the Nix:: namespace.
2011-07-13* Fix concurrency issues in download-using-manifests' handling of theEelco Dolstra
SQLite manifest cache. The DBI AutoCommit feature caused every process to have an active transaction at all times, which could indefinitely block processes wanting to update the manifest cache. * Disable fsync() in the manifest cache because we don't need integrity (the cache can always be recreated if it gets corrupted).
2011-04-11* configure: detect whether DBD::SQLite is present. If necessary theEelco Dolstra
location to DBI and DBD::SQLite can be passed with --with-dbi and --with-dbd-sqlite.
2011-04-10* Cache the manifests in /nix/var/nix/manifests in a SQLite database.Eelco Dolstra
This significantly speeds up the download-using-manifests substituter, especially if manifests are very large. For instance, one "nix-build -A geeqie" operation that updated four packages using binary patches went from 18.5s to 1.6s. It also significantly reduces memory use. The cache is kept in /nix/var/nix/manifests/cache.sqlite. It's updated automatically when manifests are added to or removed from /nix/var/nix/manifests. It might be interesting to have nix-pull store manifests directly in the DB, rather than storing them as separate flat files, but then we would need a command line interface to delete manifests from the DB.
2011-04-06* Remove the localPaths feature in manifests since it's no longer usedEelco Dolstra
and redundant anyway.
2011-03-16* Print a better error message.Eelco Dolstra
2010-12-22* In Hydra manifests the Size field is missing, so don't rely on it.Eelco Dolstra
This caused a lot of "Use of uninitialized value" warnings from Perl.
2010-12-13* When doing a query (e.g. `nix-store -r --dry-run'), don't make a lotEelco Dolstra
of expensive calls to `nix-store --check-validity'.
2010-12-05* Use CamelCase for the Perl modules.Eelco Dolstra
2010-11-17* Oops.Eelco Dolstra
2010-11-17* When checking whether a patch is applicable, for efficiency, useEelco Dolstra
`nix-store -q --hash' to get the hash of the base path rather than `nix-hash'. However, only do this for estimating the size of a download, not for the actual substitution, because sometimes the contents of store paths are modified (which they shouldn't, of course).
2010-11-17* In the download size indication, take binary patches into account.Eelco Dolstra
Hopefully this doesn't slow things down too much.
2010-11-17* Before a build, show the disk space that the downloaded store pathsEelco Dolstra
will approximately require.
2010-02-04* "Fix" incorrect help message.Eelco Dolstra
2010-02-02* Don't do a chdir to $tmpDir. It's not necessary, and Windows doesn'tEelco Dolstra
support deleting the current directory.
2009-03-19* Future proofing: assume we can read manifests up to version 10Eelco Dolstra
(which should therefore be backwards compatible).
2009-02-27* nix-install-package: don't pollute /nix/var/nix/manifests.Eelco Dolstra
2009-02-26* Handle base-16 hashes in manifests.Eelco Dolstra
2009-02-19* download-using-manifests: don't check the cryptographic hash ofEelco Dolstra
downloaded files; rather, check the hash of the unpacked store path. When the server produces bzipped NAR archives on demand (like Hydra does), the hash of the file is not known in advance; it's streamed from the server. Thus the manifest doesn't contain a hash for the bzipped NAR archive. However, the server does know the hash of the *uncompressed* NAR archive (the "NarHash" field), since it's stored in the Nix database (nix-store -q --hash /nix/store/bla). So we use that instead for checking the integrity of the download.
2008-11-20* Urgh.Eelco Dolstra
2008-09-08* Doh.Eelco Dolstra
2008-08-04* Doh.Eelco Dolstra
2008-08-02* Make nix-env --dry-run print the paths to be substituted correctlyEelco Dolstra
again. (After the previous substituter mechanism refactoring I didn't update the code that obtains the references of substitutable paths.) This required some refactoring: the substituter programs are now kept running and receive/respond to info requests via stdin/stdout.
2008-07-18* Allow read-only access to the store (e.g., non-root users on NixOSEelco Dolstra
can do operations like "nix-store -qR <path>" even without the Nix daemon).
2007-10-22(no commit message)Eelco Dolstra
2007-08-12* Get rid of the substitutes database table (NIX-47). Instead, if weEelco Dolstra
need any info on substitutable paths, we just call the substituters (such as download-using-manifests.pl) directly. This means that it's no longer necessary for nix-pull to register substitutes or for nix-channel to clear them, which makes those operations much faster (NIX-95). Also, we don't have to worry about keeping nix-pull manifests (in /nix/var/nix/manifests) and the database in sync with each other. The downside is that there is some overhead in calling an external program to get the substitutes info. For instance, "nix-env -qas" takes a bit longer. Abolishing the substitutes table also makes the logic in local-store.cc simpler, as we don't need to store info for invalid paths. On the downside, you cannot do things like "nix-store -qR" on a substitutable but invalid path (but nobody did that anyway). * Never catch interrupts (the Interrupted exception).
2007-01-23* New kind of manifest object: "localPath", which denotes that a storeEelco Dolstra
path can be created by copying it from another location in the file system. This is useful in the NixOS installation.
2007-01-23* Successors have been gone for ages.Eelco Dolstra
2007-01-13* Removed chroot support.Eelco Dolstra
2006-10-04* tmpnam() -> File::Temp::tempdir().Eelco Dolstra
2006-09-25* Clean up calls to system().Eelco Dolstra
2006-08-05prevent doing recursive chroots, by unsetting NIX_ROOT in the scripts.Armijn Hemel
2006-08-04* Remove the dependency on `date', use strftime instead.Eelco Dolstra