aboutsummaryrefslogtreecommitdiff
path: root/scripts/nix-push.in
AgeCommit message (Collapse)Author
2016-08-10Nuke nix-push.Shea Levy
Rarely used, nix copy replaces it.
2016-02-16Add C++ functions for .narinfo processing / signingEelco Dolstra
This is currently only used by the Hydra queue runner rework, but like eff5021eaa6dc69f65ea1a8abe8f3ab11ef5eb0a it presumably will be useful for the C++ rewrite of nix-push and download-from-binary-cache. (@shlevy)
2015-06-08nix-push: Support -jEelco Dolstra
Fixes #548.
2015-02-17Include NAR size in fingerprint computationEelco Dolstra
This is not strictly needed for integrity (since we already include the NAR hash in the fingerprint) but it helps against endless data attacks [1]. (However, this will also require download-from-binary-cache.pl to bail out if it receives more than the specified number of bytes.) [1] https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf
2015-02-04Sign a subset of the .narinfoEelco Dolstra
We only need to sign the store path, NAR hash and references (the "fingerprint"). Everything else is irrelevant to security. For instance, the compression algorithm or the hash of the compressed NAR don't matter as long as the contents of the uncompressed NAR are correct. (Maybe we should include derivers in the fingerprint, but they're broken and nobody cares about them. Also, it might be nice in the future if .narinfos contained signatures from multiple independent signers. But that's impossible if the deriver is included in the fingerprint, since everybody will tend to have a different deriver for the same store path.) Also renamed the "Signature" field to "Sig" since the format changed in an incompatible way.
2015-02-04Use libsodium instead of OpenSSL for binary cache signingEelco Dolstra
Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2014-08-29Shut up "Wide character" warnings in Perl scriptsEelco Dolstra
2014-08-20Use proper quotes everywhereEelco Dolstra
2014-08-13Use $XDG_RUNTIME_DIR for temporary filesEelco Dolstra
2014-01-08Support cryptographically signed binary cachesEelco Dolstra
NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2013-07-01Add support for uncompressed NARs in binary cachesEelco Dolstra
Issue NixOS/hydra#102.
2012-11-06nix-push: Handle pushing a symlinkEelco Dolstra
2012-10-18nix-push: Add a flag ‘--manifest-path’ to write the manifest to another ↵Eelco Dolstra
directory
2012-10-17nix-push: Add ‘--link’ flagEelco Dolstra
If ‘--link’ is given, nix-push will create hard links to the NAR files in the store, rather than copying them. This is faster and requires less disk space. However, it doesn't work if the store is on a different file system.
2012-10-17nix-push: Only generate and copy a NAR if it doesn't already existEelco Dolstra
This prevents unnecessary and slow rebuilds of NARs that already exist in the binary cache.
2012-10-03When ‘--help’ is given, just run ‘man’ to show the manual pageEelco Dolstra
I.e. do what git does. I'm too lazy to keep the builtin help text up to date :-) Also add ‘--help’ to various commands that lacked it (e.g. nix-collect-garbage).
2012-09-19Support xz compression in the download-using-manifests substituterEelco Dolstra
2012-07-27Allow a binary cache to declare that it doesn't support "nix-env -qas"Eelco Dolstra
Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-26nix-push: Support generating a manifest againEelco Dolstra
This makes all the tests succeed. Woohoo!
2012-07-26nix-push: Remove the upload featureEelco Dolstra
2012-07-02nix-push: Always generate base-32 hashesEelco Dolstra
2012-07-02Binary caches: use a better keyEelco Dolstra
Use the hash part of the store path as a key rather than a hash of the store path. This is enough to get the desired privacy property.
2012-07-01Allow both bzip2 and xz compressionEelco Dolstra
2012-06-29nix-push: Don't pollute the current directory with result symlinkEelco Dolstra
2012-06-29DohEelco Dolstra
2012-06-29Use XZ compression in binary cachesEelco Dolstra
XZ compresses significantly better than bzip2. Here are the compression ratios and execution times (using 4 cores in parallel) on my /var/run/current-system (3.1 GiB): bzip2: total compressed size 849.56 MiB, 30.8% [2m08] xz -6: total compressed size 641.84 MiB, 23.4% [6m53] xz -7: total compressed size 621.82 MiB, 22.6% [7m19] xz -8: total compressed size 599.33 MiB, 21.8% [7m18] xz -9: total compressed size 588.18 MiB, 21.4% [7m40] Note that compression takes much longer. More importantly, however, decompression is much faster: bzip2: 1m47.274s xz -6: 0m55.446s xz -7: 0m54.119s xz -8: 0m52.388s xz -9: 0m51.842s The only downside to using -9 is that decompression takes a fair amount (~65 MB) of memory.
2012-06-28nix-push: create a manifest-less binary cacheEelco Dolstra
Manifests are a huge pain, since users need to run nix-pull directly or indirectly to obtain them. They tend to be large and lag behind the available binaries; also, the downloaded manifests in /nix/var/nix/manifest need to be in sync with the Nixpkgs sources. So we want to get rid of them. The idea of manifest-free operation works as follows. Nix is configured with a set of URIs of binary caches, e.g. http://nixos.org/binary-cache Whenever Nix needs a store path X, it checks each binary cache for the existence of a file <CACHE-URI>/<SHA-256 hash of X>.narinfo, e.g. http://nixos.org/binary-cache/bi1gh9...ia17.narinfo The .narinfo file contains the necessary information about the store path that was formerly kept in the manifest, i.e., (relative) URI of the compressed NAR, references, size, hash, etc. For example: StorePath: /nix/store/xqp4l88cr9bxv01jinkz861mnc9p7qfi-neon-0.29.6 URL: 1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd.nar.bz2 CompressedHash: sha256:1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd CompressedSize: 202542 NarHash: sha256:1af26536781e6134ab84201b33408759fc59b36cc5530f57c0663f67b588e15f NarSize: 700440 References: 043zrsanirjh8nbc5vqpjn93hhrf107f-bash-4.2-p24 cj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 ... Deriver: 4idz1bgi58h3pazxr3akrw4fsr6zrf3r-neon-0.29.6.drv System: x86_64-linux Nix then knows that it needs to download http://nixos.org/binary-cache/1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd.nar.bz2 to substitute the store path. Note that the store directory is omitted from the References and Deriver fields to save space, and that the URL can be relative to the binary cache prefix. This patch just makes nix-push create binary caches in this format. The next step is to make a substituter that supports them.
2012-04-14Remove unnecessary "system" argumentEelco Dolstra
2012-01-03* Use Nix::Config.Eelco Dolstra
2012-01-03* Refactoring: Get rid of a few subdirectories in corepkgs/, and someEelco Dolstra
other simplifications. * Use <nix/...> to locate the corepkgs. This allows them to be overriden through $NIX_PATH. * Use bash's pipefail option in the NAR builder so that we don't need to create a temporary file.
2011-11-29* For consistency with "nix-store -q --hash", produce hashes inEelco Dolstra
base-32. (This affects Hydra manifests.)
2011-10-10* Set the executable bit on scripts.Eelco Dolstra
2011-10-10* Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm underEelco Dolstra
the Nix:: namespace.
2011-04-11* configure: detect whether DBD::SQLite is present. If necessary theEelco Dolstra
location to DBI and DBD::SQLite can be passed with --with-dbi and --with-dbd-sqlite.
2011-04-11* Subtle bug: if you import File::stat in one module, it affects otherEelco Dolstra
modules as well. So use symbolic field names everywhere (which is nicer anyway).
2011-02-17(no commit message)Eelco Dolstra
2011-02-17* nix-push: handle the case where the hash is not set in the DB.Eelco Dolstra
2010-12-05* Use CamelCase for the Perl modules.Eelco Dolstra
2010-11-17* Store the NAR size in the manifest.Eelco Dolstra
2010-11-17* nix-push: no need to compute the NAR hash, since the Nix databaseEelco Dolstra
already has it (`nix-store -q --hash').
2009-03-18* Unify exportReferencesGraph and exportBuildReferencesGraph, and makeEelco Dolstra
sure that it works as expected when you pass it a derivation. That is, we have to make sure that all build-time dependencies are built, and that they are all in the input closure (otherwise remote builds might fail, for example). This is ensured at instantiation time by adding all derivations and their sources to inputDrvs and inputSrcs.
2008-12-04* Dirty hack to make nix-push work properly on derivations: theEelco Dolstra
derivation should be a source rather than a derivation dependency of the call to the NAR derivation. Otherwise the derivation (and all its dependencies) will be built as a side-effect, which may not even succeed.
2008-11-20* Urgh.Eelco Dolstra
2008-11-19* Primop builtins.storePath for declaring a store path as aEelco Dolstra
dependency. `storePath /nix/store/bla' gives exactly the same result as `toPath /nix/store/bla', except that the former includes /nix/store/bla in the dependency context of the string. Useful in some generated Nix expressions like nix-push, which now finally does the right thing wrt distributed builds. (Previously the path to be packed wasn't an explicit dependency, so it wouldn't be copied to the remote machine.)
2008-03-20* Cleanup.Eelco Dolstra
2007-12-30* Don't use "store expression", it's obsolete.Eelco Dolstra
2007-11-16* Flag `--no-build-hook' to disable distributed builds.Eelco Dolstra
* queryDeriver in daemon mode: don't barf if the other side returns an empty string (which means there is no deriver).
2007-09-04* nix-push / generate-patches: bzip the manifest.Eelco Dolstra
2007-08-15* Show errors in nix-prefetch-url.Eelco Dolstra
2006-10-04* tmpnam() -> File::Temp::tempdir().Eelco Dolstra
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 18:30:54 +0000