aboutsummaryrefslogtreecommitdiff
path: root/scripts/nix-push.in
AgeCommit message (Collapse)Author
2014-08-29Shut up "Wide character" warnings in Perl scriptsEelco Dolstra
2014-08-20Use proper quotes everywhereEelco Dolstra
2014-08-13Use $XDG_RUNTIME_DIR for temporary filesEelco Dolstra
2014-01-08Support cryptographically signed binary cachesEelco Dolstra
NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2013-07-01Add support for uncompressed NARs in binary cachesEelco Dolstra
Issue NixOS/hydra#102.
2012-11-06nix-push: Handle pushing a symlinkEelco Dolstra
2012-10-18nix-push: Add a flag ‘--manifest-path’ to write the manifest to another ↵Eelco Dolstra
directory
2012-10-17nix-push: Add ‘--link’ flagEelco Dolstra
If ‘--link’ is given, nix-push will create hard links to the NAR files in the store, rather than copying them. This is faster and requires less disk space. However, it doesn't work if the store is on a different file system.
2012-10-17nix-push: Only generate and copy a NAR if it doesn't already existEelco Dolstra
This prevents unnecessary and slow rebuilds of NARs that already exist in the binary cache.
2012-10-03When ‘--help’ is given, just run ‘man’ to show the manual pageEelco Dolstra
I.e. do what git does. I'm too lazy to keep the builtin help text up to date :-) Also add ‘--help’ to various commands that lacked it (e.g. nix-collect-garbage).
2012-09-19Support xz compression in the download-using-manifests substituterEelco Dolstra
2012-07-27Allow a binary cache to declare that it doesn't support "nix-env -qas"Eelco Dolstra
Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-26nix-push: Support generating a manifest againEelco Dolstra
This makes all the tests succeed. Woohoo!
2012-07-26nix-push: Remove the upload featureEelco Dolstra
2012-07-02nix-push: Always generate base-32 hashesEelco Dolstra
2012-07-02Binary caches: use a better keyEelco Dolstra
Use the hash part of the store path as a key rather than a hash of the store path. This is enough to get the desired privacy property.
2012-07-01Allow both bzip2 and xz compressionEelco Dolstra
2012-06-29nix-push: Don't pollute the current directory with result symlinkEelco Dolstra
2012-06-29DohEelco Dolstra
2012-06-29Use XZ compression in binary cachesEelco Dolstra
XZ compresses significantly better than bzip2. Here are the compression ratios and execution times (using 4 cores in parallel) on my /var/run/current-system (3.1 GiB): bzip2: total compressed size 849.56 MiB, 30.8% [2m08] xz -6: total compressed size 641.84 MiB, 23.4% [6m53] xz -7: total compressed size 621.82 MiB, 22.6% [7m19] xz -8: total compressed size 599.33 MiB, 21.8% [7m18] xz -9: total compressed size 588.18 MiB, 21.4% [7m40] Note that compression takes much longer. More importantly, however, decompression is much faster: bzip2: 1m47.274s xz -6: 0m55.446s xz -7: 0m54.119s xz -8: 0m52.388s xz -9: 0m51.842s The only downside to using -9 is that decompression takes a fair amount (~65 MB) of memory.
2012-06-28nix-push: create a manifest-less binary cacheEelco Dolstra
Manifests are a huge pain, since users need to run nix-pull directly or indirectly to obtain them. They tend to be large and lag behind the available binaries; also, the downloaded manifests in /nix/var/nix/manifest need to be in sync with the Nixpkgs sources. So we want to get rid of them. The idea of manifest-free operation works as follows. Nix is configured with a set of URIs of binary caches, e.g. http://nixos.org/binary-cache Whenever Nix needs a store path X, it checks each binary cache for the existence of a file <CACHE-URI>/<SHA-256 hash of X>.narinfo, e.g. http://nixos.org/binary-cache/bi1gh9...ia17.narinfo The .narinfo file contains the necessary information about the store path that was formerly kept in the manifest, i.e., (relative) URI of the compressed NAR, references, size, hash, etc. For example: StorePath: /nix/store/xqp4l88cr9bxv01jinkz861mnc9p7qfi-neon-0.29.6 URL: 1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd.nar.bz2 CompressedHash: sha256:1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd CompressedSize: 202542 NarHash: sha256:1af26536781e6134ab84201b33408759fc59b36cc5530f57c0663f67b588e15f NarSize: 700440 References: 043zrsanirjh8nbc5vqpjn93hhrf107f-bash-4.2-p24 cj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 ... Deriver: 4idz1bgi58h3pazxr3akrw4fsr6zrf3r-neon-0.29.6.drv System: x86_64-linux Nix then knows that it needs to download http://nixos.org/binary-cache/1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd.nar.bz2 to substitute the store path. Note that the store directory is omitted from the References and Deriver fields to save space, and that the URL can be relative to the binary cache prefix. This patch just makes nix-push create binary caches in this format. The next step is to make a substituter that supports them.
2012-04-14Remove unnecessary "system" argumentEelco Dolstra
2012-01-03* Use Nix::Config.Eelco Dolstra
2012-01-03* Refactoring: Get rid of a few subdirectories in corepkgs/, and someEelco Dolstra
other simplifications. * Use <nix/...> to locate the corepkgs. This allows them to be overriden through $NIX_PATH. * Use bash's pipefail option in the NAR builder so that we don't need to create a temporary file.
2011-11-29* For consistency with "nix-store -q --hash", produce hashes inEelco Dolstra
base-32. (This affects Hydra manifests.)
2011-10-10* Set the executable bit on scripts.Eelco Dolstra
2011-10-10* Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm underEelco Dolstra
the Nix:: namespace.
2011-04-11* configure: detect whether DBD::SQLite is present. If necessary theEelco Dolstra
location to DBI and DBD::SQLite can be passed with --with-dbi and --with-dbd-sqlite.
2011-04-11* Subtle bug: if you import File::stat in one module, it affects otherEelco Dolstra
modules as well. So use symbolic field names everywhere (which is nicer anyway).
2011-02-17(no commit message)Eelco Dolstra
2011-02-17* nix-push: handle the case where the hash is not set in the DB.Eelco Dolstra
2010-12-05* Use CamelCase for the Perl modules.Eelco Dolstra
2010-11-17* Store the NAR size in the manifest.Eelco Dolstra
2010-11-17* nix-push: no need to compute the NAR hash, since the Nix databaseEelco Dolstra
already has it (`nix-store -q --hash').
2009-03-18* Unify exportReferencesGraph and exportBuildReferencesGraph, and makeEelco Dolstra
sure that it works as expected when you pass it a derivation. That is, we have to make sure that all build-time dependencies are built, and that they are all in the input closure (otherwise remote builds might fail, for example). This is ensured at instantiation time by adding all derivations and their sources to inputDrvs and inputSrcs.
2008-12-04* Dirty hack to make nix-push work properly on derivations: theEelco Dolstra
derivation should be a source rather than a derivation dependency of the call to the NAR derivation. Otherwise the derivation (and all its dependencies) will be built as a side-effect, which may not even succeed.
2008-11-20* Urgh.Eelco Dolstra
2008-11-19* Primop builtins.storePath for declaring a store path as aEelco Dolstra
dependency. `storePath /nix/store/bla' gives exactly the same result as `toPath /nix/store/bla', except that the former includes /nix/store/bla in the dependency context of the string. Useful in some generated Nix expressions like nix-push, which now finally does the right thing wrt distributed builds. (Previously the path to be packed wasn't an explicit dependency, so it wouldn't be copied to the remote machine.)
2008-03-20* Cleanup.Eelco Dolstra
2007-12-30* Don't use "store expression", it's obsolete.Eelco Dolstra
2007-11-16* Flag `--no-build-hook' to disable distributed builds.Eelco Dolstra
* queryDeriver in daemon mode: don't barf if the other side returns an empty string (which means there is no deriver).
2007-09-04* nix-push / generate-patches: bzip the manifest.Eelco Dolstra
2007-08-15* Show errors in nix-prefetch-url.Eelco Dolstra
2006-10-04* tmpnam() -> File::Temp::tempdir().Eelco Dolstra
2006-09-25* Clean up calls to system().Eelco Dolstra
2006-09-25* Use builtins.toPath.Eelco Dolstra
2006-09-20* Use paths, not strings, when calling the function that generatesEelco Dolstra
NARs. Fixes the impurity of nix-push (NIX-21). * Better help.
2006-08-09* file:/ -> file://Eelco Dolstra
2006-08-08rework the --target flag. If this flag is used, a URI should be given. DefaultArmijn Hemel
values are not changed.
2006-08-07* In nar.nix, path -> storePath, otherwise we get a collision betweenEelco Dolstra
environment variable names on Cygwin (where they are case insensitive).
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 23:58:27 +0000