aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2020-01-28Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2020-01-28Add support for tarball flake inputsEelco Dolstra
For example, $ nix flake info https://github.com/edolstra/dwarffs/archive/master.tar.gz Fixes #2929.
2020-01-27Add Mercurial testsEelco Dolstra
2020-01-24Respect lock files of inputs + fine-grained lock file controlEelco Dolstra
When computing a lock file, we now respect the lock files of flake inputs. This is important for usability / reproducibility. For example, the 'nixops' flake depends on the 'nixops-aws' and 'nixops-hetzner' repositories. So when the 'nixops' flake is used in another flake, we want the versions of 'nixops-aws' and 'nixops-hetzner' locked by the the 'nixops' flake because those presumably have been tested. This can lead to a proliferation of versions of flakes like 'nixpkgs' (since every flake's lock file could depend on a different version of 'nixpkgs'). This is not a major issue when using Nixpkgs overlays or NixOS modules, since then the top-level flake composes those overlays/modules into *its* version of Nixpkgs and all other versions are ignored. Lock file computation has been made a bit more lazy so it won't try to fetch all those versions of 'nixpkgs'. However, in case it's necessary to minimize flake versions, there now are two input attributes that allow this. First, you can copy an input from another flake, as follows: inputs.nixpkgs.follows = "dwarffs/nixpkgs"; This states that the calling flake's 'nixpkgs' input shall be the same as the 'nixpkgs' input of the 'dwarffs' input. Second, you can override inputs of inputs: inputs.nixpkgs.url = github:edolstra/nixpkgs/<hash>; inputs.nixops.inputs.nixpkgs.url = github:edolstra/nixpkgs/<hash>; or equivalently, using 'follows': inputs.nixpkgs.url = github:edolstra/nixpkgs/<hash>; inputs.nixops.inputs.nixpkgs.follows = "nixpkgs"; This states that the 'nixpkgs' input of the 'nixops' input shall be the same as the calling flake's 'nixpkgs' input. Finally, at '-v' Nix now prints the changes to the lock file, e.g. $ nix flake update ~/Misc/eelco-configurations/hagbard inputs of flake 'git+file:///home/eelco/Misc/eelco-configurations?subdir=hagbard' changed: updated 'nixpkgs': 'github:edolstra/nixpkgs/7845bf5f4b3013df1cf036e9c9c3a55a30331db9' -> 'github:edolstra/nixpkgs/03f3def66a104a221aac8b751eeb7075374848fd' removed 'nixops' removed 'nixops/nixops-aws' removed 'nixops/nixops-hetzner' removed 'nixops/nixpkgs'
2020-01-23structured-attrs: chown .attrs.* files to builderRobin Gloster
Otherwise `chmod .`'ing the build directory doesn't work anymore, which is done in nixpkgs if sourceRoot is set to '.'.
2020-01-22clang fixesEelco Dolstra
https://hydra.nixos.org/build/110757171
2020-01-22Fix clang warningEelco Dolstra
2020-01-22Fix --override-flake and add a testEelco Dolstra
2020-01-21Convert fetchMercurial to a input typeEelco Dolstra
This enables Mercurial flakes. It also fixes a bug in pure mode where you could use a branch/tag name rather than a revision.
2020-01-21Pluggable fetchersEelco Dolstra
Flakes are now fetched using an extensible mechanism. Also lots of other flake cleanups.
2020-01-21Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2020-01-21Fix derivation computation with __structuredAttrs and multiple outputsEelco Dolstra
Fixes error: derivation '/nix/store/klivma7r7h5lndb99f7xxmlh5whyayvg-zlib-1.2.11.drv' has incorrect output '/nix/store/fv98nnx5ykgbq8sqabilkgkbc4169q05-zlib-1.2.11-dev', should be '/nix/store/adm7pilzlj3z5k249s8b4wv3scprhzi1-zlib-1.2.11-dev'
2020-01-13build: remove warning when in sandboxing test modeDaiderd Jordan
Introduced in 66fccd5832d125e9162abc5ed351aa37708e9623, but somehow breaks the secure-drv-outputs test.
2020-01-13unbreak build without pchJohn Ericson
2020-01-06Fix use of uninitialized store pathEelco Dolstra
Fixes 'building of '/nix/store/00000000000000000000000000000000-': ...'.
2020-01-06Merge pull request #3303 from LnL7/darwin-sandboxEelco Dolstra
build: fix sandboxing on darwin
2020-01-06Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2020-01-05build: fix sandboxing on darwinDaiderd Jordan
Starting ba87b08f8529e4d9f8c58d8c625152058ceadb75 getEnv now returns an std::optional which means these getEnv() != "" conditions no longer happen if the variables are not defined.
2020-01-04build: fix path repairing when hash rewriting is requiredDaiderd Jordan
Handle store path repairing on darwin when sandboxing is enabled. Unlike on linux sandboxing on darwin still requires hash rewriting.
2020-01-04build: recover store path when replacing failsDaiderd Jordan
This shouldn't happen in normal circumstances, but just in case attempt to move the temporary path back if possible.
2020-01-02passAsFile: leave out the hash prefixedef
Having a colon in the path may cause issues, and having the hash function indicated isn't actually necessary. We now verify the path format in the tests to prevent regressions.
2020-01-02passAsFile: hash the attribute name instead of numbering sequentiallyPuck Meerburg
This makes the paths consistent without relying on ordering. Co-authored-by: edef <edef@edef.eu>
2020-01-01exportReferencesGraph: support workingGraham Christensen
Before, we would get: [deploy@bastion:~]$ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv these derivations will be built: /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv /nix/store/ssxwmll7v21did1c8j027q0m8w6pg41i-unit-prometheus-alertmanager-irc-notifier.service.drv /nix/store/mvyvkj46ay7pp7b1znqbkck2mq98k0qd-unit-script-network-local-commands-start.drv /nix/store/vsl1y9mz38qfk6pyirjwnfzfggz5akg6-unit-network-local-commands.service.drv /nix/store/wi5ighfwwb83fdmav6z6n2fw6npm9ffl-unit-prometheus-hydra-exporter.service.drv /nix/store/x0qkv535n75pbl3xn6nn1w7qkrg9wwyg-unit-prometheus-packet-sd.service.drv /nix/store/lv491znsjxdf51xnfxh9ld7r1zg14d52-unit-script-packet-sd-env-key-pre-start.drv /nix/store/nw4nzlca49agsajvpibx7zg5b873gk9f-unit-script-packet-sd-env-key-start.drv /nix/store/x674wwabdwjrkhnykair4c8mpxa9532w-unit-packet-sd-env-key.service.drv /nix/store/ywivz64ilb1ywlv652pkixw3vxzfvgv8-unit-wireguard-wg0.service.drv /nix/store/v3b648293g3zl8pnn0m1345nvmyd8dwb-unit-script-acme-selfsigned-status.nixos.org-start.drv /nix/store/zci5d3zvr6fgdicz6k7jjka6lmx0v3g4-unit-acme-selfsigned-status.nixos.org.service.drv /nix/store/f6pwvnm63d0kw5df0v7sipd1rkhqxk5g-system-units.drv /nix/store/iax8071knxk9c7krpm9jqg0lcrawf4lc-etc.drv /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv error: invalid file name 'closure-init-0' in 'exportReferencesGraph' This was tough to debug, I didn't figure out which one was broken until I did: nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv 2>&1 | grep nix/store | xargs -n1 nix-store -r and then looking at the remaining build graph: $ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv these derivations will be built: /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv error: invalid file name 'closure-init-0' in 'exportReferencesGraph' and knowing the initrd build is before the system, then: $ nix show-derivation /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv { "/nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv": { [...] "exportReferencesGraph": "closure-init-0 /nix/store/...-stage-1-init.sh closure-mdadm.conf-1 /nix/store/...-mdadm.conf closure-ubuntu.conf-2 ...", [...] } } I then searched the repo for "in 'exportReferencesGraph'", found this recently updated regex, and realized it was missing a "-".
2019-12-21Disable use-sqlite-wal under WSLBrian McKenna
Before: $ nix-channel --update unpacking channels... warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) After: $ inst/bin/nix-channel --update unpacking channels... created 1 symlinks in user environment I've seen complaints that "sandbox" caused problems under WSL but I'm having no problems. I think recent changes could have fixed the issue.
2019-12-20Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-19Merge branch 'libarchive' of https://github.com/yorickvP/nixEelco Dolstra
2019-12-18Fix getting the timestamp of GitHub flakesEelco Dolstra
2019-12-18Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-12-16Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-16nix-store -r: Handle symlinks to store pathsEelco Dolstra
Fixes #3270.
2019-12-13Only install *.sb files on macOSEelco Dolstra
2019-12-13SimplifyEelco Dolstra
2019-12-12libstore/ssh: Improve error message on failing `execvp`Profpatsch
If the `throw` is reached, this means that execvp into `ssh` wasn’t successful. We can hint at a usual problem, which is a missing `ssh` executable. Test with: ``` env PATH= ./result/bin/nix-copy-closure --builders '' unusedhost ``` and the bash version with ``` env PATH= ./result/bin/nix-copy-closure --builders '' localhost ```
2019-12-12Temporarily revert to using 'tar' until we have gzip supportEelco Dolstra
2019-12-11Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-12-09Remove UserLock self-lock checkEelco Dolstra
This is no longer needed since we're not using POSIX locks anymore.
2019-12-07libarchive proof of conceptYorick van Pelt
2019-12-05TypoEelco Dolstra
2019-12-05Show hash mismatch warnings in SRI formatEelco Dolstra
(cherry picked from commit 63c5c91cc053cbc1fcb8d3fe71c41142c9f51bfa)
2019-12-04Merge remote-tracking branch 'origin/master' into flakesEelco Dolstra
2019-12-02Merge remote-tracking branch 'origin/recursive-nix'Eelco Dolstra
2019-11-30Fix typosBrian Wignall
2019-11-26Drop remaining uses of external "tar"Eelco Dolstra
Also, fetchGit now runs in O(1) memory since we pipe the output of 'git archive' directly into unpackTarball() (rather than first reading it all into memory).
2019-11-26Move code aroundEelco Dolstra
2019-11-26Move code aroundEelco Dolstra
2019-11-26More Rust FFI adventuresEelco Dolstra
We can now convert Rust Errors to C++ exceptions. At the Rust->C++ FFI boundary, Result<T, Error> will cause Error to be converted to and thrown as a C++ exception.
2019-11-26Rust cleanupEelco Dolstra
2019-11-26Make <nix/unpack-channel.nix> a builtin builderEelco Dolstra
This was the last function using a shell script, so this allows us to get rid of tar, coreutils, bash etc.