aboutsummaryrefslogtreecommitdiff
path: root/src/libstore
AgeCommit message (Collapse)Author
2020-04-07CleanupEelco Dolstra
2020-04-07Backport libfetchers from the flakes branchEelco Dolstra
This provides a pluggable mechanism for defining new fetchers. It adds a builtin function 'fetchTree' that generalizes existing fetchers like 'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a set of attributes, e.g. fetchTree { type = "git"; url = "https://example.org/repo.git"; ref = "some-branch"; rev = "abcdef..."; } The existing fetchers are just wrappers around this. Note that the input attributes to fetchTree are the same as flake input specifications and flake lock file entries. All fetchers share a common cache stored in ~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}). This also adds support for Git worktrees (c169ea59049f861aaba429f48b828d0820b74d1d).
2020-04-05Don't retry on "unsupported protocol" errorCole Helbling
When encountering an unsupported protocol, there's no need to retry. Chances are, it won't suddenly be supported between retry attempts; error instead. Otherwise, you see something like the following: $ nix-env -i -f git://git@github.com/foo/bar warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 335 ms warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 604 ms warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 1340 ms warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 2685 ms With this change, you now see: $ nix-env -i -f git://git@github.com/foo/bar error: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1)
2020-03-31fix placeholder not substituted in passAsFilemlatus
2020-03-30Remove global -I flagsEelco Dolstra
(cherry picked from commit 2c692a3b144523bca68dd6de618124ba6c9bb332)
2020-03-24Misc changes from the flakes branchEelco Dolstra
2020-03-24nix path-info --json: Print hash in SRI formatEelco Dolstra
(cherry picked from commit 442e665d6d3fcbdee7dece2f62a597142f8784b1)
2020-03-24buildenv: Eliminate global variables, other cleanupEelco Dolstra
(cherry picked from commit b82f75464d1e5ae9a00d8004e5dd7b1ca05059e4)
2020-03-24Add function for quoting stringsEelco Dolstra
(cherry picked from commit 7dcf5b011a0942ecf953f2b607c4c8d0e9e652c7)
2020-03-24Use std::string_viewEelco Dolstra
(cherry picked from commit 6529490cc10018d5191e50c482ac1180b96b1a3c)
2020-03-23Merge pull request #3429 from LnL7/darwin-sandboxEelco Dolstra
darwin sandbox
2020-03-20sandbox: fix /bin/sh on catalinaDaiderd Jordan
Sadly 10.15 changed /bin/sh to a shim which executes bash, this means it can't be used anymore without also opening up the sandbox to allow bash. Failed to exec /bin/bash as variant for /bin/sh (1: Operation not permitted).
2020-03-20sandbox: allow pty devicesDaiderd Jordan
Nix now runs builds with a pseudo-terminal to enable colored build output.
2020-03-20libstore: relax default sandbox-paths on darwinDaiderd Jordan
2020-03-20libstore: disable resolve-system-dependencies hookDaiderd Jordan
This is used to determine the dependency tree of impure libraries so nix knows what paths to open in the sandbox. With the less restrictive defaults it isn't needed anymore.
2020-03-13Fix ca-references feature checkEelco Dolstra
Fixes #3406.
2020-03-12pathInfoCache: Respect disk cache TTLs #3398Robert Hensing
2020-03-04nix-build: Fix !<output> handlingEelco Dolstra
This was broken by 22a754c091f765061f59bef5ce091268493bb138. https://hydra.nixos.org/eval/1573669
2020-03-04Optimise Derivation::unparse()Eelco Dolstra
In nix-instantiate --dry-run '<nixpkgs/nixos/release-combined.nix>' -A nixos.tests.simple.x86_64-linux this reduces time spent in unparse() from 9.15% to 4.31%. The main culprit was appending characters one at a time to the destination string. Even though the string has enough capacity, push_back() still needs to check this on every call.
2020-02-28Fix GC failures on bad store path namesEelco Dolstra
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
2020-02-19Merge pull request #3353 from tbsmoest/priv_tobias_pr_set_deathsig-1.4Eelco Dolstra
Fix PR_SET_PDEATHSIG results in Broken pipe (#2395)
2020-02-18LocalStore::checkDerivationOutputs(): Improve error messageEelco Dolstra
2020-02-14Fix PR_SET_PDEATHSIG results in Broken pipe (#2395)Tobias Möst
The ssh client is lazily started by the first worker thread, that requires a ssh connection. To avoid the ssh client to be killed, when the worker process is stopped, do not set PR_SET_PDEATHSIG.
2020-02-13Prevent uninitialized StorePath creationEelco Dolstra
2020-02-07Fix segfault in gcc on i686-linuxEelco Dolstra
src/libstore/ssh-store.cc: In constructor 'nix::SSHStore::SSHStore(const string&, const Params&)': src/libstore/ssh-store.cc:31:21: internal compiler error: Segmentation fault compress) ^ Please submit a full bug report, with preprocessed source if appropriate. https://hydra.nixos.org/build/111545609
2020-02-04Merge pull request #3344 from LnL7/ssh-ng-remote-paramsEelco Dolstra
ssh-store: add remote-store and remote-program query params
2020-02-03ssh-store: add remote-store and remote-program query paramsDaiderd Jordan
Brings the functionality of ssh-ng:// in sync with the legacy ssh:// implementation. Specifying the remote store uri enables various useful things. eg. $ nix copy --to ssh-ng://cache?remote-store=file://mnt/cache --all
2020-01-29Merge pull request #3335 from domenkozar/retry-429Eelco Dolstra
retry on HTTP status code 429
2020-01-29retry on HTTP status code 429Domen Kožar
2020-01-23structured-attrs: chown .attrs.* files to builderRobin Gloster
Otherwise `chmod .`'ing the build directory doesn't work anymore, which is done in nixpkgs if sourceRoot is set to '.'.
2020-01-22Fix clang warningEelco Dolstra
2020-01-21Fix derivation computation with __structuredAttrs and multiple outputsEelco Dolstra
Fixes error: derivation '/nix/store/klivma7r7h5lndb99f7xxmlh5whyayvg-zlib-1.2.11.drv' has incorrect output '/nix/store/fv98nnx5ykgbq8sqabilkgkbc4169q05-zlib-1.2.11-dev', should be '/nix/store/adm7pilzlj3z5k249s8b4wv3scprhzi1-zlib-1.2.11-dev'
2020-01-13build: remove warning when in sandboxing test modeDaiderd Jordan
Introduced in 66fccd5832d125e9162abc5ed351aa37708e9623, but somehow breaks the secure-drv-outputs test.
2020-01-13unbreak build without pchJohn Ericson
2020-01-06Fix use of uninitialized store pathEelco Dolstra
Fixes 'building of '/nix/store/00000000000000000000000000000000-': ...'.
2020-01-06Merge pull request #3303 from LnL7/darwin-sandboxEelco Dolstra
build: fix sandboxing on darwin
2020-01-05build: fix sandboxing on darwinDaiderd Jordan
Starting ba87b08f8529e4d9f8c58d8c625152058ceadb75 getEnv now returns an std::optional which means these getEnv() != "" conditions no longer happen if the variables are not defined.
2020-01-04build: fix path repairing when hash rewriting is requiredDaiderd Jordan
Handle store path repairing on darwin when sandboxing is enabled. Unlike on linux sandboxing on darwin still requires hash rewriting.
2020-01-04build: recover store path when replacing failsDaiderd Jordan
This shouldn't happen in normal circumstances, but just in case attempt to move the temporary path back if possible.
2020-01-02passAsFile: leave out the hash prefixedef
Having a colon in the path may cause issues, and having the hash function indicated isn't actually necessary. We now verify the path format in the tests to prevent regressions.
2020-01-02passAsFile: hash the attribute name instead of numbering sequentiallyPuck Meerburg
This makes the paths consistent without relying on ordering. Co-authored-by: edef <edef@edef.eu>
2020-01-01exportReferencesGraph: support workingGraham Christensen
Before, we would get: [deploy@bastion:~]$ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv these derivations will be built: /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv /nix/store/ssxwmll7v21did1c8j027q0m8w6pg41i-unit-prometheus-alertmanager-irc-notifier.service.drv /nix/store/mvyvkj46ay7pp7b1znqbkck2mq98k0qd-unit-script-network-local-commands-start.drv /nix/store/vsl1y9mz38qfk6pyirjwnfzfggz5akg6-unit-network-local-commands.service.drv /nix/store/wi5ighfwwb83fdmav6z6n2fw6npm9ffl-unit-prometheus-hydra-exporter.service.drv /nix/store/x0qkv535n75pbl3xn6nn1w7qkrg9wwyg-unit-prometheus-packet-sd.service.drv /nix/store/lv491znsjxdf51xnfxh9ld7r1zg14d52-unit-script-packet-sd-env-key-pre-start.drv /nix/store/nw4nzlca49agsajvpibx7zg5b873gk9f-unit-script-packet-sd-env-key-start.drv /nix/store/x674wwabdwjrkhnykair4c8mpxa9532w-unit-packet-sd-env-key.service.drv /nix/store/ywivz64ilb1ywlv652pkixw3vxzfvgv8-unit-wireguard-wg0.service.drv /nix/store/v3b648293g3zl8pnn0m1345nvmyd8dwb-unit-script-acme-selfsigned-status.nixos.org-start.drv /nix/store/zci5d3zvr6fgdicz6k7jjka6lmx0v3g4-unit-acme-selfsigned-status.nixos.org.service.drv /nix/store/f6pwvnm63d0kw5df0v7sipd1rkhqxk5g-system-units.drv /nix/store/iax8071knxk9c7krpm9jqg0lcrawf4lc-etc.drv /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv error: invalid file name 'closure-init-0' in 'exportReferencesGraph' This was tough to debug, I didn't figure out which one was broken until I did: nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv 2>&1 | grep nix/store | xargs -n1 nix-store -r and then looking at the remaining build graph: $ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv these derivations will be built: /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv error: invalid file name 'closure-init-0' in 'exportReferencesGraph' and knowing the initrd build is before the system, then: $ nix show-derivation /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv { "/nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv": { [...] "exportReferencesGraph": "closure-init-0 /nix/store/...-stage-1-init.sh closure-mdadm.conf-1 /nix/store/...-mdadm.conf closure-ubuntu.conf-2 ...", [...] } } I then searched the repo for "in 'exportReferencesGraph'", found this recently updated regex, and realized it was missing a "-".
2019-12-21Disable use-sqlite-wal under WSLBrian McKenna
Before: $ nix-channel --update unpacking channels... warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL) After: $ inst/bin/nix-channel --update unpacking channels... created 1 symlinks in user environment I've seen complaints that "sandbox" caused problems under WSL but I'm having no problems. I think recent changes could have fixed the issue.
2019-12-19Merge branch 'libarchive' of https://github.com/yorickvP/nixEelco Dolstra
2019-12-17Add priority setting to storesEelco Dolstra
This allows overriding the priority of substituters, e.g. $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \ --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10' Fixes #3264.
2019-12-16nix-store -r: Handle symlinks to store pathsEelco Dolstra
Fixes #3270.
2019-12-13Only install *.sb files on macOSEelco Dolstra
2019-12-13SimplifyEelco Dolstra
2019-12-12libstore/ssh: Improve error message on failing `execvp`Profpatsch
If the `throw` is reached, this means that execvp into `ssh` wasn’t successful. We can hint at a usual problem, which is a missing `ssh` executable. Test with: ``` env PATH= ./result/bin/nix-copy-closure --builders '' unusedhost ``` and the bash version with ``` env PATH= ./result/bin/nix-copy-closure --builders '' localhost ```
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-22 23:39:08 +0000