Age | Commit message (Collapse) | Author |
|
You can now include files via the "builders" option, using the syntax
"@<filename>". Having only one option makes it easier to override
builders completely.
For backward compatibility, the default is "@/etc/nix/machines", or
"@<filename>" for each file name in NIX_REMOTE_SYSTEMS.
|
|
This ensures that command line flags such as --builders get passed
correctly.
|
|
|
|
Shift Darwin sandbox to separate installed files
|
|
This reverts commit 908590dc6cfcca3a98755b194d93b2da39aee95c. Since
hydra-server can have a different store URI from hydra-queue-runner
now, we don't really need this.
|
|
|
|
|
|
|
|
This makes it slightly more manageable to see at a glance what in a
build's sandbox profile is unique to the build and what is standard. Also
a first step to factoring more of our Darwin logic into scheme functions
that will allow us a bit more flexibility. And of course less of that
nasty codegen in C++! 😀
|
|
|
|
This speeds up commands like "nix cat-store". For example:
$ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf
real 0m4.336s
$ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf
real 0m0.045s
The primary motivation is to allow hydra-server to serve files from S3
binary caches. Previously Hydra had a hack to do "nix-store -r
<path>", but that fetches the entire closure so is prohibitively
expensive.
There is no garbage collection of the NAR cache yet. Also, the entire
NAR is read when accessing a single member file. We could generate the
NAR listing to provide random access.
Note: the NAR cache is indexed by the store path hash, not the content
hash, so NAR caches should not be shared between binary caches, unless
you're sure that all your builds are binary-reproducible.
|
|
Probably as a result of a bad merge in
4b8f1b0ec066a5b994747b1afd050f5f62d857f6, we had both a
BinaryCacheStoreAccessor and a
RemoteFSAccessor. BinaryCacheStore::getFSAccessor() returned the
latter, but BinaryCacheStore::addToStore() checked for the
former. This probably caused hydra-queue-runner to download paths that
it just uploaded.
|
|
I needed this to test ACL/xattr removal in
canonicalisePathMetaData(). Might also be useful if you need to build
old Nixpkgs that doesn't have the required patches to remove
setuid/setgid creation.
|
|
It was getting too much like whac-a-mole listing all the retriable error
conditions, so we now retry by default and list the cases where retrying
is almost certainly hopeless.
|
|
Fixes #1568.
|
|
This is a hack to make hydra-queue-runner free its temproots
periodically, thereby ensuring that garbage collection of the
corresponding paths is not blocked until the queue runner is
restarted.
It would be better if temproots could be released earlier than at
process exit. I started working on a RAII object returned by functions
like addToStore() that releases temproots. However, this would be a
pretty massive change so I gave up on it for now.
|
|
|
|
For example,
$ nix-store -q --roots /nix/store/7phd2sav7068nivgvmj2vpm3v47fd27l-patchelf-0.8pre845_0315148
{temp:1}
denotes that the path is only being kept alive by a temporary root
(i.e. /nix/var/nix/temproots/). Similarly,
$ nix-store --gc --print-roots
...
{memory:9} -> /nix/store/094gpjn9f15ip17wzxhma4r51nvsj17p-curl-7.53.1
shows that curl is being used by some process.
|
|
|
|
I don't remember what the reasoning was here, but security is provided
by the signatures, not by whether the hash is provided by the other
store.
|
|
Since we may use a dedicated file descriptor in the future, this
allows us to change it. So builders can do
if [[ -n $NIX_LOG_FD ]]; then
echo "@nix { message... }" >&$NIX_LOG_FD
fi
|
|
|
|
Nix can now automatically run the garbage collector during builds or
while adding paths to the store. The option "min-free = <bytes>"
specifies that Nix should run the garbage collector whenever free
space in the Nix store drops below <bytes>. It will then delete
garbage until "max-free" bytes are available.
Garbage collection during builds is asynchronous; running builds are
not paused and new builds are not blocked. However, there also is a
synchronous GC run prior to the first build/substitution.
Currently, no old GC roots are deleted (as in "nix-collect-garbage
-d").
|
|
Since file locks are per-process rather than per-file-descriptor, the
garbage collector would always acquire a lock on its own temproots
file and conclude that it's stale.
|
|
Maybe this will fix the curl hangs on macOS. (We could also use
CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
|
|
Without this, substitute info is fetched sequentially, which is
superslow. In the old UI (e.g. nix-build), we call printMissing(),
which calls queryMissing(), thereby preheating the binary cache
cache. But the new UI doesn't do that.
|
|
|
|
In particular, drop the "build-" and "gc-" prefixes which are
pointless. So now you can say
nix build --no-sandbox
instead of
nix build --no-build-use-sandbox
|
|
And print them (separately from the progress bar) given sufficient -v
flags.
|
|
|
|
This makes the progress bar work for non-root users.
|
|
|
|
So the progress bar can show
[1/0/1 built, 0.0 MiB DL] building hello-2.10 (configuring): checking whether pread is declared without a macro... yes
|
|
In particular, this allows more relevant activities ("substituting X")
to supersede inferior ones ("downloading X").
|
|
This allows the progress bar to display "building perl-5.22.3" instead
of "building /nix/store/<hash>-perl-5.22.3.drv".
|
|
|
|
Actually, currently they can only create download activities. Thus,
downloads by builtins.fetchurl show up in the progress bar.
|
|
|
|
This replaces "nix-store --optimise". Main difference is that it has a
progress indicator.
|
|
|
|
|
|
|
|
They're the same thing after all.
Example:
$ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
[0/1 built, 49/98 copied, 16.3/92.8 MiB DL, 55.8/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/0pl9li1jigcj2dany47hpmn0r3r48wc4nz48v5mqhh426lgz3bz6.nar.xz'
|
|
E.g.
$ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
[0/1 built, 1/97/98 fetched, 65.8/92.8 MiB DL, 203.2/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/1czm9fk0svacy4h6a3fzkpafi4f7a9gml36kk8cq1igaghbspg3k.nar.xz'
|
|
It now shows the amount of data copied:
[8/1038 copied, 160.4/1590.9 MiB copied] copying path '...'
|
|
|
|
|
|
|
|
Issue #1506.
|
|
|