| Age | Commit message (Collapse) | Author |
|---|
|
tl;dr: With this 1 line change I was able to get a speedup of 1.5x on 1Gbit/s
wan connections by enabling zstd compression in nginx.
Also nix already supported all common compression format for http
transfer, webservers usually only enable them if they are advertised
through the Accept-Encoding header.
This pull requests makes nix advertises content compression support for
zstd, br, gzip and deflate.
It's particular useful to add transparent compression for binary caches
that serve packages from the host nix store in particular nix-serve,
nix-serve-ng and harmonia.
I tried so far gzip, brotli and zstd, whereas only zstd was able to bring
me performance improvements for 1Gbit/s WAN connections.
The following nginx configuration was used in combination with the
[zstd module](https://github.com/tokers/zstd-nginx-module) and
[harmonia](https://github.com/nix-community/harmonia/)
```nix
{
services.nginx.virtualHosts."cache.yourhost.com" = {
locations."/".extraConfig = ''
proxy_pass http://127.0.0.1:5000;
proxy_set_header Host $host;
proxy_redirect http:// https://;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
zstd on;
zstd_types application/x-nix-archive;
'';
};
}
```
For testing I unpacked a linux kernel tarball to the nix store using
this command `nix-prefetch-url --unpack https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.8.tar.gz`.
Before:
```console
$ nix build && rm -rf /tmp/hello && time ./result/bin/nix copy --no-check-sigs --from https://cache.thalheim.io --to 'file:///tmp/hello?compression=none' '/nix/store/j42mahch5f0jvfmayhzwbb88sw36fvah-linux-6.1.8.tar.gz'
warning: Git tree '/scratch/joerg/nix' is dirty
real 0m18,375s
user 0m2,889s
sys 0m1,558s
```
After:
```console
$ nix build && rm -rf /tmp/hello && time ./result/bin/nix copy --no-check-sigs --from https://cache.thalheim.io --to 'file:///tmp/hello?compression=none' '/nix/store/j42mahch5f0jvfmayhzwb
b88sw36fvah-linux-6.1.8.tar.gz'
real 0m11,884s
user 0m4,130s
sys 0m1,439s
```
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Update src/libstore/filetransfer.cc
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
|
|
Move `trustedUsers` and `allowedUsers` to separate config struct
|
|
Get rid of the `authHook` parameter on `processConnection`
|
|
These settings are not needed for libstore at all, they are just used by
the nix daemon *command* for authorization on unix domain sockets. My
moving them to a new configuration struct just in that file, we avoid
them leaking anywhere else.
Also, it is good to break up the mammoth `Settings` struct in general.
Issue #5638 tracks this.
The message is not changed because I do not want to regress in
convenience to the user. Just saying "this connection is not trusted"
doesn't tell them out to fix the issue. The ideal thing to do would be
to somehow parameterize `processCommand` on how the error should be
displayed, so different sorts of connections can display different
information to the user based on how authentication is performed for the
connection in question. This, however, is a good bit more work, so it is
left for the future.
This came up with me thinking about the tcp:// store (#5265). The larger
project is not TCP *per se*, but the idea that it should be possible for
something else to manage access control to services like the Nix Daemon,
and those services simply trust or trust the incoming connection as they
are told. This is a more capability-oriented way of thinking about trust
than "every server implements its own auth separately" as we are used to today.
Its very great that libstore itself already implements just this model,
and so via this refactor I basically want to "enshrine" that so it
continues to be the case.
|
|
This is (morally) dead code.
As @edolstra pointed out in
https://github.com/NixOS/nix/pull/5226#discussion_r1073470813, this is
no longer needed.
I created this in 8d4162ff9e940ea9e2f97b07f3030a722695901a, so it is
fitting that I now destroy it :).
|
|
Expand installation.md
|
|
Since #7478 it's mandatory that `initLibStore()` is called for store
operations. However that's not the case when running `openStore()` in
Perl using the perl-bindings. That breaks e.g. `hydra-eval-jobset` when
built against Nix 2.13 which uses small portions of the store API.
|
|
Don't send plugin-files to the daemon.
|
|
The setting itself was already ignored due to exception trying to set pluginFiles.
|
|
nix-shell: Colour the prompt red if the user is root
|
|
This is radically unsafe and the daemon has already loaded its plugins
anyway.
Fixes cachix/devenv#276
|
|
This matches the nixos prompt colours - green for standard user, red for
root
|
|
Fix C++20 compilation with clang
|
|
|
|
|
|
nix store ping: add --json flag
|
|
Separate `path.hh` from `content-address.hh`
|
|
Delete dead code
|
|
Fix configure.ac rapidcheck tests
|
|
Better error message for nix search when attr is not found
|
|
manual: Document that the store timestamp is now 1, not 0
|
|
nix flake show: Ignore empty attrsets
|
|
Proposal: Use C++20
|
|
For frameworks it's important that structures are as lazy as possible
to prevent infinite recursions, performance issues and errors that
aren't related to the thing to evaluate. As a consequence, they have
to emit more attributes than strictly (sic) necessary.
However, these attributes with empty values are not useful to the user
so we omit them.
|
|
|
|
|
|
doc: add `__structuredAttrs`, `outputChecks`, `unsafeDiscardReferences`
|
|
|
|
Commit 14bc3ce3d6d5745717fa19b8b43b5fdd117ff757 (0.13~43) changed the
timestamps in the Nix store from 0 to 1. Update the nix-store man
page to match.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
|
|
Add more property tests
|
|
- `AC_LANG_PUSH(C++)` is needed for the header check
- The library check is hopeless (without lots of third-party macros I
don't feel like getting) because name mangling
Pkg-config would make all this easier. I previously opened
https://github.com/emil-e/rapidcheck/issues/302, I should write a PR
too.
|
|
The references set seems to have been unused since `LegacySSHStore`
references were first created in
caa5793b4a74049ee37dd88eb1c5b785456ce40d.
The method decls never were upstream, and accidentally added by me in
062533f7cdb74026096ca8c7d5b6e393893d59ef (probably due to `git rerere`).
Sorry!
This reduces the diff from #3746.
|
|
No other getDefaultFlakeAttrPaths implementation has this trailing dot,
and the dot can show up in error messages like:
error: flake '...' does not provide attribute 'packages.x86_64-linux.', ...
|
|
- Clarify doc comments, Installables::getCursors returns non-empty
vector
- Use vector::at in Installable::getCursor instead of checking for empty
vector and throwing an exception with error message.
|
|
Handle the case where none of getActualAttrPaths() actually exists,
in which case instead of returning an empty vector.
This fixes the case where the user misspells the attribute name in nix
search. Instead of getting no search results, now it shows an error with
suggestions.
Also remove InstallableFlake::getCursor() override since it's now
equivalent to the base class version.
|
|
This is a nice idea that @roberth requested. If we could factor our a
generic `std::variant` impl as a follow-up it would be even better!
|
|
It is good to separate concerns; `StorePath` (in general) has nothing to
do with `ContentAddress` anyways.
This reduces the diff from #3746.
|
|
Avoid some `StorePath` <-> `Path` round trips
|
|
Avoid needless work and throwing away invariants.
These conversions date back to when `StorePath` was in Rust and there
were issues with it missing utility methods.
|
|
|
|
Move the default profiles to the user’s home
|
|
nix flake show: don't evaluate derivations for foreign systems by default
|
|
Things leading to another...
|
|
Self-contained outputs
|
|
Don't hardcode “x86_64-linux” as this won't work too nicely on other
platforms
|
|
typetetris/fix-url-parsing-file-as-application-scheme
Fix url parsing for urls using `file+`
|
|
Slightly butchered during the merge
|
|
|
|
getDefaultNixPath: actually respect `{restrict,pure}-eval`
|
|
Also put proper comparison methods on `DerivedPath` and
`NixStringContextElem`, which is needed for the tests but good in
general.
|
