aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-04-29docs: give translation examples from nix-build -E/-A to installablesQyriad
Change-Id: I03f4c7c1049063539a35ba500a07bb8f866d4cb7
2024-04-29docs: clarify how the different kinds of installables are selectedQyriad
Change-Id: I146736bb97ebe035e04be69ce9fb60a557e38c6c
2024-04-29docs: guide to installables docs in installable commands' docsQyriad
The installables syntax is not documented in any of the man pages or docbook pages for any of those individual commands. And while these commands really should at least peripherally individually document how installables work, in the meantime we can at least direct people to the right place. This commit also clarifies the unexpected fact that `nix profile remove` and `nix profile upgrade` do *not* take installables. Change-Id: I3b1453cb197a613bbab639c66a466365c3592c6d
2024-04-29add VM test for nix upgrade-nixQyriad
This commit adds a new NixOS VM test, which tests that `nix upgrade-nix` works on both kinds of profiles (manifest.nix and manifest.json). Done as a separate commit from 831d18a13, since it relies on the --store-path argument from 026c90e5f as well. Change-Id: I5fc94b751d252862cb6cffb541a4c072faad9f3b
2024-04-29nix3-upgrade-nix: allow manually specifying new nixQyriad
This allows manually specifying a store path for the new Nix that gets linked into Nix's profile. Change-Id: Ib71711ffb466febf4a6892e3fdbda644e053770d
2024-04-29fix `nix upgrade-nix` on new-style profilesQyriad
nix3-profile automatically migrates any profile its used on to its style of profile -- the ones with manifest.json instead of manifest.nix. On non-NixOS systems, Nix is conventionally installed to the profile at /nix/var/nix/profiles/default, so if a user passed that to `--profile` of `nix profile`, then it would break upgrade-nix from ever working again, without recreating the profile. This commit fixes that, and allows upgrade-nix to work on either kind of profile. Fixes #16. Change-Id: I4c49b1beba93bb50e8f8a107edc451affe08c3f7
2024-04-29refactor some nix-env and profile code to libcmdQyriad
Notably, ProfileManifest and ProfileElement are useful generic profile management code, and nix profile is not the only place in the codebase where profiles are relevant. This commit is in preparation for fixing upgrade-nix's interaction with new-style profiles. Change-Id: Iefc8bbd34b4bc6012175cb3d6e6a8207973bc792
2024-04-28Merge "remove Github workflow files" into mainQyriad
2024-04-28remove Github workflow filesQyriad
Fixes #251. We aren't using them. Change-Id: Ib24da6e0277bdd6afa9103c553c743a610f7b4e1
2024-04-27Run all derivation builders inside the sandbox on macOSPuck Meerburg
This replaces the external sandbox-exec call with direct calls into libsandbox. This API is technically deprecated and is missing some prototypes, but all major browsers depend on it, so it is unlikely to materially change without warning. This commit also ensures the netrc file is only written if the derivation is in fact meant to be able to access the internet. This change commits a sin of not actually actively declaring its dependency on macOS's libsandbox.dylib; this is due to the dylib cache in macOS making that explicit dependency unnecessary. In the future this might become a problem, so this commit marks our sins. Co-authored-by: Artemis Tosini <lix@artem.ist> Co-authored-by: Lunaphied <lunaphied@lunaphied.me> Change-Id: Ia302141a53ce7b0327c1aad86a117b6645fe1189
2024-04-27Merge "docs(nix-env): summarize of each subcommand in --help" into mainQyriad
2024-04-27Fix progress bar on copyPathsPuck Meerburg
This variable should not be shared between activities. Change-Id: I4eee89bc7acb320a3972dc3a55bfb087d3a9eb3a
2024-04-27libstore/ssh: shutK900
This is just logspam, and we have NIX_SSHOPTS for people that want the logspam. Change-Id: Ieff71473686f0661f9c53c212f8952dd2c9565c3
2024-04-26docs(nix-env): summarize of each subcommand in --helpQyriad
This should have been there from the beginning. As much as nix-env is a pile of problems we don't need trivial docs papercuts like this adding to it. Change-Id: I0c53e4b146af2fefdd0e4743d850672729cb2194
2024-04-26Merge "ssh-ng: Set log-fd for ssh to `4` by default" into mainMaximilian Bosch
2024-04-26Merge "gc: Find roots using libproc on Darwin" into mainArtemis Tosini
2024-04-26ssh-ng: Set log-fd for ssh to `4` by defaultMaximilian Bosch
That's expected by `build-remote` and makes sure that errors are correctly forwarded to the user. For instance, let's say that the host-key of `example.org` is unknown and nix-build ../nixpkgs -A hello -j0 --builders 'ssh-ng://example.org' is issued, then you get the following output: cannot build on 'ssh-ng://example.org?&': error: failed to start SSH connection to 'example.org' Failed to find a machine for remote build! derivation: yh46gakxq3kchrbihwxvpn5bmadcw90b-hello-2.12.1.drv required (system, features): (x86_64-linux, []) 2 available machines: [...] The relevant information (`Host key verification failed`) ends up in the daemon's log, but that's not very obvious considering that the daemon isn't very chatty normally. This can be fixed - the same way as its done for legacy-ssh - by passing fd 4 to the SSH wrapper. Now you'd get the following error: cannot build on 'ssh-ng://example.org': error: failed to start SSH connection to 'example.org': Host key verification failed. Failed to find a machine for remote build! [...] ...and now it's clear what's wrong. Please note that this is won't end up in the derivation's log. For previous discussion about this change see https://github.com/NixOS/nix/pull/7659. Change-Id: I5790856dbf58e53ea3e63238b015ea06c347cf92
2024-04-26filetransfer: don't decompress in curl wrapper itselfeldritch horrors
only decompress the response once all data has been received (in the fully buffered case), or at least outside of the curl wrapper itself (in the receive-to-sink case). unfortunately this means we will have to duplicate decompression logic for these two cases for time being, but once the curl wrapper has been rewritten to return a real future or Source we can deduplicate this logic again. the curl wrapper will have to turn into a proper Source first and use decompression source logic which also does not currently exist—only decompression *sinks* Change-Id: I66bc692f07d9b9e69fe10689ee73a2de8d65e35c
2024-04-26filetransfer: remove dataCallback from interfaceeldritch horrors
this is highly questionable. single-arg download calls will misbehave with it set, and two-arg download calls will just overwrite it. being an implementation detail this should not have been in the API at all. Change-Id: I613772951ee03d8302366085f06a53601d13f132
2024-04-26filetransfer: make two-arg download abstracteldritch horrors
this lets each implementation of FileTransfer (of which currently only the one exists at all) implement appropriate handling for its internal behaviours that are not otherwise exposed. in curl this lets us switch the buffer-full handling method from "block the entire curl thread" to "pause just the one transfer", move the non-libcurl body decompression out of the actual curl wrapper (which will let us eventually morph the curl wrapper intto an actual source of Sources), and some other things Change-Id: Id6d3593cde6b4915aab3e90a43b175c103cc3f18
2024-04-26Merge "justfile: allow passing args to `meson compile`" into mainMaximilian Bosch
2024-04-25gc: Find roots using libproc on DarwinArtemis Tosini
Previously, the garbage collector found runtime roots on Darwin by shelling out to `lsof -n -w -F n` then parsing the result. However, this requires an lsof binary and can be extremely slow. The official Apple lsof returns in a reasonable amount of time, about 250ms in my tests, but the lsof packaged in nixpkgs is quite slow, taking about 40 seconds to run the command. Using libproc directly is about the same speed as Apple lsof, and allows us to reënable several tests that were disabled on Darwin. Change-Id: Ifa0adda7984e13c15535693baba835aae79a3577
2024-04-25justfile: allow passing args to `meson compile`Maximilian Bosch
My main motivation for this change is to limit the amount of compile jobs to make sure my machine is still usable for something else when building a fresh Lix locally. Also made `build` a dependency of `install`: this is analogous to `make install` in CppNix where this both recompiles changed files and installs the artifacts into `outputs/out`. May be a little more pleasant to work with that, especially when you're used to contributing to CppNix. Change-Id: I321e2b0daf1c5e20f82c04e2dd158056c80ed86c
2024-04-25filetransfer: drop errorSinkeldritch horrors
just accumulate error data into result.data as we would for successful transfers without a dataCallback. errorSink and data would contain the same data in error cases anyway, so splitting them is not very useful. Change-Id: I00e449866454389ac6a564ab411c903fd357dabf
2024-04-25filetransfer: restore http status line reportingeldritch horrors
this was broken in 75b62e52600a44b42693944b50638bf580a2c86e. Change-Id: If8583e802afbcde822623036bf41a9708fbc7c8d
2024-04-25filetransfer: remove decompress request parametereldritch horrors
this is never read. Change-Id: I4c46f140519843a21e452958900e81edd2f78be2
2024-04-24Merge "libstore: Create platform LocalStore subclasses" into mainArtemis Tosini
2024-04-23meson: remove unnecessary parts of cross fileQyriad
Meson cross files layer, the last value of each key takes effect. https: //mesonbuild.com/Machine-files.html#loading-multiple-machine-files Change-Id: I22d886f71cd51f0ce520d3fc22aed4bcf074bb91
2024-04-23libstore: Create platform LocalStore subclassesArtemis Tosini
This creates new subclasses of LocalStore for each OS to include platform-specific functionality. Currently this just includes garbage collector roots but it could be extended to sandboxing as well. In order to make sure that the generic LocalStore is not accidentally constructed, its constructor is protected. A Fallback is provided which implements no functionality except constructors. Change-Id: I836a28e90b68309873f75afb83e0f1b2e2c89fb3
2024-04-23Merge changes Ia3e7b1e6,If09be814 into mainQyriad
* changes: meson: flip the switch!! meson: fix cross compilation
2024-04-22meson: flip the switch!!Qyriad
This commit makes Meson the default buildsystem for Lix. The Make buildsystem is now deprecated and will be removed soon, but has not yet, which will be done in a later commit when all seems good. The mesonBuild jobs have been removed, and have not been replaced with equivalent jobs to ensure the Make buildsystem still works. The full, new commands in a development shell are: $ meson setup ./build "--prefix=$out" $mesonFlags (A simple `meson setup ./build` will also build, but will do a different thing, not having the settings from package.nix applied.) $ meson compile -C build $ meson test -C build --suite=check $ meson install -C build $ meson test -C build --suite=installcheck (Check and installcheck may both be done after install, allowing you to omit the --suite argument entirely, but this is the order package.nix runs them in.) If tests fail and Meson helpfully has no output for why, use the `--print-error-logs` option to `meson test`. Why this is not the default I cannot explain. If you change a setting in the buildsystem, most cases will automatically regenerate the Meson configuration, but some cases, like trying to build a specific target whose name is new to the buildsystem (e.g. `meson compile -C build src/libmelt/libmelt.dylib`, when `libmelt.dylib` did not exist as a target the last time the buildsystem was generated), then you can reconfigure using new settings but existing options, and only recompiling stuff affected by the changes: $ meson setup --reconfigure build Note that changes to the default values in `meson.options` or in the `default_options :` argument to project() are NOT propagated with `--reconfigure`. If you want a totally clean build, you can use: $ meson setup --wipe build That will work regardless of if `./build` exists or not. Specific, named targets may be addressed in `meson build -C build <target>` with the "target ID" if there is one, which is the first string argument passed to target functions that have one, and unrelated to the variable name, e.g.: libexpr_dylib = library('nixexpr', …) can be addressed with: $ meson compile -C build nixexpr All targets may be addressed as their output, relative to the build directory, e.g.: $ meson compile -C build src/libexpr/libnixexpr.so But Meson does not consider intermediate files like object files targets. To build a specific object file, use Ninja directly and specify the output file relative to the build directory: $ ninja -C build src/libexpr/libnixexpr.so.p/nixexpr.cc.o To inspect the canonical source of truth on what the state of the buildsystem configuration is, use: $ meson introspect Have fun! Change-Id: Ia3e7b1e6fae26daf3162e655b4ded611a5cd57ad
2024-04-22meson: fix cross compilationQyriad
This should fix cross compilation in the base case, but this is difficult to test as cross compilation is broken in many different places right now. This should bring Meson back up to cross parity with the Make buildsystem though. Change-Id: If09be8142d1fc975a82b994143ff35be1297dad8
2024-04-23libstore: use curl functions for reading headerseldritch horrors
don't reimplement header parsing. this was only really needed due to the ancient github bug we no longer care about, everything else we have done in custom code can also be done using curl itself. doing this also fixes possible sources of header smuggling (because the header function didn't unfold headers and we'd trim them before parsing, which would've made us read contents of one header as a fully formed header in itself). this is a slight behavior change because we now honor only the first instance of a given header where previous behavior was to honor either the last or a combination of all of them (accept-ranges was logical-or'd by accident). Change-Id: I93cb93ddb91ab98c8991f846014926f6ef039fdb
2024-04-23libstore: remove github etag workaroundeldritch horrors
this was a workaround for a *github* bug that happend *in 2015*. not only is github no longer buggy, it shouldn't have been nix's responsibility to work around these bugs like this to begin with while we're at it we'll also remove another workaround—again for github specifically and again for etag handling—from 2021 that's also not needed any more. future workarounds for serverside bugs should probably come with an expiration date that mutates into a build warning after a while, otherwise this *will* happen again. Change-Id: I74f739ae3e36d40350f78bebcb5869aa8cc9adcd
2024-04-22tests: add error messages to the asserts in tarball flakes testQyriad
In hopes of avoiding opaque error messages like the one in https://buildbot.lix.systems/#/builders/49/builds/1054/steps/1/logs/stdio Traceback (most recent call last): File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module> sys.exit(main()) ^^^^^^ File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/__init__.py", line 126, in main driver.run_tests() File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 159, in run_tests self.test_script() File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 151, in test_script exec(self.tests, symbols, None) File "<string>", line 13, in <module> AssertionError Change-Id: Idd2212a1c3714ce58c7c3a9f34c2ca4313eb6d55
2024-04-22libstore: use curl_multi_{poll,wakeup}eldritch horrors
the previous solution to the wakeup problem (adding a pipe and passing it as an additional fd to curl_multi_wait) worked, but there have been builtin alternatives for this since 2020. not only do these save code, they're also a lot more likely to work natively on windows when needed Change-Id: Iab751b900997110a8d15de45ea3ab0c42f7e5973
2024-04-22libstore: remove ancient libcurl feature checkseldritch horrors
the oldest version checked for here is 7.47, which was released in 2016. it's probably safe to say that we do not need these any more Change-Id: I003411f6b2ce6d56f7ca337390df3ea86bd59a99
2024-04-21Merge "Fix exportReferencesGraph when given store subpath" into mainPuck Meerburg
2024-04-21remove extraneous cache entry from github fetcherPuck Meerburg
This isn't necessary, as it's already covered by the tarball fetcher's cache. Change-Id: I85e35f5a61594f27b8f30d82145f92c5d6559e1f
2024-04-21Fix exportReferencesGraph when given store subpathAlyssa Ross
With Nix 2.3, it was possible to pass a subpath of a store path to exportReferencesGraph: with import <nixpkgs> {}; let hello = writeShellScriptBin "hello" '' echo ${toString builtins.currentTime} ''; in writeClosure [ "${hello}/bin/hello" ] This regressed with Nix 2.4, with a very confusing error message, that presumably indicates it was unintentional: error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store (cherry picked from commit 0774e8ba33c060f56bad3ff696796028249e915a) Change-Id: I00920fb33077b831a1bb4a1b68d515ba8c3c2a69
2024-04-18meson: correctly embed sandbox shell when askedQyriad
Change-Id: I2f6c0d42245204a516d2e424eea26a6391e975ad
2024-04-18tests: unhaunt the flakes nixos testseldritch horrors
these should really wait for networks to come up, otherwise they can fail. fixes #235 Change-Id: I08989e8bdb0de280df74660ac43983de5c34fa9d
2024-04-18meson: embed source paths as relative to the source root and avoid ../srcQyriad
Change-Id: Ifab83cb7a3bfde717a4d6032ede8be75dc61f2b1
2024-04-18meson: correctly set -DSANDBOX_SHELL if we have itQyriad
The statically embedded busybox is not required for Lix to work, but package.nix explicitly sets this, which was accidentally being ignored. Change-Id: Ieeff830ac7d1f5fabe84d1a6cfd82f13d79035bf
2024-04-17Merge "build internal API docs with Meson" into mainQyriad
2024-04-16Merge "libstore/build: set NO_NEW_PRIVS for the sandbox" into mainIlya K
2024-04-15build internal API docs with MesonQyriad
This commit adds the capability for building the Doxygen internal API docs in the Meson buildsystem, and also makes doing so the default for the internal-api-docs hydra job. Aside from the /nix-support directory, which differed only by the hash part of a store path, the outputs of hydraJobs.internal-api-docs before and after this commit were bit-for-bit identical on my machine. Change-Id: I98f0017891c25b06866c15f7652fe74f706ec8e1
2024-04-15fix probable format bug in DerivationGoal::buildDoneQyriad
Either the contents of `line` could cause format errors, or this usage is Technically safe. However, I trust nothing, especially with boost::format. Change-Id: I07933b20bde3b305a6e5d61c2a7bab6ecb042ad9
2024-04-15abort with a descriptive message on bad HintFmt usageQyriad
Change-Id: Ic2f05572042343a8160fd971394372f5f2706fc4
2024-04-15libstore/build: set NO_NEW_PRIVS for the sandboxK900
Change-Id: I711f64e2b68495ed9c85c1a4bd5025405805e43a