From ef606760abd87c98371fbc08c1f25ad897823a2a Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Wed, 22 Jul 2020 23:17:48 +0200 Subject: Pandoc conversion --- doc/manual/src/advanced-topics/cores-vs-jobs.md | 42 +++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 doc/manual/src/advanced-topics/cores-vs-jobs.md (limited to 'doc/manual/src/advanced-topics/cores-vs-jobs.md') diff --git a/doc/manual/src/advanced-topics/cores-vs-jobs.md b/doc/manual/src/advanced-topics/cores-vs-jobs.md new file mode 100644 index 000000000..846b6356e --- /dev/null +++ b/doc/manual/src/advanced-topics/cores-vs-jobs.md @@ -0,0 +1,42 @@ +# Tuning Cores and Jobs + +Nix has two relevant settings with regards to how your CPU cores will be +utilized: [???](#conf-cores) and [???](#conf-max-jobs). This chapter +will talk about what they are, how they interact, and their +configuration trade-offs. + + - [???](#conf-max-jobs) + Dictates how many separate derivations will be built at the same + time. If you set this to zero, the local machine will do no builds. + Nix will still substitute from binary caches, and build remotely if + remote builders are configured. + + - [???](#conf-cores) + Suggests how many cores each derivation should use. Similar to `make + -j`. + +The [???](#conf-cores) setting determines the value of +NIX\_BUILD\_CORES. NIX\_BUILD\_CORES is equal to [???](#conf-cores), +unless [???](#conf-cores) equals `0`, in which case NIX\_BUILD\_CORES +will be the total number of cores in the system. + +The maximum number of consumed cores is a simple multiplication, +[???](#conf-max-jobs) \* NIX\_BUILD\_CORES. + +The balance on how to set these two independent variables depends upon +each builder's workload and hardware. Here are a few example scenarios +on a machine with 24 cores: + +| [???](#conf-max-jobs) | [???](#conf-cores) | NIX\_BUILD\_CORES | Maximum Processes | Result | +| --------------------- | ------------------ | ----------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | 24 | 24 | 24 | One derivation will be built at a time, each one can use 24 cores. Undersold if a job can’t use 24 cores. | +| 4 | 6 | 6 | 24 | Four derivations will be built at once, each given access to six cores. | +| 12 | 6 | 6 | 72 | 12 derivations will be built at once, each given access to six cores. This configuration is over-sold. If all 12 derivations being built simultaneously try to use all six cores, the machine's performance will be degraded due to extensive context switching between the 12 builds. | +| 24 | 1 | 1 | 24 | 24 derivations can build at the same time, each using a single core. Never oversold, but derivations which require many cores will be very slow to compile. | +| 24 | 0 | 24 | 576 | 24 derivations can build at the same time, each using all the available cores of the machine. Very likely to be oversold, and very likely to suffer context switches. | + +Balancing 24 Build Cores + +It is up to the derivations' build script to respect host's requested +cores-per-build by following the value of the NIX\_BUILD\_CORES +environment variable. -- cgit v1.2.3 From f3903035667e158112dfd414091d8d50ef90c5f4 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Thu, 23 Jul 2020 10:44:54 +0200 Subject: Reconvert --- doc/manual/src/advanced-topics/cores-vs-jobs.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'doc/manual/src/advanced-topics/cores-vs-jobs.md') diff --git a/doc/manual/src/advanced-topics/cores-vs-jobs.md b/doc/manual/src/advanced-topics/cores-vs-jobs.md index 846b6356e..91e95d5b4 100644 --- a/doc/manual/src/advanced-topics/cores-vs-jobs.md +++ b/doc/manual/src/advanced-topics/cores-vs-jobs.md @@ -16,18 +16,18 @@ configuration trade-offs. -j`. The [???](#conf-cores) setting determines the value of -NIX\_BUILD\_CORES. NIX\_BUILD\_CORES is equal to [???](#conf-cores), -unless [???](#conf-cores) equals `0`, in which case NIX\_BUILD\_CORES +`NIX_BUILD_CORES`. `NIX_BUILD_CORES` is equal to [???](#conf-cores), +unless [???](#conf-cores) equals `0`, in which case `NIX_BUILD_CORES` will be the total number of cores in the system. The maximum number of consumed cores is a simple multiplication, -[???](#conf-max-jobs) \* NIX\_BUILD\_CORES. +[???](#conf-max-jobs) \* `NIX_BUILD_CORES`. The balance on how to set these two independent variables depends upon each builder's workload and hardware. Here are a few example scenarios on a machine with 24 cores: -| [???](#conf-max-jobs) | [???](#conf-cores) | NIX\_BUILD\_CORES | Maximum Processes | Result | +| [???](#conf-max-jobs) | [???](#conf-cores) | `NIX_BUILD_CORES` | Maximum Processes | Result | | --------------------- | ------------------ | ----------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | 24 | 24 | 24 | One derivation will be built at a time, each one can use 24 cores. Undersold if a job can’t use 24 cores. | | 4 | 6 | 6 | 24 | Four derivations will be built at once, each given access to six cores. | @@ -38,5 +38,5 @@ on a machine with 24 cores: Balancing 24 Build Cores It is up to the derivations' build script to respect host's requested -cores-per-build by following the value of the NIX\_BUILD\_CORES +cores-per-build by following the value of the `NIX_BUILD_CORES` environment variable. -- cgit v1.2.3 From da3d776cb91123a4d0528251b7ce909419ca0c7a Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 24 Jul 2020 14:31:33 +0200 Subject: Fix some dangling references --- doc/manual/src/advanced-topics/cores-vs-jobs.md | 35 +++++++++++-------------- 1 file changed, 16 insertions(+), 19 deletions(-) (limited to 'doc/manual/src/advanced-topics/cores-vs-jobs.md') diff --git a/doc/manual/src/advanced-topics/cores-vs-jobs.md b/doc/manual/src/advanced-topics/cores-vs-jobs.md index 91e95d5b4..4a9058ca1 100644 --- a/doc/manual/src/advanced-topics/cores-vs-jobs.md +++ b/doc/manual/src/advanced-topics/cores-vs-jobs.md @@ -1,33 +1,32 @@ # Tuning Cores and Jobs -Nix has two relevant settings with regards to how your CPU cores will be -utilized: [???](#conf-cores) and [???](#conf-max-jobs). This chapter -will talk about what they are, how they interact, and their -configuration trade-offs. +Nix has two relevant settings with regards to how your CPU cores will +be utilized: `cores` and `max-jobs`. This chapter will talk about what +they are, how they interact, and their configuration trade-offs. - - [???](#conf-max-jobs) + - `max-jobs` Dictates how many separate derivations will be built at the same - time. If you set this to zero, the local machine will do no builds. - Nix will still substitute from binary caches, and build remotely if - remote builders are configured. + time. If you set this to zero, the local machine will do no + builds. Nix will still substitute from binary caches, and build + remotely if remote builders are configured. - - [???](#conf-cores) - Suggests how many cores each derivation should use. Similar to `make - -j`. + - `cores` + Suggests how many cores each derivation should use. Similar to + `make -j`. -The [???](#conf-cores) setting determines the value of -`NIX_BUILD_CORES`. `NIX_BUILD_CORES` is equal to [???](#conf-cores), -unless [???](#conf-cores) equals `0`, in which case `NIX_BUILD_CORES` -will be the total number of cores in the system. +The `cores` setting determines the value of +`NIX_BUILD_CORES`. `NIX_BUILD_CORES` is equal to `cores`, unless +`cores` equals `0`, in which case `NIX_BUILD_CORES` will be the total +number of cores in the system. The maximum number of consumed cores is a simple multiplication, -[???](#conf-max-jobs) \* `NIX_BUILD_CORES`. +`max-jobs` \* `NIX_BUILD_CORES`. The balance on how to set these two independent variables depends upon each builder's workload and hardware. Here are a few example scenarios on a machine with 24 cores: -| [???](#conf-max-jobs) | [???](#conf-cores) | `NIX_BUILD_CORES` | Maximum Processes | Result | +| `max-jobs` | `cores` | `NIX_BUILD_CORES` | Maximum Processes | Result | | --------------------- | ------------------ | ----------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | 24 | 24 | 24 | One derivation will be built at a time, each one can use 24 cores. Undersold if a job can’t use 24 cores. | | 4 | 6 | 6 | 24 | Four derivations will be built at once, each given access to six cores. | @@ -35,8 +34,6 @@ on a machine with 24 cores: | 24 | 1 | 1 | 24 | 24 derivations can build at the same time, each using a single core. Never oversold, but derivations which require many cores will be very slow to compile. | | 24 | 0 | 24 | 576 | 24 derivations can build at the same time, each using all the available cores of the machine. Very likely to be oversold, and very likely to suffer context switches. | -Balancing 24 Build Cores - It is up to the derivations' build script to respect host's requested cores-per-build by following the value of the `NIX_BUILD_CORES` environment variable. -- cgit v1.2.3 From 31313d1401ffe387e69b84c62ba8798bf23a8914 Mon Sep 17 00:00:00 2001 From: regnat Date: Fri, 23 Apr 2021 14:30:42 +0200 Subject: Replace the trailing markdown spaces by a backslash MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit They are equivalent according to , and the trailing spaces tend to be a pain (because the make git complain, editors tend to want to remove them − the `.editorconfig` actually specifies that − etc..). --- doc/manual/src/advanced-topics/cores-vs-jobs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/manual/src/advanced-topics/cores-vs-jobs.md') diff --git a/doc/manual/src/advanced-topics/cores-vs-jobs.md b/doc/manual/src/advanced-topics/cores-vs-jobs.md index 4a9058ca1..9e91ab9c7 100644 --- a/doc/manual/src/advanced-topics/cores-vs-jobs.md +++ b/doc/manual/src/advanced-topics/cores-vs-jobs.md @@ -4,13 +4,13 @@ Nix has two relevant settings with regards to how your CPU cores will be utilized: `cores` and `max-jobs`. This chapter will talk about what they are, how they interact, and their configuration trade-offs. - - `max-jobs` + - `max-jobs`\ Dictates how many separate derivations will be built at the same time. If you set this to zero, the local machine will do no builds. Nix will still substitute from binary caches, and build remotely if remote builders are configured. - - `cores` + - `cores`\ Suggests how many cores each derivation should use. Similar to `make -j`. -- cgit v1.2.3