Mastering Nixpkgs Overlays: Techniques and Best Practice
Our last post prepared you for laziness in Nix. This time, we shift up a gear and apply it to Nix overlays, one of the most powerful mechanisms in nixpkgs. Overlays make projects truly composable and make it easy to quickly manipulate packages for different variants of a package or system image build.
The NixOS wiki has a dedicated page on overlays and thereâs also the âDoâs and Donâts or nixpkgs overlaysâ article from 2017. This article aims to answer all the questions I have been asked regularly by customers and also provide guidelines for potential gray area questions.
For everyone who wonders what overlays are for, we look at different kinds of problems that overlays solve. Afterward, we demonstrate how to install them into package trees and system images. At last, we show some positive and negative examples along with explanations.
What Problem Do Overlays Solve?
Every package from the pkgs
attribute set is defined in the nixpkgs
project folder.
The nixpkgs project can in that regard be seen as a huge monorepo of package descriptions and pinned references to third-party open-source repositories.
We could fork this repository and perform all the changes we like. We could add our own patches to random packages, upgrade/downgrade what we like, etc. For quick experiments, this is not a bad idea - but growing our patches on top of nixpkgs quickly becomes unhandy when we drift away from the original nixpkgs with its (very!) frequent updates.
Even if nixpkgs would not change for some time, patching the repository does also not scale well if you want to build different variants of your packages or systems: Many projects build production and debug variants of their packages and NixOS system images. Some of those are provided as downloads for consumers, but before that happens every release, developers need debuggable versions of the packages and systems, and/or instrumented versions for fuzzing, and/or automated testing. How would you express multiple variants of different packages in different variants of the same product/image?
Enter overlays: An overlay is a basic Nix function that describes the changes that we would like to lay over an existing package collection. Different product or system image variants can be built with the same description but different overlays. This way, overlays represent a freely exchangeable and composable parameter to your system build.
What Do Overlays Look Like?
Imagine we have a full NixOS system deployment or image that runs a few applications like this, where different applications to some extent share the same library dependencies:
Letâs look at two different ways of patching this system with overlays. Afterward, we will learn about the different ways of installing an overlay.
Overriding a Package for All Consumers
Letâs say we would like to patch the underlying library A for all the applications.
On a Debian/Ubuntu/Fedora/etc. GNU/Linux system, this is not considered a big challenge:
Libraries are in /usr/lib64/...
or similar and could simply be replaced.
In NixOS, there is typically no such folder because binaries reference libraries directly from the Nix store. To change these references, we could write an overlay function that looks like this:
# file overlay-system-1.nix
final: prev: {
library-a = prev.library-a.overrideAttrs (...);
}
Please note that we are going to use override
and overrideAttrs
in this article and not explain it a lot (this will happen in another article),
as we concentrate on overlay functions today.
These are the standard per-package override functions in nixpkgs and are documented here.
The override in this example uses (...)
as a placeholder and would not work if you copy and pasted it as it is.
All following examples in this article are meant to be tried at home and will work.
This overlay function accepts two parameters and returns an attribute set.
This attribute set is laid over the initial pkgs
attribute set and this way, it extends it.
If there already was a library-a
attribute in pkgs
, it would be overwritten.
If there wasnât, it would be added.
We assume our fantasy package library-a
to exist already, so it would be overwritten:
When we say foo = prev.foo....;
, then this means that we reference the existing package foo
and then do something with it.
prev
is the previous version of pkgs
without our changes.
final
is similar, but we will get at this detail later in this article.
In the example, applying the overlay to the package set will make every package that referenced the unpatched library-a
package now
refer to the new one.
(In NixOS this can even lead to a huge recompilation effort. We will also mention later how that could be avoided.)
Imagine creating a patched .deb
package and manipulating the master image to get debugging symbols or profiling added to a product image:
That might be a lot more work than just injecting a one-liner overlay.
Overriding a Package for Specific Consumers
In a different scenario, we might like to patch libraries for just one application (e.g. for profiling, debugging, fuzzing, âŠ), although other applications consume them, too:
This would be quite a bit more tedious in legacy distros but turns out to be very easy in Nix(OS). Letâs see what the overlay function has to look like to achieve this:
# file overlay-system-2.nix
final: prev: {
app-c = prev.app-c.override {
library-b = prev.library-b.override {
library-a = prev.library-a.overrideAttrs (...);
};
};
}
This overlay is similar but different:
It does not override the library-a
package symbol for everyone.
Library A is patched in exactly the same way as before, but the way it is injected into the package tree is different:
We only override the package app-c
, which represents Application C from the diagram.
The way we override it is that we only replace its input library-b
with a new variant of Library B.
This variant of Library B is in turn manipulated in a way that it receives a patched version of Library A.
So What Do final
and prev
Mean?
An overlay function accepts two parameters:
prev
is the version of nixpkgsâspkgs
attribute before applying the overlay.final
is the final resultingpkgs
attribute after applying all the overlays.
Please note that by earlier convention, final
was once self
and prev
was once super
.
The words final
and prev
are broadly considered more comprehensible.
The command flake check
even enforces final
and prev
.
Finally, the function returns an attribute set that is laid over the previous version of pkgs
, which explains why itâs called âoverlayâ.
This diagram helps develop an intuition of how the pkgs
attribute is constructed from nixpkgs and overlays, and how the
final
and prev
parameters play together.
The //
operator in the boxes at the bottom is Nixâs update operator.
This is where the package collection is âupdatedâ with the changes from an overlay.
When Shall I Use prev
or final
?
Later in the article, we will demonstrate some common pitfalls and also what the errors look like when doing it wrong. But letâs first establish some rules that can be memorized easily.
After observing different discussions (for example this one and this one), I have seen two flavors of rules that community members typically adhere to:
The Graft-Preserving Rules Flavor
General rules of thumb:
- Use
prev
by default. - Use
final
only if you reference a package/derivation from some other package.
One advantage is that this set of rules avoids unnecessary round trips over the âlongerâ final
calls.
This strategy also enables the efficient implementation of the so-called grafting feature, which historically should have been introduced by this pull request: When an overlay contains critical security updates, the patched packages and all depending packages would have to be rebuilt, which might take too long with given build infrastructure. The grafting feature would allow a rebuild of only the patched packages without the need for a mass rebuild of all other dependent packages. Instead of rebuilding depending packages, it would patch their dynamic library path references, which is much faster. Unfortunately, this work seems to be discontinued at this time. The recompilation times are real, but they donât seem to be enough of a burden for the majority of Nix users.
There is more information in Nicholas Pierronâs talk (slides) and this blog post about faster security updates.
The Everything-Is-Overridable Rules Flavor
- Use
final
by default. - Use
prev
if you override a symbol to avoid infinite recursion.
The advantage of this set of rules is that every symbol becomes overridable, even functions.
Following the logic of this ruleset, we would for example write foo = final.callPackage ./foo { };
(following the other ruleset, we would write prev.callPackage
because callPackage
is not a package).
This way, if a later overlay overrides the callPackage
symbol, earlier final.callPackage
calls would be using the override.
Which rule set to choose?
Both rule sets prevent infinite recursion, which is the most important concern. The other concerns seem like a matter of project requirements and taste, especially with the inactivity of the mentioned grafting/security-update pull request. None of these rule sets are wrong. Ideally, pick one rule set and stick to it in your projects.
Using Overlays
Even with the diagram, this is very theoretical and also potentially confusing. đ€Ż So letâs start with very concrete examples that you can try yourself. This will also add some intuition to these rules.
Preparation
If you donât have a nixpkgs checkout on disk, letâs check out nixpkgs somewhere locally:
git clone --depth 1 https://github.com/nixos/nixpkgs
The --depth 1
parameter accelerates the git checkout by not downloading the entire history.
I usually have a checkout of nixpkgs at ~/src/nixpkgs
and assume this path for the rest of this article.
An overlay cannot be âopenedâ with Nix commands directly. Overlays are injected into the nixpkgs parameter object during the import of nixpkgs:
pkgs = import ~/src/nixpkgs {
overlays = [
someOverlayFunction
];
config = { };
}
We donât necessarily need the config = { };
part.
Itâs included in this example because this is best practice:
When left empty, it might be filled with (impure) default values.
This can be avoided with an empty config object.
You probably noticed that the overlays
parameter accepts a list of overlay functions.
We can apply as many overlays as we want.
The Nixpkgs documentation of course explains this with different detail, too.
We use this example overlay file in the following snippets:
# file: overlay.nix
final: prev: {
hello-script = prev.writeShellScriptBin "hello" "echo Hello World!";
}
In the REPL
If you enter this in the Nix REPL, the resulting pkgs
object contains the changes from the overlay function:
$ nix repl
Welcome to Nix 2.18.2. Type :? for help.
nix-repl> pkgs = import ~/src/nixpkgs {
overlays = [ (import ./overlay.nix ) ];
}
nix-repl> :b pkgs.hello-script
...
This is an easy way to play with overlays:
The package pkgs.hello-script
would not be available without the overlay.
Please note that whenever you change imported files in your text editor and then replay your
Nix expressions in the REPL, the files are not reloaded automatically.
Make sure to use :r
to reload regularly.
With Flakes
This is how to install an overlay into a Nix Flake:
{
description = "Example flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs }:
let
system = "x86_64-linux"; # change this to your system string
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlays.default ];
config = { };
};
# The next two lines are alternative ways. Not prefered.
pkgs2 = nixpkgs.legacyPackages.${system}.extend self.overlays.default;
pkgs3 = nixpkgs.legacyPackages.${system}.appendOverlays [
self.overlays.default
];
in
{
packages.${system}.hello-script = pkgs.hello-script;
overlays.default = import ./overlay.nix;
};
}
Instead of using pkgs = nixpkgs.legacyPackages.${system};
as most Flakes do,
we import nixpkgs manually and inject the overlay function.
Note that we also export the overlay via the overlays
output attribute.
This way other flakes can consume our overlay with their own Nixpkgs selection and combine them with other overlays, too.
We can consume our own overlays via self.overlays....
.
Generally, the first way with the explicit import of nixpkgs
that results in the variable pkgs
is the preferred one.
The instantiations of pkgs2
and pkgs3
showcase alternative ways, but they lead to multiple instantiations of nixpkgs,
which might impact evaluation performance.
With Flake-Parts Flakes
With the flake.parts library (which is extremely useful in complex projects), overlays are installed differently:
{
description = "Overlays in Flake-parts Flakes";
inputs = {
flake-parts.url = "github:hercules-ci/flake-parts";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = inputs: inputs.flake-parts.lib.mkFlake { inherit inputs; } {
systems = [ "x86_64-linux" "aarch64-linux" "aarch64-darwin" "x86_64-darwin" ];
perSystem = { pkgs, system, ... }: {
_module.args.pkgs = import inputs.nixpkgs {
inherit system;
overlays = [ inputs.self.overlays.default ];
config = { };
};
packages.hello-script = pkgs.hello-script;
};
flake = {
overlays.default = import ./overlay.nix;
};
};
}
See also the flake.parts documentation.
Without Flakes
In traditional non-Flakes Nix expressions, importing nixpkgs manually is a more common task:
let
pkgs = import ./path/to/nixpkgs {
overlays = [ (import ./overlay.nix) ];
};
in
pkgs.hello-script
In a NixOS Configuration
Inside a NixOS configuration module, nixpkgs overlays are added to the nixpkgs.overlays
attribute path.
{ ... }:
{
nixpkgs.overlays = [ (import ./overlay.nix) ];
environment.systemPackages = [ pkgs.hello-script ];
}
Different modules can append overlays. To keep control of the order in which overlays are applied, it is advisable to keep the overlays list at a central place, as top-level as possible.
In a Home Manager Configuration
In home-manager configurations, overlays can be selected per-user because there are per-user home-manager configuration modules:
{ pkgs, ... }:
{
nixpkgs.overlays = [ (import ./overlay.nix) ];
home.packages = [ pkgs.hello-script ];
}
See also in the home-manager options reference.
Overlay Best Practice
Now we know what overlays generally are and also roughly understand how they work and how they are used. Itâs still a complicated topic and we might be confused by their possibilities. Letâs look at practical examples to familiarize ourselves with overlays.
Adding New Packages to Nixpkgs
Letâs write an overlay that adds a new package called hello-script
to the nixpkgs collection:
# file: overlay-hello.nix
final: prev: {
hello-script = prev.writeShellScriptBin "hello" "echo Hello World!";
}
As we already know, the overlay file itself cannot be run with Nix tools directly. Letâs use the REPL to check it out:
$ nix repl
Nix 2.23.1
Type :? for help.
nix-repl> overlayFunction = import ./overlay-hello.nix
nix-repl> pkgs = import ~/src/nixpkgs {
overlays = [ overlayFunction ];
}
nix-repl> :b pkgs.hello-script
This derivation produced the following outputs:
out -> /nix/store/nl5h9rfv0dcwl2wzk6zyggmw0wz7gy8p-hello
$ /nix/store/nl5h9rfv0dcwl2wzk6zyggmw0wz7gy8p-hello/bin/hello
Hello World!
The :b
REPL command builds a derivation.
(Use :?
to look up all available commands.)
Please note that after building the package, I pressed ctrl + d
to exit the Nix REPL.
In the vanilla nixpkgs, there is no such package as hello-script
.
With the overlay, it is suddenly available - and it looks like it was always there!
If we wrote an overlay for a local package function that is designed to be used with
the typical callPackage
pattern (its documentation is a bit scattered over the
nixpkgs docs), we could add it like this:
final: prev: {
myPackage = prev.callPackage ./path/to/default.nix { };
}
The best about this is that for everyone who gets access to the resulting pkgs
attribute,
it feels like our package is a first-class citizen of nixpkgs.
Changing Existing Packages
The GNU Hello project is packaged in Nixpkgs as pkgs.hello
.
It only prints âHello, world!â when invoked.
This makes it a perfect example app for patching.
Letâs patch it to print âHello, Nixcademy!â:
# file: overlay-hello-nixcademy.nix
final: prev: {
hello = prev.hello.overrideAttrs (oldAttrs: {
postPatch = ''
substituteInPlace src/hello.c --replace "world!" "Nixcademy!"
'';
doCheck = false;
});
}
What is crucial here is that we emit the pkgs
attribute set key hello
into the packages collection, although it already exists.
This is why we use prev.hello
to reference the original hello
package.
prev.hello
means âthe hello
attribute from before our overlayâ.
The script snippet in postPatch
substitutes all occurrences of the substring âworld!â to âNixcademy!â in the right source file of the packageâs repository.
(The function substituteInPlace
is part of the stdenv
and is documented here)
This would make the unit tests fail, so we disable them via the doCheck
attribute.
(In production code, we would of course fix the tests, too. đ»)
Letâs try if it works:
$ nix repl
Nix 2.23.1
Type :? for help.
nix-repl> pkgs = import ~/src/nixpkgs {
overlays = [ (import ./overlay-hello-nixcademy.nix) ];
}
nix-repl> :u pkgs.hello
$ hello
Hello, Nixcademy!
This time, we used the :u
command in the REPL instead of :b
:
It builds the package and then launches a Nix shell that has our new package in its PATH
,
so we can just run it without manually fiddling with the nix store paths as in the earlier example.
It works! đ
Infinite Recursion Errors from Overriding Symbols
Why again did we use prev
instead of final
when referencing the original hello
package?
Letâs use hello = final.hello...
instead of hello = prev.hello...
in the overlay function
and see what happens:
nix-repl> :u pkgs.hello
error:
⊠while evaluating the attribute 'hello'
at ~/src/nixpkgs/overlay-hello-nixcademy.nix:2:3:
1| final: prev: {
2| hello = final.hello.overrideAttrs (oldAttrs: {
| ^
3| postPatch = ''
⊠while evaluating the attribute 'hello.overrideAttrs'
at ~/src/nixpkgs/overlay-hello-nixcademy.nix:2:3:
1| final: prev: {
2| hello = final.hello.overrideAttrs (oldAttrs: {
| ^
3| postPatch = ''
error: infinite recursion encountered
at ~/src/nixpkgs/overlay-hello-nixcademy.nix:2:11:
1| final: prev: {
2| hello = final.hello.overrideAttrs (oldAttrs: {
| ^
3| postPatch = ''
Letâs draw the references in the diagram from the beginning of this post to understand what exactly happened here:
hello = final.hello...
references the hello
symbol that we just created, so this is why we are moving in a circle!
prev.hello
is the original hello
from nixpkgs (or the hello
from the last overlay, if multiple overlays override the same symbol).
We internalize the most important rule:
Always use prev
if you are overriding an existing symbol (i.e. without renaming it).
Donât worry, this rule is not an additional one to the earlier presented rule sets: You are already following it if you are following them.
Overriding Nested Attributes
Overriding existing symbols like packages or nix library functions, or adding new ones, is a bit more complicated if they are not
directly in pkgs
but nested in some sub-attribute set.
There are multiple ways:
- Overriding plain nested sets
- Overriding extensible nested sets
- Overriding scopes
Overriding Plain Nested Sets
This is for example how we could add a function pkgs.lib.plusOne
:
# file overlay-lib.nix
final: prev: {
lib = prev.lib or { } // {
plusOne = x: x + 1;
};
}
If we didnât use the update operator //
on the existing lib
set,
our overlay would drop all pre-existing symbols! đŠ
> nix repl
Nix 2.23.1
Type :? for help.
nix-repl> pkgs = import ~/src/nixpkgs {
overlays = [ (import ./overlay-lib.nix) ];
}
nix-repl> pkgs.lib.plusOne 10
Another more complex example:
Letâs put the function in pkgs.lib.subCategory.plusOne
:
# file overlay-lib-nested.nix
final: prev: {
lib = prev.lib or { } // {
subCategory = prev.lib.subCategory or { } // {}
plusOne = x: x + 1;
};
};
}
This pattern is not only interesting if we want to add library functions.
It could also be used if we put packages in subcategories like pkgs.myCompany.myProduct
or similar.
Overriding Extensible Nested Sets
It shall be mentioned that for the special case of extending pkgs.lib
, there is already a facility for this purpose:
# file overlay-lib2.nix
final: prev: {
lib = prev.lib.extend (libFinal: libPrev: {
plusOne = x: x + 1;
});
}
The extend
function inside lib
provides us another overlay mechanism in the overlay mechanism just for extending this sub-attribute set.
This is worthwhile to mention because if we ever end up wanting to make our own attribute sets overridable,
the function makeExtensible
is the way to provide exactly these mechanics.
Overriding Scopes
What is a scope?
The nested set pkgs.gnome
for example contains many gnome-related packages that reference each other.
This is slightly similar to the nested set of pkgs.lib
but this one is created using the makeScope
function (documentation).
One reason is that a scope also provides a specialized callPackage
function symbol that makes it easier to obtain
dependencies from inside the scope (otherwise the user of pkgs.callPackage
would have to write the nested paths out for each dependency).
No matter for what reason a scope has been created, it can be overridden like this:
final: prev: {
gnome = prev.gnome.overrideScope (gnomeFinal: gnomePrev: {
updateScript = gnomePrev.updateScript.override {
versionPolicy = "odd_unstable";
};
});
}
This example overlay would change the version policy in the GNOME package update script.
We have the same overlay-in-overlay pattern here as before, but this is the way to inject it when a package scope is in play.
Overriding Python Packages
Talking about extending nested attribute sets, this is also relevant for Python packages (and other language-specific sub-sets in nixpkgs). Letâs concentrate on Python now because Python is so popular and keep in mind that it works similarly for all the other languages.
The nixpkgs documentation has specific advice for most programming language specific package tooling. The Python language ecosystem in Nixpkgs provides two different ways.
This section is a great example of how fine-grained precision and control overlays give us. We can override Python packages for:
- All Python Package Set versions in
pkgs
(e.g.pkgs.python3Packages
,pkgs.python37Packages
, etc.) - Only a specific Python Package set
- We can place all overrides into a completely new isolated package set that we only use for certain packages
Overriding the Python Package Set for All Python Versions
This overlay creates a Python-specific overlay function and adds it to the global list pythonPackagesExtensions
.
The Python-specific overlay function in this example upgrades pytest
to a different version:
final: prev:
let
pytestOverlay = pythonFinal: pythonPrev: {
pytest = pythonPrev.pytest.overridePythonAttrs (oldAttrs: {
version = "8.3.2";
src = pythonPrev.fetchPypi {
pname = "pytest";
version = "8.3.2";
hash = "sha256-wTI0XRLOVRJCyHJp3oEkg/W8yHzbtHIuSEh7oZT5/c4=";
};
patches = []; # existing patch does not apply any longer
});
};
in
{
pythonPackagesExtensions = prev.pythonPackagesExtensions ++ [ pytestOverlay ];
}
This overlay affects all Python package sets for all versions of Python in nixpkgs.
Overriding a Specific Python Package Set
This example performs the same override on the package pytest
but applies it only to the package set
in pkgs.python3.pkgs
and pkgs.python3Packages
:
final: prev:
let
pytestOverlay = pythonFinal: pythonPrev: {
pytest = pythonPrev.pytest.overridePythonAttrs (oldAttrs: {
version = "8.3.2";
src = pythonPrev.fetchPypi {
pname = "pytest";
version = "8.3.2";
hash = "sha256-wTI0XRLOVRJCyHJp3oEkg/W8yHzbtHIuSEh7oZT5/c4=";
};
patches = []; # existing patch does not apply any longer
});
};
python3 = prev.python3.override {
packageOverrides = pytestOverlay;
};
in
{
inherit python3;
python3Packages = python3.pkgs;
}
We could even go further and let the overlay create new attributes like myPython3
and myPython3Packages
.
This way, the overlay would only have an effect on the projects that use myPython3Packages
while
all the other Python 3 projects would remain unaffected.
Bad Practice / Donât Do
This section lists some typical anti-patterns and explains why you should avoid them.
Donât Collect Overlays in ~/.config/nixpkgs/overlays/
We can put multiple overlay function files into the folder ~/.config/nixpkgs/overlays
and get them automatically applied to
pkgs
in non-Flakes Nix expressions and the REPL whenever we import nixpkgs without an explicit overlays
list.
(Implementation)
This is very handy if we end up using the same overlays over and over again, but itâs an impurity: Imagine some package or system description implicitly works because of your local overlay collection. Your teammates will get different characteristics and probably errors if you forget to synchronize with them. (Not having to deal with âworks for meâ situations is one of the main selling points of Nix!)
Donât Use rec
Attribute Sets
Imagine we create two packages of which one is the dependency of another. A bad way to do this would be:
final: prev: rec {
pkg-a = prev.callPackage ./a { };
pkg-b = prev.callPackage ./b { dependency-a = pkg-a; }
}
In this example, pkg-b
explicitly consumes pkg-a
as a dependency.
This works, but:
If someone else adds another overlay that somehow overrides and patches pkg-a
, pkgs-b
will not notice that.
pkg-b
will still use the original, unpatched pkg-a
.
How do we fix that? Like this:
final: prev: {
pkg-a = prev.callPackage ./a { };
pkg-b = prev.callPackage ./b { dependency-a = final.pkg-a; }
}
No matter how often pkg-a
gets overridden by other overlays, pkg-b
will always observe those changes.
Please note that if I didnât rename pkg-b
âs dependency on pkg-a
to dependency-a
, we could have
left the second callPackage
parameter empty, as it would be filled in automatically.
This choice is just to keep the example very simple so that we can focus on the anti-pattern at hand.
Donât Use External Parameters
Imagine, an overlay does something like this:
# file: overlay.nix
{ boost }:
final: prev: {
myPackage1 = prev.callPackage ./my-package2 { inherit boost; };
myPackage2 = prev.callPackage ./my-package2 { inherit boost; };
myPackage3 = prev.callPackage ./my-package3 { inherit boost; };
# ...
}
Such an overlay could be consumed like that:
# ...
pkgs = import ./path/to/nixpkgs {
overlays = [
(import ./overlay.nix { boost = pkgs.boost185; })
];
}
# ...
On the first look, this seems handy because we can âconfigureâ the overlay for different boost library versions. If many packages consume this parameter, it is âlockedâ in there and can only be overridden on each package individually: This anti-pattern breaks composability.
A better approach would be to create another package symbol and then reuse it:
# file: overlay.nix
final: prev: {
myBoostVersion = final.boost185;
myPackage1 = prev.callPackage ./my-package2 { boost = final.myBoostVersion; };
myPackage2 = prev.callPackage ./my-package2 { boost = final.myBoostVersion; };
myPackage3 = prev.callPackage ./my-package3 { boost = final.myBoostVersion; };
# ...
}
That way, we can import and change it in a later overlay like this:
# ...
pkgs = import ./path/to/nixpkgs {
overlays = [
(import ./overlay.nix)
(final: prev: { myBoostVersion = final.boost180; })
];
}
# ...
Beware When Importing Extra Nixpkgs in Overlays
Sometimes you might want to mix different versions of nixpkgs. This is not generally wrong but should be done with caution.
This example overlay tries to upgrade or downgrade a package by overwriting its symbol with the version of a pinned newer/older version of nixpkgs.
# file: overlay.nix
let
pkgs = import ./path/to/nixpkgs { };
in
final: prev: {
inherit (pkgs) somePackage;
}
This works in many cases, but:
- It pulls not only the older/newer package version into your packages tree but also all its dependencies.
- It breaks cross-compilation: If you build this package via
pkgs.pkgsCross....
(or a package that depends on it), it breaks.
An alternative way might be:
# file: overlay.nix
let
pkgs = import ./path/to/nixpkgs { };
in
final: prev: {
somePackage = prev.callPackage pkgs.somePackage.override { };
}
This way the packageâs package function is re-invoked with all the dependencies and also cross-compilation/linking settings of your current nixpkgs invocation. The older/newer package function might however not work with the newer/older dependencies, so these might need some specific overrides, too.
Donât Reference other packages via prev
If we want to override specific dependencies by default, we might do it like this C++ example project:
final: prev: {
myCppProject = prev.callPackage ./my-project {
stdenv = prev.clangStdenv;
boost = prev.boost185;
};
}
This way, the project builds with the Clang compiler (Nixpkgs selects GCC by default on Linux) and a specific boost version. It works, but:
If another overlay adds global patches or changes the version of the symbols clangStdenv
or boost185
, they will go unnoticed
by this package.
Better:
final: prev: {
myCppProject = prev.callPackage ./my-project {
stdenv = final.clangStdenv;
boost = final.boost185;
};
}
Summary
Overlays are complex, but powerful. Learning how to use them is well worth the effort, as they make projects and products (packages or full system images) composable and easily extensible. They are also an often overlooked feature of the Nix Flake era.
Especially if the users are also familiar with override
and overrideAttrs
, they can work wonders with entire system images.
Developers turn an image inside out for debugging purposes and donât even have to check out the project folders of patched projects
and reparameterized build systems.
In our Nix & NixOS 101 course, we have a great exercise that deals intensively with overlays only and has been proven to clear up any confusion with overlays. After the course, our participants are well-versed in overlays and use them productively in their everyday project work.
We also typically cover questions like:
- How should enterprise projects be structured to take full advantage of nixpkgs and overlays?
- What design guidelines/specifications should we use for our projects to allow for faster and easier testing and debugging, maintainable per-customer variants, cross-compilation, downstream testing, etc.?
- How can we move quickly with patched packages in a high-pressure deadline environment, but also simplify the upstreaming of generally useful changes so that we donât have to maintain an ever-growing list of patches?
- How do overlays help us create our own corporate GNU/Linux distribution for internal or product use?
If your team is too small for an onsite group training or you are an individual, give our public online training sessions a try!