Mastering Nixpkgs Overlays: Techniques and Best Practice

Mastering Nixpkgs Overlays: Techniques and Best Practice
📆 Thu Aug 01 2024 von Jacek Galowicz
(24 Min. Lesezeit)

Our last post prepared you for laziness in Nix. This time, we shift up a gear and apply it to Nix overlays, one of the most powerful mechanisms in nixpkgs. Overlays make projects truly composable and make it easy to quickly manipulate packages for different variants of a package or system image build.

The NixOS wiki has a dedicated page on overlays and there’s also the “Do’s and Don’ts or nixpkgs overlays” article from 2017. This article aims to answer all the questions I have been asked regularly by customers and also provide guidelines for potential gray area questions.

For everyone who wonders what overlays are for, we look at different kinds of problems that overlays solve. Afterward, we demonstrate how to install them into package trees and system images. At last, we show some positive and negative examples along with explanations.

What Problem Do Overlays Solve?

Every package from the pkgs attribute set is defined in the nixpkgs project folder. The nixpkgs project can in that regard be seen as a huge monorepo of package descriptions and pinned references to third-party open-source repositories.

We could fork this repository and perform all the changes we like. We could add our own patches to random packages, upgrade/downgrade what we like, etc. For quick experiments, this is not a bad idea - but growing our patches on top of nixpkgs quickly becomes unhandy when we drift away from the original nixpkgs with its (very!) frequent updates.

Even if nixpkgs would not change for some time, patching the repository does also not scale well if you want to build different variants of your packages or systems: Many projects build production and debug variants of their packages and NixOS system images. Some of those are provided as downloads for consumers, but before that happens every release, developers need debuggable versions of the packages and systems, and/or instrumented versions for fuzzing, and/or automated testing. How would you express multiple variants of different packages in different variants of the same product/image?

Enter overlays: An overlay is a basic Nix function that describes the changes that we would like to lay over an existing package collection. Different product or system image variants can be built with the same description but different overlays. This way, overlays represent a freely exchangeable and composable parameter to your system build.

What Do Overlays Look Like?

Imagine we have a full NixOS system deployment or image that runs a few applications like this, where different applications to some extent share the same library dependencies:

NixOS System Variant 1

Let’s look at two different ways of patching this system with overlays. Afterward, we will learn about the different ways of installing an overlay.

Overriding a Package for All Consumers

Let’s say we would like to patch the underlying library A for all the applications. On a Debian/Ubuntu/Fedora/etc. GNU/Linux system, this is not considered a big challenge: Libraries are in /usr/lib64/... or similar and could simply be replaced.

In NixOS, there is typically no such folder because binaries reference libraries directly from the Nix store. To change these references, we could write an overlay function that looks like this:

# file overlay-system-1.nix

final: prev: {
    library-a = prev.library-a.overrideAttrs (...);
}

Please note that we are going to use override and overrideAttrs in this article and not explain it a lot (this will happen in another article), as we concentrate on overlay functions today. These are the standard per-package override functions in nixpkgs and are documented here. The override in this example uses (...) as a placeholder and would not work if you copy and pasted it as it is. All following examples in this article are meant to be tried at home and will work.

This overlay function accepts two parameters and returns an attribute set. This attribute set is laid over the initial pkgs attribute set and this way, it extends it. If there already was a library-a attribute in pkgs, it would be overwritten. If there wasn’t, it would be added.

We assume our fantasy package library-a to exist already, so it would be overwritten: When we say foo = prev.foo....;, then this means that we reference the existing package foo and then do something with it. prev is the previous version of pkgs without our changes. final is similar, but we will get at this detail later in this article.

In the example, applying the overlay to the package set will make every package that referenced the unpatched library-a package now refer to the new one. (In NixOS this can even lead to a huge recompilation effort. We will also mention later how that could be avoided.) Imagine creating a patched .deb package and manipulating the master image to get debugging symbols or profiling added to a product image: That might be a lot more work than just injecting a one-liner overlay.

Overriding a Package for Specific Consumers

In a different scenario, we might like to patch libraries for just one application (e.g. for profiling, debugging, fuzzing, 
), although other applications consume them, too:

NixOS System Variant 2

This would be quite a bit more tedious in legacy distros but turns out to be very easy in Nix(OS). Let’s see what the overlay function has to look like to achieve this:

# file overlay-system-2.nix

final: prev: {
    app-c = prev.app-c.override {
        library-b = prev.library-b.override {
            library-a = prev.library-a.overrideAttrs (...);
        };
    };
}

This overlay is similar but different: It does not override the library-a package symbol for everyone. Library A is patched in exactly the same way as before, but the way it is injected into the package tree is different: We only override the package app-c, which represents Application C from the diagram. The way we override it is that we only replace its input library-b with a new variant of Library B. This variant of Library B is in turn manipulated in a way that it receives a patched version of Library A.

So What Do final and prev Mean?

An overlay function accepts two parameters:

Please note that by earlier convention, final was once self and prev was once super. The words final and prev are broadly considered more comprehensible. The command flake check even enforces final and prev.

Finally, the function returns an attribute set that is laid over the previous version of pkgs, which explains why it’s called “overlay”.

This diagram helps develop an intuition of how the pkgs attribute is constructed from nixpkgs and overlays, and how the final and prev parameters play together.

Final and prev in nixpkgs overlays

The // operator in the boxes at the bottom is Nix’s update operator. This is where the package collection is “updated” with the changes from an overlay.

When Shall I Use prev or final?

Later in the article, we will demonstrate some common pitfalls and also what the errors look like when doing it wrong. But let’s first establish some rules that can be memorized easily.

After observing different discussions (for example this one and this one), I have seen two flavors of rules that community members typically adhere to:

The Graft-Preserving Rules Flavor

General rules of thumb:

  1. Use prev by default.
  2. Use final only if you reference a package/derivation from some other package.

One advantage is that this set of rules avoids unnecessary round trips over the “longer” final calls.

This strategy also enables the efficient implementation of the so-called grafting feature, which historically should have been introduced by this pull request: When an overlay contains critical security updates, the patched packages and all depending packages would have to be rebuilt, which might take too long with given build infrastructure. The grafting feature would allow a rebuild of only the patched packages without the need for a mass rebuild of all other dependent packages. Instead of rebuilding depending packages, it would patch their dynamic library path references, which is much faster. Unfortunately, this work seems to be discontinued at this time. The recompilation times are real, but they don’t seem to be enough of a burden for the majority of Nix users.

There is more information in Nicholas Pierron’s talk (slides) and this blog post about faster security updates.

The Everything-Is-Overridable Rules Flavor

  1. Use final by default.
  2. Use prev if you override a symbol to avoid infinite recursion.

The advantage of this set of rules is that every symbol becomes overridable, even functions.

Following the logic of this ruleset, we would for example write foo = final.callPackage ./foo { }; (following the other ruleset, we would write prev.callPackage because callPackage is not a package). This way, if a later overlay overrides the callPackage symbol, earlier final.callPackage calls would be using the override.

Which rule set to choose?

Both rule sets prevent infinite recursion, which is the most important concern. The other concerns seem like a matter of project requirements and taste, especially with the inactivity of the mentioned grafting/security-update pull request. None of these rule sets are wrong. Ideally, pick one rule set and stick to it in your projects.

Using Overlays

Even with the diagram, this is very theoretical and also potentially confusing. đŸ€Ż So let’s start with very concrete examples that you can try yourself. This will also add some intuition to these rules.

Preparation

If you don’t have a nixpkgs checkout on disk, let’s check out nixpkgs somewhere locally:

git clone --depth 1 https://github.com/nixos/nixpkgs

The --depth 1 parameter accelerates the git checkout by not downloading the entire history.

I usually have a checkout of nixpkgs at ~/src/nixpkgs and assume this path for the rest of this article.

An overlay cannot be “opened” with Nix commands directly. Overlays are injected into the nixpkgs parameter object during the import of nixpkgs:

pkgs = import ~/src/nixpkgs { 
  overlays = [
    someOverlayFunction
  ];
  config = { };
}

We don’t necessarily need the config = { }; part. It’s included in this example because this is best practice: When left empty, it might be filled with (impure) default values. This can be avoided with an empty config object.

You probably noticed that the overlays parameter accepts a list of overlay functions. We can apply as many overlays as we want.

The Nixpkgs documentation of course explains this with different detail, too.

We use this example overlay file in the following snippets:

# file: overlay.nix

final: prev: {
  hello-script = prev.writeShellScriptBin "hello" "echo Hello World!";
}

In the REPL

If you enter this in the Nix REPL, the resulting pkgs object contains the changes from the overlay function:

$ nix repl
Welcome to Nix 2.18.2. Type :? for help.

nix-repl> pkgs = import ~/src/nixpkgs { 
  overlays = [ (import ./overlay.nix ) ]; 
}

nix-repl> :b pkgs.hello-script
...

This is an easy way to play with overlays: The package pkgs.hello-script would not be available without the overlay.

Please note that whenever you change imported files in your text editor and then replay your Nix expressions in the REPL, the files are not reloaded automatically. Make sure to use :r to reload regularly.

With Flakes

This is how to install an overlay into a Nix Flake:

{
  description = "Example flake";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }:
    let
      system = "x86_64-linux"; # change this to your system string
      pkgs = import nixpkgs {
        inherit system;
        overlays = [ self.overlays.default ];
        config = { };
      };
      # The next two lines are alternative ways. Not prefered.
      pkgs2 = nixpkgs.legacyPackages.${system}.extend self.overlays.default;
      pkgs3 = nixpkgs.legacyPackages.${system}.appendOverlays [ 
        self.overlays.default 
      ];
    in
    {
      packages.${system}.hello-script = pkgs.hello-script;

      overlays.default = import ./overlay.nix;
    };
}

Instead of using pkgs = nixpkgs.legacyPackages.${system}; as most Flakes do, we import nixpkgs manually and inject the overlay function.

Note that we also export the overlay via the overlays output attribute. This way other flakes can consume our overlay with their own Nixpkgs selection and combine them with other overlays, too. We can consume our own overlays via self.overlays.....

Generally, the first way with the explicit import of nixpkgs that results in the variable pkgs is the preferred one. The instantiations of pkgs2 and pkgs3 showcase alternative ways, but they lead to multiple instantiations of nixpkgs, which might impact evaluation performance.

With Flake-Parts Flakes

With the flake.parts library (which is extremely useful in complex projects), overlays are installed differently:

{
  description = "Overlays in Flake-parts Flakes";

  inputs = {
    flake-parts.url = "github:hercules-ci/flake-parts";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs = inputs: inputs.flake-parts.lib.mkFlake { inherit inputs; } {
    systems = [ "x86_64-linux" "aarch64-linux" "aarch64-darwin" "x86_64-darwin" ];
    perSystem = { pkgs, system, ... }: {
      _module.args.pkgs = import inputs.nixpkgs {
        inherit system;
        overlays = [ inputs.self.overlays.default ];
        config = { };
      };

      packages.hello-script = pkgs.hello-script;
    };

    flake = {
      overlays.default = import ./overlay.nix;
    };
  };
}

See also the flake.parts documentation.

Without Flakes

In traditional non-Flakes Nix expressions, importing nixpkgs manually is a more common task:

let
  pkgs = import ./path/to/nixpkgs {
    overlays = [ (import ./overlay.nix) ];
  };
in

pkgs.hello-script

In a NixOS Configuration

Inside a NixOS configuration module, nixpkgs overlays are added to the nixpkgs.overlays attribute path.

{ ... }: 
{
  nixpkgs.overlays = [ (import ./overlay.nix) ];

  environment.systemPackages = [ pkgs.hello-script ];
}

Different modules can append overlays. To keep control of the order in which overlays are applied, it is advisable to keep the overlays list at a central place, as top-level as possible.

In a Home Manager Configuration

In home-manager configurations, overlays can be selected per-user because there are per-user home-manager configuration modules:

{ pkgs, ... }: 
{
  nixpkgs.overlays = [ (import ./overlay.nix) ];

  home.packages = [ pkgs.hello-script ];
}

See also in the home-manager options reference.

Overlay Best Practice

Now we know what overlays generally are and also roughly understand how they work and how they are used. It’s still a complicated topic and we might be confused by their possibilities. Let’s look at practical examples to familiarize ourselves with overlays.

Adding New Packages to Nixpkgs

Let’s write an overlay that adds a new package called hello-script to the nixpkgs collection:

# file: overlay-hello.nix

final: prev: {
  hello-script = prev.writeShellScriptBin "hello" "echo Hello World!";
}

As we already know, the overlay file itself cannot be run with Nix tools directly. Let’s use the REPL to check it out:

$ nix repl
Nix 2.23.1
Type :? for help.
nix-repl> overlayFunction = import ./overlay-hello.nix

nix-repl> pkgs = import ~/src/nixpkgs { 
  overlays = [ overlayFunction ]; 
}

nix-repl> :b pkgs.hello-script

This derivation produced the following outputs:
  out -> /nix/store/nl5h9rfv0dcwl2wzk6zyggmw0wz7gy8p-hello

$ /nix/store/nl5h9rfv0dcwl2wzk6zyggmw0wz7gy8p-hello/bin/hello                
Hello World!

The :b REPL command builds a derivation. (Use :? to look up all available commands.) Please note that after building the package, I pressed ctrl + d to exit the Nix REPL.

In the vanilla nixpkgs, there is no such package as hello-script. With the overlay, it is suddenly available - and it looks like it was always there!

If we wrote an overlay for a local package function that is designed to be used with the typical callPackage pattern (its documentation is a bit scattered over the nixpkgs docs), we could add it like this:

final: prev: {
    myPackage = prev.callPackage ./path/to/default.nix { };
}

The best about this is that for everyone who gets access to the resulting pkgs attribute, it feels like our package is a first-class citizen of nixpkgs.

Changing Existing Packages

The GNU Hello project is packaged in Nixpkgs as pkgs.hello. It only prints “Hello, world!” when invoked. This makes it a perfect example app for patching. Let’s patch it to print “Hello, Nixcademy!”:

# file: overlay-hello-nixcademy.nix

final: prev: {
  hello = prev.hello.overrideAttrs (oldAttrs: {
    postPatch = ''
      substituteInPlace src/hello.c --replace "world!" "Nixcademy!"
    '';

    doCheck = false;
  });
}

What is crucial here is that we emit the pkgs attribute set key hello into the packages collection, although it already exists. This is why we use prev.hello to reference the original hello package. prev.hello means “the hello attribute from before our overlay”.

The script snippet in postPatch substitutes all occurrences of the substring “world!” to “Nixcademy!” in the right source file of the package’s repository. (The function substituteInPlace is part of the stdenv and is documented here) This would make the unit tests fail, so we disable them via the doCheck attribute. (In production code, we would of course fix the tests, too. đŸ‘»)

Let’s try if it works:

$ nix repl
Nix 2.23.1
Type :? for help.
nix-repl> pkgs = import ~/src/nixpkgs { 
  overlays = [ (import ./overlay-hello-nixcademy.nix) ]; 
}

nix-repl> :u pkgs.hello                                                                          

$ hello
Hello, Nixcademy!

This time, we used the :u command in the REPL instead of :b: It builds the package and then launches a Nix shell that has our new package in its PATH, so we can just run it without manually fiddling with the nix store paths as in the earlier example.

It works! 🎉

Infinite Recursion Errors from Overriding Symbols

Why again did we use prev instead of final when referencing the original hello package? Let’s use hello = final.hello... instead of hello = prev.hello... in the overlay function and see what happens:

nix-repl> :u pkgs.hello                                                                          
error:
       
 while evaluating the attribute 'hello'
         at ~/src/nixpkgs/overlay-hello-nixcademy.nix:2:3:
            1| final: prev: {
            2|   hello = final.hello.overrideAttrs (oldAttrs: {
             |   ^
            3|     postPatch = ''

       
 while evaluating the attribute 'hello.overrideAttrs'
         at ~/src/nixpkgs/overlay-hello-nixcademy.nix:2:3:
            1| final: prev: {
            2|   hello = final.hello.overrideAttrs (oldAttrs: {
             |   ^
            3|     postPatch = ''

       error: infinite recursion encountered
       at ~/src/nixpkgs/overlay-hello-nixcademy.nix:2:11:
            1| final: prev: {
            2|   hello = final.hello.overrideAttrs (oldAttrs: {
             |           ^
            3|     postPatch = ''

Let’s draw the references in the diagram from the beginning of this post to understand what exactly happened here:

Graphical explanation for infinite recursion

hello = final.hello... references the hello symbol that we just created, so this is why we are moving in a circle! prev.hello is the original hello from nixpkgs (or the hello from the last overlay, if multiple overlays override the same symbol).

We internalize the most important rule:

Always use prev if you are overriding an existing symbol (i.e. without renaming it).

Don’t worry, this rule is not an additional one to the earlier presented rule sets: You are already following it if you are following them.

Overriding Nested Attributes

Overriding existing symbols like packages or nix library functions, or adding new ones, is a bit more complicated if they are not directly in pkgs but nested in some sub-attribute set.

There are multiple ways:

Overriding Plain Nested Sets

This is for example how we could add a function pkgs.lib.plusOne:

# file overlay-lib.nix

final: prev: {
    lib = prev.lib or { } // {
        plusOne = x: x + 1;
    };
}

If we didn’t use the update operator // on the existing lib set, our overlay would drop all pre-existing symbols! 😩

> nix repl
Nix 2.23.1
Type :? for help.
nix-repl> pkgs = import ~/src/nixpkgs { 
  overlays = [ (import ./overlay-lib.nix) ]; 
}

nix-repl> pkgs.lib.plusOne 10

Another more complex example: Let’s put the function in pkgs.lib.subCategory.plusOne:

# file overlay-lib-nested.nix

final: prev: {
    lib = prev.lib or { } // {
        subCategory = prev.lib.subCategory or { } // {}
            plusOne = x: x + 1;
        };
    };
}

This pattern is not only interesting if we want to add library functions. It could also be used if we put packages in subcategories like pkgs.myCompany.myProduct or similar.

Overriding Extensible Nested Sets

It shall be mentioned that for the special case of extending pkgs.lib, there is already a facility for this purpose:

# file overlay-lib2.nix

final: prev: {
    lib = prev.lib.extend (libFinal: libPrev: {
        plusOne = x: x + 1;
    });
}

The extend function inside lib provides us another overlay mechanism in the overlay mechanism just for extending this sub-attribute set. This is worthwhile to mention because if we ever end up wanting to make our own attribute sets overridable, the function makeExtensible is the way to provide exactly these mechanics.

Overriding Scopes

What is a scope? The nested set pkgs.gnome for example contains many gnome-related packages that reference each other. This is slightly similar to the nested set of pkgs.lib but this one is created using the makeScope function (documentation). One reason is that a scope also provides a specialized callPackage function symbol that makes it easier to obtain dependencies from inside the scope (otherwise the user of pkgs.callPackage would have to write the nested paths out for each dependency).

No matter for what reason a scope has been created, it can be overridden like this:

final: prev: {
  gnome = prev.gnome.overrideScope (gnomeFinal: gnomePrev: {
    updateScript = gnomePrev.updateScript.override {
      versionPolicy = "odd_unstable";
    };
  });
}

This example overlay would change the version policy in the GNOME package update script.

We have the same overlay-in-overlay pattern here as before, but this is the way to inject it when a package scope is in play.

Overriding Python Packages

Talking about extending nested attribute sets, this is also relevant for Python packages (and other language-specific sub-sets in nixpkgs). Let’s concentrate on Python now because Python is so popular and keep in mind that it works similarly for all the other languages.

The nixpkgs documentation has specific advice for most programming language specific package tooling. The Python language ecosystem in Nixpkgs provides two different ways.

This section is a great example of how fine-grained precision and control overlays give us. We can override Python packages for:

Overriding the Python Package Set for All Python Versions

This overlay creates a Python-specific overlay function and adds it to the global list pythonPackagesExtensions. The Python-specific overlay function in this example upgrades pytest to a different version:

final: prev:
let
  pytestOverlay = pythonFinal: pythonPrev: {
    pytest = pythonPrev.pytest.overridePythonAttrs (oldAttrs: {
      version = "8.3.2";
      src = pythonPrev.fetchPypi {
        pname = "pytest";
        version = "8.3.2";
        hash = "sha256-wTI0XRLOVRJCyHJp3oEkg/W8yHzbtHIuSEh7oZT5/c4=";
      };
      patches = []; # existing patch does not apply any longer
    });
  };

in
{
  pythonPackagesExtensions = prev.pythonPackagesExtensions ++ [ pytestOverlay ];
}

This overlay affects all Python package sets for all versions of Python in nixpkgs.

Overriding a Specific Python Package Set

This example performs the same override on the package pytest but applies it only to the package set in pkgs.python3.pkgs and pkgs.python3Packages:

final: prev:
let
  pytestOverlay = pythonFinal: pythonPrev: {
    pytest = pythonPrev.pytest.overridePythonAttrs (oldAttrs: {
      version = "8.3.2";
      src = pythonPrev.fetchPypi {
        pname = "pytest";
        version = "8.3.2";
        hash = "sha256-wTI0XRLOVRJCyHJp3oEkg/W8yHzbtHIuSEh7oZT5/c4=";
      };
      patches = []; # existing patch does not apply any longer
    });
  };

  python3 = prev.python3.override {
    packageOverrides = pytestOverlay;
  };
in
{
  inherit python3;
  python3Packages = python3.pkgs;
}

We could even go further and let the overlay create new attributes like myPython3 and myPython3Packages. This way, the overlay would only have an effect on the projects that use myPython3Packages while all the other Python 3 projects would remain unaffected.

Bad Practice / Don’t Do

This section lists some typical anti-patterns and explains why you should avoid them.

Don’t Collect Overlays in ~/.config/nixpkgs/overlays/

We can put multiple overlay function files into the folder ~/.config/nixpkgs/overlays and get them automatically applied to pkgs in non-Flakes Nix expressions and the REPL whenever we import nixpkgs without an explicit overlays list. (Implementation)

This is very handy if we end up using the same overlays over and over again, but it’s an impurity: Imagine some package or system description implicitly works because of your local overlay collection. Your teammates will get different characteristics and probably errors if you forget to synchronize with them. (Not having to deal with “works for me” situations is one of the main selling points of Nix!)

Don’t Use rec Attribute Sets

Imagine we create two packages of which one is the dependency of another. A bad way to do this would be:

final: prev: rec {
   pkg-a = prev.callPackage ./a { };
   pkg-b = prev.callPackage ./b { dependency-a = pkg-a; }
}

In this example, pkg-b explicitly consumes pkg-a as a dependency. This works, but:

If someone else adds another overlay that somehow overrides and patches pkg-a, pkgs-b will not notice that. pkg-b will still use the original, unpatched pkg-a.

How do we fix that? Like this:

final: prev: {
   pkg-a = prev.callPackage ./a { };
   pkg-b = prev.callPackage ./b { dependency-a = final.pkg-a; }
}

No matter how often pkg-a gets overridden by other overlays, pkg-b will always observe those changes.

Please note that if I didn’t rename pkg-b’s dependency on pkg-a to dependency-a, we could have left the second callPackage parameter empty, as it would be filled in automatically. This choice is just to keep the example very simple so that we can focus on the anti-pattern at hand.

Don’t Use External Parameters

Imagine, an overlay does something like this:

# file: overlay.nix

{ boost }:
final: prev: {
    myPackage1 = prev.callPackage ./my-package2 { inherit boost; };
    myPackage2 = prev.callPackage ./my-package2 { inherit boost; };
    myPackage3 = prev.callPackage ./my-package3 { inherit boost; };
    # ...
}

Such an overlay could be consumed like that:

# ...
pkgs = import ./path/to/nixpkgs {
  overlays = [ 
    (import ./overlay.nix { boost = pkgs.boost185; }) 
  ];
}
# ...

On the first look, this seems handy because we can “configure” the overlay for different boost library versions. If many packages consume this parameter, it is “locked” in there and can only be overridden on each package individually: This anti-pattern breaks composability.

A better approach would be to create another package symbol and then reuse it:

# file: overlay.nix

final: prev: {
  myBoostVersion = final.boost185;
  myPackage1 = prev.callPackage ./my-package2 { boost = final.myBoostVersion; };
  myPackage2 = prev.callPackage ./my-package2 { boost = final.myBoostVersion; };
  myPackage3 = prev.callPackage ./my-package3 { boost = final.myBoostVersion; };
  # ...
}

That way, we can import and change it in a later overlay like this:

# ...
pkgs = import ./path/to/nixpkgs {
  overlays = [ 
    (import ./overlay.nix)
    (final: prev: { myBoostVersion = final.boost180; })
  ];
}
# ...

Beware When Importing Extra Nixpkgs in Overlays

Sometimes you might want to mix different versions of nixpkgs. This is not generally wrong but should be done with caution.

This example overlay tries to upgrade or downgrade a package by overwriting its symbol with the version of a pinned newer/older version of nixpkgs.

# file: overlay.nix

let
  pkgs = import ./path/to/nixpkgs { };
in

final: prev: {
  inherit (pkgs) somePackage;
}

This works in many cases, but:

An alternative way might be:

# file: overlay.nix 

let
  pkgs = import ./path/to/nixpkgs { };
in

final: prev: {
  somePackage = prev.callPackage pkgs.somePackage.override { };
}

This way the package’s package function is re-invoked with all the dependencies and also cross-compilation/linking settings of your current nixpkgs invocation. The older/newer package function might however not work with the newer/older dependencies, so these might need some specific overrides, too.

Don’t Reference other packages via prev

If we want to override specific dependencies by default, we might do it like this C++ example project:

final: prev: {
  myCppProject = prev.callPackage ./my-project {
    stdenv = prev.clangStdenv;
    boost = prev.boost185;
  };
}

This way, the project builds with the Clang compiler (Nixpkgs selects GCC by default on Linux) and a specific boost version. It works, but:

If another overlay adds global patches or changes the version of the symbols clangStdenv or boost185, they will go unnoticed by this package.

Better:

final: prev: {
  myCppProject = prev.callPackage ./my-project {
    stdenv = final.clangStdenv;
    boost = final.boost185;
  };
}

Summary

Overlays are complex, but powerful. Learning how to use them is well worth the effort, as they make projects and products (packages or full system images) composable and easily extensible. They are also an often overlooked feature of the Nix Flake era.

Especially if the users are also familiar with override and overrideAttrs, they can work wonders with entire system images. Developers turn an image inside out for debugging purposes and don’t even have to check out the project folders of patched projects and reparameterized build systems.

In our Nix & NixOS 101 course, we have a great exercise that deals intensively with overlays only and has been proven to clear up any confusion with overlays. After the course, our participants are well-versed in overlays and use them productively in their everyday project work.

We also typically cover questions like:

If your team is too small for an onsite group training or you are an individual, give our public online training sessions a try!

Nixcademy on Twitter/X

Follow us on X

Nixcademy on LinkedIn

Follow us on LinkedIn

Get Online Nix(OS) Classes

Nixcademy online classes

Are you looking for...

  • a guided learning experience?
  • motivating group exercises?
  • a personal certificate?
Nixcademy Certificate

The Nixcademy Newsletter: Receive Monthly Nix(OS) Blog Digests

Nixcademy Newsletter

Receive a monthly digest of blog articles and podcasts with useful summaries to stay connected with the world of Nix and NixOS!

Stay updated on our latest classes, services, and exclusive discounts for newsletter subscribers!

Subscribe Now