C++ with Nix in 2023, Part 2: Package Generation and Cross-Compilation
![C++ with Nix in 2023, Part 2: Package Generation and Cross-Compilation](/_astro/package-machine-with-portal.DSZpPZ3e_Z2aAynl.webp)
This is Part 2 of our series on using C++ with Nix in 2023 we’re moving a step further to focus on packaging and cross-compilation. Packaging is crucial for making your applications easily distributable, while cross-compilation allows you to build software for different platforms from a single source.
In part 1 of this series, we explored the basics of setting up a Nix shell for C++ development. This week, we learn how to create a Nix package from our example project.
A package definition in Nix allows for sandboxed single-command package builds that are truly reproducible, even on computers of non-developers who happen to have Nix installed. With a Nix package definition, we also get the following other goodies:
- Developer shells can be automatically generated from package definitions. So we can ditch our old manual development shell definition to avoid duplicate efforts.
- Nix flakes provide a nice way for users to run our application without even checking out the code.
- Assuming portable code and build system setup, Nix gives us cross-compilation for free!
- It is also easy to swap compilers after the fact. This is especially interesting if you want to regularly check the compatibility of your code with multiple new and old compilers.
The final code for this article is online in the same GitHub repo as the first
article, but on branch part2
:
https://github.com/tfc/cpp-nix-2023/tree/part2
Create a Package for Our App
Until now, we got to know the nix develop
use case for developers who need all
toolchain and dependency packages in their shell environment to work on the
project.
This is already better than Docker images or VMs because we don’t need to mount
the source code into containers or similar.
Our favorite text editors also have access to tools and libs when we start them
from a Nix shell (We can in fact start as many different Nix shells per project
as we like).
However, consumers of our apps and libraries don’t care about our development
environment.
They simply want to have the package for consumption.
Wouldn’t it be great if they could simply run nix build
and get it without
even having to know what language it is written in or how to set up the shell
and run the build?
Let’s do that by first creating a package.nix
file that describes the package.
The code is very short, but I added many comments for comprehensibility:
# file: package.nix
# The items in the curly brackets are function parameters as this is a Nix
# function that accepts dependency inputs and returns a new package
# description
{ lib
, stdenv
, cmake
, boost
, catch2
}:
# stdenv.mkDerivation now accepts a list of named parameters that describe
# the package itself.
stdenv.mkDerivation {
name = "cpp-nix";
# good source filtering is important for caching of builds.
# It's easier when subprojects have their own distinct subfolders.
src = lib.sourceByRegex ./. [
"^src.*"
"^test.*"
"CMakeLists.txt"
];
# We now list the dependencies similar to the devShell before.
# Distinguishing between `nativeBuildInputs` (runnable on the host
# at compile time) and normal `buildInputs` (runnable on target
# platform at run time) is an important preparation for cross-compilation.
nativeBuildInputs = [ cmake ];
buildInputs = [ boost catch2 ];
# Instruct the build process to run tests.
# The generic builder script of `mkDerivation` handles all the default
# command lines of several build systems, so it knows how to run our tests.
doCheck = true;
}
This package description contains no information about how to run the
configuration step, how to build the binaries, how to run the tests, and how to
finally install the cpp-nix
app.
The mkDerivation
function automagically knows it.
But the process is not magical: Making our package depend on cmake
also
installs certain hooks that educate mkDerivation
’s builder script with the
knowledge of how to deal with CMake projects.
If our CMake project definition was highly customized so that it deviates from standard projects too much, these customizations could be added to the package definition. Many C++ projects have ill-formed CMake/Meson/autotools build systems, which increases the effort on the package maintainer side, but this is nothing new to nixpkgs.
We can now “call” the package from the flake and also let the dev shell reuse it:
# file: flake.nix
# ...
perSystem = { config, self', inputs', pkgs, system, ... }: {
# The `callPackage` automatically fills the parameters of the function
# in package.nix with what's inside the `pkgs` attribute.
packages.default = pkgs.callPackage ./package.nix { };
# The `config` variable contains our own outputs, so we can reference
# neighbor attributes like the package we just defined one line earlier.
devShells.default = config.packages.default;
};
# ...
The callPackage
function fills out all parameters from the dependency list
at the beginning of our package.nix
file.
This way the package.nix
file describes the mechanism of how to build the
package, while the flake.nix
file contains the policy decisions of where to
get which package from (in this case: which nixpkgs pin to use).
This is also the point where we could override the compiler or library inputs.
By assigning the package to our devShells.default
attribute, the nix develop
command simply takes all dependency information from there, which perfectly
works for smaller repos.
Monorepos with multiple projects work a bit differently, but this article
doesn’t deal with that.
Let’s build it.
Note that we don’t need the nix develop
shell for this:
$ nix build
$ ./result/bin/cpp-nix
Hello World!
Compiler: g++ 12.3.0
Boost: 1.81.0
There is no trace of a build
folder or anything, which we get when we compile
the code manually in the developer shell.
Nix creates a sandbox environment from scratch, builds the package inside, and
then throws away everything but the package output.
We only get a symlink with the name result
in the current folder which points
to the /nix/store/...cpp-nix/
path that contains our package outputs.
We can also directly run it in a single command without having to build and run in two steps:
$ nix run
Hello World!
Compiler: g++ 12.3.0
Boost: 1.81.0
Note that nix automatically finds out if it needs to rebuild the package based on content changes in the project folder.
We can even build and run it from GitHub without having to clone the repository first:
$ nix run github:tfc/cpp-nix-2023
Hello World!
Compiler: g++ 12.3.0
Boost: 1.81.0
At this point, there is practically no difference in the user experience between “real” Nix packages and ours.
Swap the Compiler
We already did that with the developer shell environment in the last article,
but on a package, it’s even a bit more straight-forward:
The packages that we list in package.nix
’s first lines as function inputs can
all be overridden with a different choice than the defaults.
Swapping the compiler now looks like this:
# file: flake.nix
# ...
packages = {
default = pkgs.callPackage ./package.nix { };
clang = pkgs.callPackage ./package.nix { stdenv = pkgs.clangStdenv; };
};
# ...
This new attribute can be built with nix build .#clang
.
A CI could now always build both variants so that developers can keep supporting
multiple compiler toolchains.
On GNU/Linux, Nix’s default C++ compiler is GCC, while it’s clang on macOS. Use
pkgs.clangStdenv
orpkgs.gccStdenv
respectively.
Cross-Compilation
There is nothing inherently architecture-specific about our little hello-world app. To cross-compile it, we need a cross-compiler and the Boost library for the target platform. We would also need to disable the tests because even if we can cross-compile them, we would usually not be able to run them on the same host.
Let’s first make testing optional in our package.nix
file:
# file: package.nix
{ lib
, stdenv
, cmake
, boost
, catch2
# Tests are by default enabled, but the caller can disable them
, enableTests ? true
}:
# ...
# The third category `checkInputs` is ignored if tests are disabled
nativeBuildInputs = [ cmake ];
buildInputs = [ boost ];
checkInputs = [ catch2 ];
doCheck = enableTests;
# Our CMakeLists.txt has already been prepared from the beginning
cmakeFlags = lib.optional (!enableTests) "-DTESTING=off";
}
We will later see how to call this package with disabled tests.
Selecting the cross-compiler is generally done by creating a different stdenv
which not only contains the right compiler but also selects the right libc
and
other toolchain settings.
In addition to that, all the dependency libraries that are needed for/by the
target platform, also need to be exchanged.
While this could be a lot of work for big projects, Nixpkgs makes this really
simple by providing a pkgs.pkgsCross.XXX
attribute that contains all the
pkgs
for architecture XXX
.
In the following, we use aarch64-multiplatform
for cross-compiling from
systems from non-ARM systems to ARM platforms and gnu64
to build packages for
Intel-based platforms.
macOS users would select aarch64-darwin
and x86_64-linux-darwin
.
We will have a little outlook on what else is there in the following subsection.
Our next changes on the flake look like this:
# file: flake.nix
# ...
packages = {
default = pkgs.callPackage ./package.nix { };
} // pkgs.lib.optionalAttrs (system != "x86_64-linux") {
crossIntel = pkgs.pkgsCross.gnu64.callPackage ./package.nix {
enableTests = false;
};
} // pkgs.lib.optionalAttrs (system != "aarch64-linux") {
crossAarch64 = pkgs.pkgsCross.aarch64-multiplatform.callPackage ./package.nix {
enableTests = false;
};
};
# macOs users exchange the `-linux` suffix for `-darwin`
# ...
What we are doing here is populating the packages
attribute set with more
packages.
On non-Intel systems, we add the crossIntel
package and on non-ARM systems,
we add the crossAarch64
package.
The //
operator can be used to merge two attribute sets (technically, it
updates/overwrites the attributes of its left operand with the ones of its right
operand -
see docs)
The optionalAttrs
function accepts a bool condition variable and an attribute
set as parameters.
If the condition is true, it returns the whole attribute.
Otherwise, it returns an empty attribute set.
Merging one attribute set with an empty one has no effect of course.
Now, we can build the respective cross-package.
Assuming we are on an Intel-based Linux machine, we would use crossArch64
:
$ nix build .#crossAarch64
$ file ./result/bin/cpp-nix
./result/bin/cpp-nix: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/2hy50jbh5bfdn4ial7j7lrr5r7byavgw-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, not stripped
The general experience with cross-compilation of packages on Nix is very
painless, but only with packages that are truly portable.
Many C/C++ projects in the wild use Autotools/Cmake/Meson in non-portable ways
and hence need to be changed for portability to provide the same seamless
experience.
The general rule of thumb is:
Build systems should focus on building and not do dependency management tasks
other than finding library paths (e.g. using CMake’s official find_package
functionality).
Then, cross-compilation and static compilation come for free most of the time.
Which Cross-Architectures Are Available?
Cross-compilation is generally possible on one of the given axes or a mix of those:
- different CPU architectures
- different operating systems (/system libraries, including libc)
- dynamic or static linking
Their combination can be considered a platform. We can look the available platforms up for our given nixpkgs pin:
$ nix repl
nix-repl> :lf .
nix-repl> pkgs = import inputs.nixpkgs {}
nix-repl> pkgs.pkgsCross.<Press TAB>
pkgs.pkgsCross.aarch64-android pkgs.pkgsCross.loongarch64-linux pkgs.pkgsCross.ppc64-musl
pkgs.pkgsCross.aarch64-android-prebuilt pkgs.pkgsCross.m68k pkgs.pkgsCross.ppcle-embedded
pkgs.pkgsCross.aarch64-darwin pkgs.pkgsCross.mingw32 pkgs.pkgsCross.raspberryPi
pkgs.pkgsCross.aarch64-embedded pkgs.pkgsCross.mingwW64 pkgs.pkgsCross.remarkable1
pkgs.pkgsCross.aarch64-multiplatform pkgs.pkgsCross.mips-embedded pkgs.pkgsCross.remarkable2
pkgs.pkgsCross.aarch64-multiplatform-musl pkgs.pkgsCross.mips-linux-gnu pkgs.pkgsCross.riscv32
pkgs.pkgsCross.aarch64be-embedded pkgs.pkgsCross.mips64-embedded pkgs.pkgsCross.riscv32-embedded
pkgs.pkgsCross.arm-embedded pkgs.pkgsCross.mips64-linux-gnuabi64 pkgs.pkgsCross.riscv64
pkgs.pkgsCross.armhf-embedded pkgs.pkgsCross.mips64-linux-gnuabin32 pkgs.pkgsCross.riscv64-embedded
pkgs.pkgsCross.armv7a-android-prebuilt pkgs.pkgsCross.mips64el-linux-gnuabi64 pkgs.pkgsCross.rx-embedded
pkgs.pkgsCross.armv7l-hf-multiplatform pkgs.pkgsCross.mips64el-linux-gnuabin32 pkgs.pkgsCross.s390
pkgs.pkgsCross.avr pkgs.pkgsCross.mipsel-linux-gnu pkgs.pkgsCross.s390x
pkgs.pkgsCross.ben-nanonote pkgs.pkgsCross.mmix pkgs.pkgsCross.sheevaplug
pkgs.pkgsCross.bluefield2 pkgs.pkgsCross.msp430 pkgs.pkgsCross.ucrt64
pkgs.pkgsCross.fuloongminipc pkgs.pkgsCross.musl-power pkgs.pkgsCross.vc4
pkgs.pkgsCross.ghcjs pkgs.pkgsCross.musl32 pkgs.pkgsCross.wasi32
pkgs.pkgsCross.gnu32 pkgs.pkgsCross.musl64 pkgs.pkgsCross.x86_64-darwin
pkgs.pkgsCross.gnu64 pkgs.pkgsCross.muslpi pkgs.pkgsCross.x86_64-embedded
pkgs.pkgsCross.i686-embedded pkgs.pkgsCross.or1k pkgs.pkgsCross.x86_64-freebsd
pkgs.pkgsCross.iphone32 pkgs.pkgsCross.pogoplug4 pkgs.pkgsCross.x86_64-netbsd
pkgs.pkgsCross.iphone32-simulator pkgs.pkgsCross.powernv pkgs.pkgsCross.x86_64-netbsd-llvm
pkgs.pkgsCross.iphone64 pkgs.pkgsCross.ppc-embedded pkgs.pkgsCross.x86_64-unknown-redox
pkgs.pkgsCross.iphone64-simulator pkgs.pkgsCross.ppc64
There’s also pkgs.pkgsStatic
which automatically builds static binaries that
are linked against musl
instead of the GNU
libc
.
Can we cross compile from x86_64-linux
to aarch64-darwin
or the other way
around?
Compiling from macOS for Linux just works.
Trying it out the other way around throws this error:
error: don't yet have a `targetPackages.darwin.LibsystemCross for x86_64-apple-darwin`
It might be generally possible, but no one implemented that, yet.
Summary
This part of our series has illuminated how to package our C++ application in a manner that makes it easily accessible and usable by end-users, without them needing to understand the intricacies of the build process. Moreover, we’ve seen the simplicity and effectiveness of cross-compiling for different architectures using Nix. The ability to compile for multiple platforms with minimal hassle is a game-changer, especially in an environment where time and resources are precious.
- There is no real difference between our own packages and the “real” packages from nixpkgs.
nix build
andnix run
also work on remote repository URLs without cloning the repo.- If we created a library package instead of an executable package, other
build recipes (as our
package.nix
file) can consume it like any other first-class package - As long as our code,
build system, and Nix recipe are portable,
cross-compilation basically comes for free via
pkgs.pkgsCross
. - This is also the way to get statically linked packages.
I hope this demonstrates to you that Nix is not just a tool but a powerful ally in the software development process, providing flexibility, consistency, and efficiency. These qualities make Nix an invaluable asset for companies looking to streamline their development pipeline and maintain high standards across diverse computing environments.