Don't Let Flakes Pin You Down: Unlocking Better Inputs
Nix flakes became a great improvement for the Nix ecosystem, but many people are still not happy, and you might not be aware of what youâre missing out on.
While flakes do many things, one of the key features newcomers latch onto is input pinning.
We need our Nix recipes to download code, e.g., from https://github.com/nixos/nixpkgs, but without additionally pinning down which exact commit, this could mean something else every day.
Without pinning, we wonât be able to get that magic word: reproducibility.
Before flakes, developers would often write something like this to get nixpkgs:
{ pkgs ? import <nixpkgs> { } }:
# ...
The shortcut <nixpkgs> points to some version of nixpkgs on the developerâs machine/CI.
It typically resolves to a different nixpkgs commit on each machine, hence the build may be working for one developer and fail for another.
Another code smell comes from external code references in Nix expressions like this one:
builtins.fetchTarball "https://our.git.forge/~user/proj/archive/main.tar.gz"
This address is a moving target as the main branchâs commit will change over time.
Flakes fix everything or not?
Popular tooling like niv and npins provides a nice user interface that allows users to add and frequently update commit-specific inputs comfortably, but Nix never forced users to use them. Non-flakes Nix did not really prevent developers from committing irreproducible code to a production database.
Flakes fixed that by implementing pinning support directly in the C++ implementation of Nix in Nix flake files.
Nix flakes provide an inputs attribute that lets the user list all external inputs and automatically locks the exact commits in a flake.lock file.
In addition to that, Nix flakes introduced a âlocked downâ evaluation mode for Nix expressions that doesnât allow for external code fetcher calls that donât provide output hashes.
Limitations of pinning tooling (both flakes and non-flakes)
With this in mind, we must acknowledge that needing to automate the pinning to stable references has been a concept in the Nix ecosystem and best practices long before flakes. Flakes do input pinning, but it isnât without its limitationsâŠ
Lack of version control systems (VCSs) that arenât Git (or Mercurial)
flakes and the fetchTree API can fetch files, archives (tarballs, zip files, etc.), and Git and Mercurial Version control systems (VCS).
There are many more VCSs than Git and Mercurial; have a look at the non-exhaustive Wikipedia VCS list page. Nixpkgs generally supports fetching from most of those:
$ ls pkgs/build-support | grep -e "^fetch" | head -n5
fetch9front
fetchbitbucket
fetchbzr
fetchcodeberg
fetchcvs
$ ls pkgs/build-support | grep -e "^fetch" | wc -l
40
We can see there are roughly 40 different fetchers that are supported in Nixpkgs already, but arenât exposed for flake inputs. Why is that? Any VCS that shall be supported in flakes must be compiled into the Nix C++ binary. This ends up being a massive maintenance burden on the Nix C++ codebase (as well as forks) to maintain, but also more or less demands a C/C++ library or binding to be used. Anything implemented in the Nix C++ binary must practically be kept forever, as it would break backwards compatibility otherwise.
There are quite a few widely used projects out there that donât use Git or Mercurial:
- SQLite - Fossil
- Many Apache projects - Subversion
- Docutils - Subversion
- Tcl - Fossil
- Thrussh Rust SSH library - Pijul
- Many Haskell libraries - Darcs
- The gaming industry - Perforce
A lot of software using other VCSs is usually still available as an HTTP link to a tarball. Tarballs, however, donât come with the same sort of inherent stable reference mechanism. No one knows if a tarball really contains a specific commit without further changes.
Moreover, a short-sighted design choice was to give special privilege to specific code forges like GitHub and GitLab. A convenience today is yet another maintenance burden tomorrow, as some of these code forges are proprietary and break their API from time to time. What is in fashion today might not be in the future, but now it will be solidified for backwards compatibility in the Nix binary. The primary motivator seems to be using Git to get hash info to hand to a Git archive tarball, since we discussed how tarballs donât have a stable reference, but if the user could template input and define their own procedure to refresh the URL (as discussed in the next section), this could be solved.
In the FOSS world, we love the freedom our software can afford us to not be locked into a specific platform, but flakes being compiled into the Nix binary limits us.
Tarballs will refetch every time you nix flake update
If you find yourself using a tarball (instead of a direct VCS source because flakes donât support it) or say you pinned a several-gigabyte model from HuggingFace, and then run nix flake update, you might find yourself in a surprise.
Nix flakes will happily refetch that whole payload for you, often just to give you the olâ đâ to say âyep, the hash didnât changeâ.
This is incredibly wasteful, especially if you are on metered data or in some place that doesnât have access to high-speed internet.
If flakesâ design gave you the ability to define how to see if your input is fresh or stale, you could have avoided this waste.
But without a way to define âfreshnessâ, flakes will be limiting us.
No mirror support
Weâve all seen it: The server is down. It doesnât matter if itâs your self-hosted server or some major corporate offering; none of them have 100% uptime. Maybe the server was DDoSâd, or perhaps itâs a censorship campaign by a corporation or government. In ye olden times, we always had multiple mirrorsâââeven on different protocols like FTP or BitTorrent. While uptime has gotten a bit better over time, the need for fallbacks hasnât changed.
But itâs cached at cache.nixos.org
ââSome reader out there
Not only can the cache itself be down, but not all software is marked as redistributable for the cache,âand your internal libraries arenât going to be in the public cache (if you are lucky, your internal binary cache has it).
Even our trusty pkgs.fetchurl has the urls attribute for just this case, but the reality is that a lot of projects are offering a mirror (maybe you are inspired too), but the flake input URI has no ?mirrors=⊠option, limiting us once more.
Cannot apply patches to the input itself
With overrideAttrs and overlays, your average intermediate-level Nix developer already knows how to patch up their software, but have you ever wanted to patch at the input level before importing it?
If your response is âwhen/why would I need that?â, thereâs a clear use case you have probably run into: Awaiting Nixpkgs PRs.
You see some WIP fix on Nixpkgsâs code forge, or maybe it just hasnât yet trickled into your channel (such as nixos-25.11 or nixos-unstable).
Itâs fixing a module or derivation that you want now, and itâs not a trivial version override like a package.
What do you do?
Some of us have taken the lazy approach and pinned yet another Nixpkgs version (which these tarballs are getting huge @ 50+ MiB) and monkeyed together an overlay just for a small fix.
What if we could just pull and apply the patch to our current Nixpkgs?
We have a raw patch, for example, just waiting to be used, but flakes limited us by not having a ?patch=âŠandpatch=⊠in the input schema.
Canât change the hash algorithm
You read the latest Nix manual experimental features and saw blake3-hashes.
You read more about how BLAKE3 is much faster at hashing using SIMD and multithreading, primed for tree-like data structures, like the file system, which Nix is always creating hashes for.
It offers modern security expected from hashing functions.
You want to try BLAKE3 on some of your inputs, but you canât as flake inputs have limited you by not offering ?hash_algorithm=âŠ.
What would a different future look like?
When we look at the nix-prefetch-scripts available in nixpkgs:
$ find pkgs/build-support -type f -name "nix-prefetch-*"
pkgs/build-support/docker/nix-prefetch-docker
pkgs/build-support/docker/nix-prefetch-docker.nix
pkgs/build-support/fetchbzr/nix-prefetch-bzr
pkgs/build-support/fetchcvs/nix-prefetch-cvs
pkgs/build-support/fetchdarcs/nix-prefetch-darcs
pkgs/build-support/fetchfossil/nix-prefetch-fossil
pkgs/build-support/fetchgit/nix-prefetch-git
pkgs/build-support/fetchhg/nix-prefetch-hg
pkgs/build-support/fetchpijul/nix-prefetch-pijul
pkgs/build-support/fetchsvn/nix-prefetch-svn
We can see the start of a collection of scripts shared both to Nixpkgs itself and useful to other scripts to build atop it. Many of these scripts have rich, structured data to work with, too, to get more than just the hash value. If we saw more of these pop up for different types of fetchers, we could have a rich input system too. That system would be reliant on Nixpkgs since many of the VCS tools arenât standalone binaries, and so Nixpkgs solves the bootstrapping + toolchain problem. Nixpkgs is a monolith, but were you really building much without it?
A new tool wouldâŠ
- not be built into the Nix binary, but be independent and not rely on experimental Nix features.
- encourage using existing patterns with high compositionality like overlays, or at least something not inside an opaquish box like
flake.nix. - not define which VCSs are the chosen ones, but allow us to experiment.
- be ready for the post-Git world (if not philosophically, then for some new tool for âagenticâ efficiency).
- let users define lightweight, extensible ways to test âis this input staleâ.
- encourage mirroring our software to many locations to avoid downtime and censorship,âas well as self-hosting our own lightweight repos just in case.
Enter Nixtamal
Nixtamal is a solution for fulfilling input pinning. It exposes the properties of that âfuture systemâ in a bit of a sandbox to construct many things in your own way. It can:
- Automate the manual work of input pinning for dependency management
- Allow easy ways to lock and refresh those inputs
- Declarative KDL manifest file over imperative CLI flags
- diff/grep-friendly lockfile
- Host, forge, VCS-agnostic
- Choose eval time fetchers (builtins) or build time fetchers (Nixpkgs, default)âââwhich opens up fetching now-supported Darcs, Pijul, and Fossil
- Supports mirrors
- Override hash algorithm on a per-project and per-input basisâââincluding BLAKE3 support
- Custom freshness commands (command to find out if the input is stale)
- No experimental Nix features required
npins, niv, Yae, and others cover some of these features, but no other tool is trying to solve all of these.
Letâs first check out how simple it is to get started with Nixtamal and then have a look at a full-fledged annotated Nixtamal manifest file to understand whatâs possible.
Getting started
With Nixtamal already in NixOS unstable, you can use it today from Nixpkgs.
If on the nixos-unstable channel:
$ nix-shell -p nixtamal
If flakes are enabled:
$ nix shell github:NixOS/nixpkgs?ref=nixos-unstable#nixtamal
Then you can set up a basic project lock file like this:
$ nixtamal set-up
âââ»+â» â±ââłâââââłââââ»
ââââââââčââčâŁâ«ââââŁâ«â
âčâââčâ± âč âč âčâčâč âčâčâčââ
Fetching fresh value for ănixpkgsă âŠ
Prefetching input ănixpkgsă ⊠(this may take a while)
Prefetched ănixpkgsă.
Making manifest file @ version:1.0.0
$ nixtamal tweak
The tweak sub command will open our manifest.kdl for tweaking, so we can have a look at how the input description in Nixtamal generally looks:
version "1.0.0"
inputs {
nixpkgs {
archive {
url "https://github.com/NixOS/nixpkgs/archive/{{fresh_value}}.tar.gz"
}
hash algorithm=SHA-256
fresh-cmd {
$ git ls-remote "https://github.com/NixOS/nixpkgs.git" --refs "refs/heads/nixos-unstable"
| cut -f1
}
}
}
Letâs understand how Nixtamal sees our data:
$ nixtamal show
nixpkgs: (archive)
fetch-time: build
url: https://github.com/NixOS/nixpkgs/archive/b40629efe5d6ec48dd1efba650c797ddbd39ace0.tar.gz
fresh-cmd: $ git ls-remote https://github.com/NixOS/nixpkgs.git --refs refs/heads/nixos-unstable | cut -f1
fresh-value: b40629efe5d6ec48dd1efba650c797ddbd39ace0
hash-algorithm: SHA-256
hash-value: sha256-TJ3lSQtW0E2JrznGVm8hOQGVpXjJyXY2guAxku2O9A4=
frozen: false
We can see the fresh-value is our stable reference to a Git commit hash, the hash-value using SHA-256, as well as the resolved url for our archive kind.
Now weâll set up a file to use this pinned Nixpkgs input.
In this example, letâs build the GNU Hello project as a starter:
# default.nix
let
inputs = import ./nix/tamal { };
pkgs = import inputs.nixpkgs { };
in
pkgs.hello
$ nix-build default.nix
/nix/store/8qi947kixhz1nw83dkwxm6d0wndprqkj-hello-2.12.2
$ ./result/bin/hello
Hello, world!
You can then use nixtamal tweak to make more edits, and then run nixtamal lock to lock your new inputs.
When itâs time to recheck freshness, nixtamal refresh.
Have a look at more examples in the Nixtamal cookbook section.
On âfreshnessâ
One of the bigger features of note here is the fresh-cmd in the manifest file.
It allows you to define arbitrary, string-returning shell commands that can be used to determine if the input is âstaleâ, which will short-circuit downloading our input (be that a tarball, Darcs repository, whatever).
With Git, we can use Gitâs built-in ls-remote subcommand (Nixtamalâs default when unspecified for Git input kinds), or we can override it to cURL an API like we might do for another input type.
The point being: What is more flexible and extensible than allowing the user to be fully in charge of the freshness canary?
When itâs done being called, if we got a new âfresh valueâ, that inputâs lock.json lockfile is updated, resolving any string templates along the way.
Since the lockfile is not handwritten, the lockfile plus a Nix shim @ $NIXTAMAL_DIRECTORY/default.nix gets us and our teams reproducible inputs.
Show me the features!
Letâs have a look at a more complex Nixtamal manifest that uses more features:
version "1.0.0"
// By default in this project, use experimental BLAKE3 algorithm for
// quicker, safer hashing
default-hash-algorithm BLAKE3
// Define and even reuse patches
patches {
// Unique name for referencing in manifest inputs
chroma-0.22.0 "https://patch-diff.githubusercontent.com/raw/NixOS/nixpkgs/pull/478519.patch" {
// Override the project default hash algorithm
hash algorithm=SHA-512 expected="1mdsfx204bgia572fydnmjy78dkybbcnjx20qn9l4q65r29ry28c"
}
}
// Define inputs
inputs {
// Unique name for referencing in Nix
nixpkgs {
// Fetch an archive with string templating support
archive {
url "https://github.com/NixOS/nixpkgs/archive/{{fresh_value}}.tar.gz"
}
hash algorithm=SHA-256
// Apply patches to the source now while awaiting review
patches chroma-0.22.0
// cURL an Atom feed for updates, stat a directory, whatever you
// need so long as it returns a string, you can use it!
// This also means you can prevent downloading massive files by
// deciding yourself what âfreshâ means to you.
fresh-cmd {
$ git ls-remote --branches "https://github.com/NixOS/nixpkgs.git" --refs nixpkgs-unstable
| cut -f1
}
}
nixtamal {
// Use VCSs not supported by `builtins`
darcs {
repository "https://darcs.toastal.in.th/nixtamal/stable"
// fallback to mirrors when a host is down
mirrors "https://smeder.ee/~toastal/nixtamal.darcs"
}
fresh-cmd {
$ curl -sL "https://darcs.toastal.in.th/nixtamal/stable/_darcs/weak_hash"
}
}
// Download and pin a model from Hugging Face, then can be used like with
// `specialArgs = { inherit inputs; }`:
//
Qwen3-Coder-Next {
file {
url "https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF/resolve/{{fresh_value}}/Qwen3-Coder-Next-Q3_K_M.gguf"
}
fresh-cmd {
$ curl -fsL "https://huggingface.co/api/models/unsloth/Qwen3-Coder-Next-GGUF/commits/main?path=Qwen3-Coder-Next-Q3_K_M.gguf"
| jq -r ".[0].id"
}
}
}
Using and consuming Nixtamal
With a manifest defined, we can run nixtamal lock or nixtamal refresh to get our lockfile + Nix shim.
From there, we are ready to go.
Assuming we use the prior manifest.kdl, hereâs how to use it in a release.nix:
let
inputs = import ./nix/tamal { };
pkgs = import inputs.nixpkgs {
overlay = [
# inputs.nixtamal exposes its overlays at nix/overlay/
# Refer to the project documentation on where its overlay may lie
(import "${inputs.nixtamal}/nix/overlay")
];
};
in
{
nixtamal = pkgs.nixtamal.default;
}
$ nix-build release.nix -A nixtamal
/nix/store/q0w48bwv7snd87qi5awvh5z7b0j1j5if-ocaml5.4.1-nixtamal-1.1.4
$ result/bin/nixtamal --version
1.1.4
and the HuggingFace can be used in a NixOS module like:
# NOTE: `specialArgs = { inherit inputs; }` used
# to pass inputs to our modules.
# (overlays are cleaner but let's not blow up the example)
{ inputs, ... }:
{
services.llama-cpp = {
enable = true;
model = "${inputs.Qwen3-Coder-Next}";
};
}
Note that by using HuggingFaceâs JSON API, we are able to get the exact update to that model file specifically. We do not need to know the entire Git history in the repository. More importantly, we donât need to redownload a 45+ GiB file just to see if it is stale. We were also able to reference the file directly.
Takeaway
Nixtamal offers new ways to handle inputsâââfrom new VCSs, to mirror support, to faster hashing, all the way to being in control of when your inputs are fresh vs. staleâââit might be the missing piece to your setup.
As Nixtamal is just an input pinner, compatible with other schemas like flakes or Nilla, either with other methods of pinning or to replace your existing pinning setup.
If you are ready to unlock your input pinning potential, check out the Nixtamal project today!