Why NixOS is the perfect OS for me

Among the various Linux distribution, one that has captured my attention for years is NixOS. Last year, I did the final jump and now I use NixOS on my personal machine. Since this week marks 20 years since Nix was created, I thought I’d pick up this post from my drafts, polish it up, and publish it – not to brag about using a niche operating system, but to explain why it is my distro of choice.

Note: In this article I will use Nix, NixOS, nixpkgs mostly interchangeable, even though they are not the same thing. Nix is the package manager that provides support of several of the features I am listing. It can be used on MacOS and any other Linux distribution to install packages from nixpkgs. NixOS is a Linux distribution built around the Nix ecosystem. While these are not the same, for this article I have considered them to be, for ease of exposition.

History 🔗

Before going into why Nix works for me, let’s first go through a trip down memory lane, to explore why for more than 10 years I have been looking at this distribution.

First contact 🔗

More than a decade ago – so far ago that parts of my memory are fuzzy –, I was planning to work on a new blog engine in Haskell as that seemed like a good idea to help me learn the language. The idea was to write the article in multiple files and then combine everything together in a file with a syntax similar to .ini files (if I recall correctly). So, the first step in writing that blog engine was to write a lexer and a parser for these files. I picked alex for the lexer. There was a tutorial, I followed it, everything worked until the very last section. At that point, I got a compile error that some template or function was missing, even though the package’s page was showing it. After some debugging I realized what was the problem: the version shipped by Ubuntu – my operating system at the time – was more than 2 years old, although I have just installed the operating system and had the most recent packages in the distribution.

This was before Cabal was a tool – or one that I knew existed and could use –, so my idea of getting to have up to date Haskell packages was to switch to using a bleeding edge distribution, to have each package available as close to the moment it was released. Since I already had some experience with compiling GHC, I excluded Gentoo, as I didn’t want to spend time compiling everything. So, I selected Arch to be my next operating system.

All was good for nearly half a year. Until a package update had some unchecked assumptions. If I recall correctly, the update wanted to skip a symlink redirection: the package depended on some shared object, say /usr/bin/libfoo.so, that on most Arch system was actually a symlink to /bin/libfoo.so (after which it was a symlink to the correctly version shared object). So the author of the script to update this package wanted to link directly to /bin/libfoo.so and remove the symlink from /usr, since they assumed that that would become obsolete. Well, on my system – and a few hundreds of others, as I recall there being a significant issue at the time –, the actual DSO was in /usr/bin and /bin only contained a symlink to it. So by performing the deletion mentioned above, some systems were left in a state where the dynamic linker could not find some needed library.

This wouldn’t have been a big issue in general, except it so happened that bash itself needed that shared library. Hence, after restarting, my system was unable to boot into the operating system. I had the option to spend some time and fix the issue from the rescue prompt or reinstall. Given time pressure, I installed a new Ubuntu, then, later, I cycled through Debian, Open SUSE, Linux Mint, etc. and then ended with Fedora which I have used until recently.

At this point, you might be asking why are you talking about these distributions instead of Nix?. And here is the answer: while looking at these distributions, I found a mention in a random paragraph on the internet about Nix and NixOS. There was a nice property that these had that made me try them: the entire configuration was declarative and allowed rollbacks for every change. Given the Arch story, that seemed ideal to have at the time.

However, when I tried it on a virtual machine, I only managed to get an xterm window up and then the entire thing resulted in a kernel panic. So, I put everything in background memory and moved on with the rest of the OS experiments as mentioned before.

Update packages? Upgrade distro? No time for that 🔗

Sometime during my PhD I realized that I was using a very old version of Fedora, not having updated the distribution to the new version for several releases. This was motivated by the fact that I was using a significant number of LaTeX packages for my research papers as well as other similar dependencies with various project configurations for my research. Updating the operating system almost always meant that then I had to spend time trying to reinstall all of these packages. At some point I made it a habit to keep a running file of everything that I ever installed on the system, so that after a reinstall I could just do yum install $pkg for every package in that file. But, of course, that didn’t work for all dependencies and, after a while, I also forgot to update the file.

So, for large periods of time during my PhD I would be using an out-of-date operating system, not even receiving security updates, only because I was fearing that there won’t be enough time to resolve a missing dependency when discovering that it is not there 2 days before a paper deadline.

This was the second time that Nix showed up: its declarative configuration allows specifying the dependencies you are using and it makes sure that you always get the exact same thing, even if you have already have different version requirements. Unfortunately, that was not the time for experimenting with Nix so it went back to the back burner.

Third time’s the charm 🔗

Last summer, my old laptop that I have been using for a long time started having disk errors. Fortunately, I was planning to switch to a new one, so I migrated all my data with minimal data loss – I know I lost something, as I realized later, but not a significant loss.

After I migrated all this data, I decided to experiment with NixOS on the old laptop, to learn the system and play with it until the laptop will die completely. That was one of the good decisions taken that summer. I got several weekends to toy with the language, to run experiments and learn pitfalls to avoid. I decided to switch to use NixOS as my main distribution at the nearest occasion.

This was last December, immediately after Advent of Code finished. Since then, I am a happy NixOS user.

Why does NixOS work for me? 🔗

I got attracted to Nix because of the possibility of being on the bleeding edge. According to repology Nix related package repositories are far up and to the right, their own cluster. In fact, only Arch is getting close by, but not being as good in terms of freshness, as can be seen from the position of AUR below:

This is possible because the entire set of packages is available as a GitHub repository. Each time I check my GitHub notifications, this repository is the one with the most activity, followed by PyTorch and then a big gap before the remaining watched repositories. While I have not yet contributed any package upgrade or review, I am planning to do so in the future, giving back to community.

But being on the bleeding edge just by itself is not enough, as my story with Arch above says. What is really important here is that I have the assurance that if an upgrade breaks something I can always revert. Either as soon as I detect it, or immediately after a reboot, from the GRUB menu.

Next on the list of reasons why NixOS is good for me is the declarative nature of its configuration. While I am not yet using this to its full extent – I still have to read the manual or peruse the large number of guides (1, 2, 3, 4 as just a selection) or the wiki articles and there’s also a manual just for the packages –, it is already bringing benefits. I have the following in my /etc/nixos/configuration.nix file (with some lines removed):

{ config, pkgs, ... }:

{
  # ...
  networking.hostName = "mithlond";
  time.timeZone = "America/Los_Angeles";
  i18n.defaultLocale = "en_US.UTF-8";

  services.xserver.enable = true;
  services.xserver.videoDrivers = [ "nvidia" ];
  services.xserver.displayManager.gdm.enable = true;
  services.xserver.desktopManager.gnome.enable = true;
  services.xserver = {
    layout = "ro";
    xkbVariant = "";
  };

  services.printing.enable = true;
  sound.enable = true;
  hardware.pulseaudio.enable = false;

  users.users.mihai = {
    isNormalUser = true;
    description = "Mihai Maruseac";
    extraGroups = [ "networkmanager" "wheel" ];
    packages = with pkgs; [ ];
  };

  nixpkgs.config.allowUnfree = true;
  environment.systemPackages = with pkgs; [
    evince
    file
    git
    gnome.gnome-tweaks
    google-chrome
    htop
    jq
    keybase-gui
    ncdu
    ripgrep
    skypeforlinux
    tig
    tree
    unzip
    vim
    wget
  ];
  # ...
}

This allows me full control over what gets installed into the system. I no longer need to take in the default packages that the distro ships with, a large number of which I would never use.

Furthermore, I can back-up this file to external media and if I ever need to install this system (on the same laptop or even somewhere else), I just copy the file to the system and after a few steps will have an exact replica of the current system, with all packages and configurations in place.

It is time to also talk about what are confusing aspects. In the paste above, I could have listed the packages I’m using under users.users.mihai.packages instead of under environment.systemPackages. This way, they would be installed only for my personal user. Or, I could have used HomeManager and configure these in a file in ~, so I wouldn’t need to edit a global config file as root. Or, I could have used nix flakes, enabling the experimental nix command? This gets confusing, and I plan to peruse the manuals and tutorials linked above and slowly migrate to what is the recommended way of things now. At least, I am no longer installing packages via nix-env -i; that would go against having declarative packages – but still bringing in the other benefits of being on the bleeding edge, ability to rollback, ability to have multiple versions of the same package, as needed.

The last line is another great benefit of using Nix. If I need install 2 packages that depend on the same transitive dependency but one uses version A and the other an incompatible version B, I would be in trouble outside of Nix. Installing the second package would break the first one and I would have only noticed that breakage much later, when I needed to use the first one again. I would need to use docker or virtualisation to have both packages. That is needless complexity. With Nix, I can just simply state that I want these packages and be done. Because everything in Nix is a symlink to some path in the Nix store, I will have both A and B versions on my system and the symbolic links are set such that the duplication is minimized.

Some people might remember the scenario above as DLL hell, Cabal hell (in Haskell land) or even the mess that is managing Python environments, as illustrated on XKCD. This is all solved on Nix.

In fact, this also enables one additional super power: I can quickly test new things. For example, if I want to test a new desktop environment or window manager, I change the corresponding line in the above configuration and log in again. Or, for simpler tests, I can just start a new nix shell with the new package, like in nix-shell -p go gimp python3 from one of my recent experiments.

For projects, now I can add a nix.shell file in the directory and whenever I go back to working on them I can just cd to the directory, run nix-shell and get an environment that is using exactly the same stuff as last time I used it. For example, this is the config I used for hindent:

{ pkgs ? import <nixpkgs> {} }:
with pkgs;
mkShell {
  buildInputs = [
    cabal-install
    haskell.compiler.ghc925
    zlib.dev
  ];
  shellHook = ''
    export PS1="[\[\033[01;32m\]nix-shell\[\033[00m\]:\W] \[\033[01;32m\]λ\[\033[00m\] "
  '';
}

There is a customization for the PS1 prompt that is optional, there is also possibility to be more fancy and even have custom dependencies being built when starting the shell. But I haven’t needed it yet, Nix allows one to start using it without going into full details about how it works.

I don’t need to remember the package names, there is a search index and another one for configuration options. I can see licenses, where the package / config is declared, examples, maintainers, environments where the package works, etc. All of this before I even attempt to install it.

Updating the system is simple, I need to run nix-channel --update to get the latest state of nixpkgs and then nixos-rebuild switch to install newer versions of the installed packages and immediately switch to the new configuration:

[nixos-and-me] λ nixos-rebuild switch
this derivation will be built:
  /nix/store/v0skn74pflg1ajij8hdx7r27lcci3ydk-nixos-rebuild.drv
these 61 paths will be fetched (22.96 MiB download, 104.26 MiB unpacked):
  /nix/store/17k0adcgdz7xvsa38x8pry6rassg07ys-libarchive-3.6.2-lib
  /nix/store/2hh33wn2h6ckmwb4vjvvnxxnbw4255cd-nlohmann_json-3.11.2
  ...
  /nix/store/zypaxh4a67vf4h4ks27hgw5mkvxz9ab6-aws-c-mqtt-0.7.13
copying path '/nix/store/xgi6q4a0ib2lzf0by8jny65ny8fpp4a1-busybox-static-x86_64-unknown-linux-musl-1.35.0' from 'https://cache.nixos.org'...
copying path '/nix/store/fb79s109m7vcirzia8qhblvkzkj4w51z-nix-2.11.1-man' from 'https://cache.nixos.org'...
...
copying path '/nix/store/znn8hhrmr3vy06vi80ffwj74rsn5d9af-nix-2.11.1' from 'https://cache.nixos.org'...
building '/nix/store/v0skn74pflg1ajij8hdx7r27lcci3ydk-nixos-rebuild.drv'...
building Nix...
building the system configuration...
these 283 derivations will be built:
  /nix/store/08xinpflxrfj45ankybkbryx0hr15r31-builder.pl.drv
  ...
  /nix/store/zxynhp7kcwpw2lh9fff7qf8sv6rw07x4-libdrm-2.4.113
copying path '/nix/store/hd4b9bpyf3xz9q1s7907gvmwdf5d2cx3-glibc-locales-2.35-224' from 'https://cache.nixos.org'...
copying path '/nix/store/z6vmhagxd308pffxnx47pvi27pbi5y5q-linux-5.15.102' from 'https://cache.nixos.org'...
...
setting up /etc...
restarting systemd...
reloading user units for mihai...
setting up tmpfiles
reloading the following units: dbus.service, firewall.service, reload-systemd-vconsole-setup.service
restarting the following units: nix-daemon.service, polkit.service, systemd-journald.service
starting the following units: ModemManager.service, NetworkManager-wait-online.service, ...
the following new units were started: NetworkManager-dispatcher.service, ...

If there is an error in building, the system stays in the old state. If the new state does not work, I can always switch back to the previous version.

Finally, one more thing left to do: after so many experiments, updates, etc., we might have a lot of packages no longer referenced by other places in Nix. So, we can also collect this garbage (which reminds me, I haven’t done so since I installed Nix, so let’s do a live demo).

[nixos-and-me] λ nix-collect-garbage -d
...
deleting '/nix/store/2x0h67cqxa7bnffb63q2scly64rb2nlp-env_var_for_system_dir-ff86.patch'
deleting unused links...
note: currently hard linking saves -13.87 MiB
19004 store paths deleted, 32708.09 MiB freed

That is nearly 32GB of disk :o

I will probably do deeper dives into Nix as I explore the system more or read the manuals/guides, but for now this article should be enough. Happy 20th anniversary!

PS: for a very deep-dive, see this video series by Tweag