I recently bought a little server on ebay (a Lenovo ThinkCentre M93p Tiny) to use for some "home lab" stuff, i.e. to set up with some media and a Plex server. I wanted it to route all of its traffic through a VPN. I happen to already be subscribed to two separate VPN services, Mozilla VPN and Proton VPN. They both provide Linux apps, and both can be easily installed on NixOS, Mozilla VPN with a typical service declaration, while Proton VPN just has a package definition.

Unfortunately, I'm mostly running this little guy headless, and I couldn't get either of those to reliably run on a headless server. Proton VPN, though, makes it convenient to download a wireguard config file, so I decided to go the route of directly configuring WireGuard. The NixOS wiki on WireGuard got me most of the way there, but there were still some bits that I wound up having to figure out for myself, so I wanted to document the process here.

Initially, I tried to use what seemed to me to be the simpler option of just using networking.wireguard. However, I just could not get this to work at all. I think that the default config as described in that link winds up looping traffic? I'm not 100% sure, but I was reliably able to break the internet by turning on the WireGuard interface, so I guess that was progress.

Next I tried the systemd.network approach. This was more promising off the bat, getting me to a point where, on the box, I could easily swap back and forth between having the VPN on and off via networkctl up/down wg0 and validating my IP with curl -4 ipecho.net/plain (or curl -6). The main problem I ran into once I got it working was that I locked myself out of SSH, which made headless management challenging.

Ultimatley, this is what I wound up with, which solves all of the problems nicely. I've annotated things up a fair bit here, and retained some comments directly from the wiki.

{
  pkgs,
  lib,
  config,
  ...
}:
let
  wireguardEndpoint = "89.187.178.173";
  wireguardPort = 51820;
in
{
  networking.firewall.enable = true;
  # use systemd-networkd
  networking.useNetworkd = true;
  # open the wireguard port, obviously
  networking.firewall.allowedUDPPorts = [ wireguardPort ]; # wireguard

  # Stolen from the wiki, which says "NixOS firewall will block wg traffic because of rpfilter"
  networking.firewall.checkReversePath = "loose";

  # Here, we enable systemd.network, set up the wireguard device, and
  # then define the wireguard network.
  systemd.network = {
    enable = true;
    # Wireguard device
    netdevs."50-wg0" = {
      # name & type, which will show up in like 'nmcli conn' or whatever
      netdevConfig = {
        Kind = "wireguard";
        Name = "wg0";
      };

      wireguardConfig = {
        ListenPort = wireguardPort;
        # Ensure file is readable by `systemd-network` user. The Wiki
        # uses 'age' for this, which is undoubtedly better, but to
        # get things working I just SSH'ed my key over, put it
        # somewhere everyone could read, and chowned it for the
        # network user.
        PrivateKeyFile = "/etc/wg.key";
        # To automatically create routes for everything in AllowedIPs,
        # add RouteTable=main
        RouteTable = "main";
        # FirewallMark marks all packets send and received by wg0
        # with the number 42, which can be used to define policy rules on these packets.
        FirewallMark = 42;
      };

      # Tell wireguard about the VPN server.
      wireguardPeers = [
        {
          # Copied from the downlaoded wireguard config file.
          PublicKey = "/KM6QESKJRK7GiMqWstUl1Yn9pzc6DPzqCtNauxYgn8=";
          AllowedIPs = [
            # proxy all traffic
            "0.0.0.0/0"
            "::/0"
          ];

          # Routing all DNS over WireGuard (i.e. Domains=~.) will prevent the DNS resolution of endpoints.
          Endpoint = "${wireguardEndpoint}:${toString (wireguardPort)}";

          # RouteTable line specifies that a new routing table with id 1000 is created
          # for the wireguard interface, and no rules are set on the main routing table.
          RouteTable = 1000;
        }
      ];
    };
    # Wireguard network
    networks."50-wg0" = {
      # Does this match against the device name? I'm not sure.
      matchConfig.Name = "wg0";

      # The addresses in the wireguard config file of the server.
      # /32 and /128 specifies a single address for use on this wg peer machine
      address = [
        "10.2.0.2/32"
        "2a07:b944::2:2/128"
      ];

      # This initial set of rules is copied mostly verbatim from the wiki.
      routingPolicyRules = [
        # rule 1: redirect traffic
        {
          Family = "both"; # ipv4 & 6
          # For all packets *not* marked with 42 (i.e. all non-wireguard/normal traffic),
          InvertRule = true;
          FirewallMark = 42;

          # (... continued) we specify that the routing table 1000 must be used
          # (which is the wireguard routing table). This rule routes all traffic through wireguard.
          # inside routingPolicyRules section is called Table, not RouteTable
          Table = 1000;

          # this routing policy rule has a lower priority (10) than
          # endpoint exclusion rule (5).
          Priority = 10;
        }
        # rule 2: exclude endpoint ip
        {
          # Use a routing policy rule to exclude the endpoint IP address,
          # so that wireguard can still connect to it.
          # it has a higher priority (5) than (10).

          # We exempt our endpoint with a higher priority by routing it
          # through the main table (Table=main is default).
          To = "${wireguardEndpoint}/32";
          Priority = 5;
        }
      ]
      # The rest of this is my addition: for any services that you
      # want to be able to connect to your box directly, you will need
      # to exclude them from the VPN rule. Below are examples for
      # SSH and transmission.

      # If SSH is enabled
      ++ (lib.optionals config.services.openssh.enable (
        # For each SSH port, add a rule for that port to route through the
        # 'main' table.
        lib.lists.forEach config.services.openssh.ports (port: {
          SourcePort = port;
          Priority = 3;
          Table = "main";
        })
      ))
      # If transmission is enabled, route its port through the 'main' table.
      ++ lib.optionals config.services.transmission.enable [
        {
          SourcePort = config.services.transmission.settings.rpc_port;
          Priority = 3;
          Table = "main";
        }
      ];
    };
  };
}

It's utterly classic for NixOS that this seems so simple now that I have figured it out and gotten it encoded in config, but that config represents probably four hours of distilled experimentation, so hopefully it will be of use to someone.

You can find the real live config here. Of course, it may change over time or even move, so if you want to see the config at the time of this blogpost, it's commit 9336e005.