Bootstrapping OpenSSH: Remote Disk Decryption #
This is part two (of two) of my journey to remote unlocking encrypted (LUKS) root partitions using SSH during early boot. Here we’ll configure dracut to include the OpenSSH daemon in early boot to type in the passphrase remotely. In the first part, I outlined the steps required to configure a system to use dracut instead of mkinitcpio. If you want to read about that, you can find the first part here: Goodbye mkinitcpio, hello dracut!
If you just want the details of what files need to be modified or created, which packages to be installed and stuff, you can skip to this section here.
What and why? #
Everyone should be using full disk encryption (FDE) at this point. If you aren’t, then you really should be! Otherwise, anyone with physical access to your computer can access all your files. And no, user passwords on linux don’t really mean anything if you have physical system access. Take a look at the concept of chroot-ing and you’ll understand why. But FDE also comes with its own difficulties. Assuming one is using LUKS for FDE, there are many options how to protect the disk: passphrases, keyfiles, FIDO2 tokens, smartcards, etc. And all either require physical access at boot time, or decrese the FDE-security.
Here, we’ll simply look at setups using passphrases which a system prompts for at boot time. Depending on the partition, either at early boot, or late boot. Essentially, if the disk is required to access the root filesystem and build up the actual userspace, then it gets prompted for at early boot. Otherwise at late boot. If your FDE disk or partition contains the root partition, then it will be early boot. And that is what this guide assumes, because in this case, the boot process is actively blocked by the FDE prompt and won’t continue without supplying a valid passphrase. But especially for servers, it is not always possible to have physical access to the system during (early) boot. And therefore, a solution would be to start an SSH daemon during early boot.
Two fairly wide-spread SSH daemons, Dropbear and tinyssh are commonly used in scenarios where disk space is limited. Because boot partitions and initramfs image sizes are often limited, many people fall back to one of these solutions and simply include them as a module in initramfs. But Dropbear is known for implementing a very old and insecure SSH version without support for modern key algorithms, including FIDO2-based ones. And tinyssh, while more modern, is less known, a lot younger and way less tried-and-tested than OpenSSH. And considering that disk space is a lot less of a problem nowadays, even for embedded systems, adding a few dozens of extra megabytes to the initramfs image doesn’t really make the difference. So one might as well include a full OpenSSH daemon which supports modern, tried and tested key algorithms.
So with that out of the way, let’s look at what steps are actually required.
Prerequisites #
This configuration assumes your system is using the dracut initramfs builder and a root partition with FDE setup.
Before we can start up an SSH daemon at early boot, we’ll need to ensure that a networking (IP) stack is up and running. This include setting up a network interface, too. We’ll be using systemd-networkd, but legacy-network and other network-modules for dracut should work too. For starters, we’ll need to create a network configuration for systemd-networkd. The following configuration assumes you are using an ethernet interface that starts with e, like eth0 or enp1s0. If you use a wireless network interface, you need to change this to respect that. We’ll store this config in /etc/systemd/network/20-wired.network:
[Match]
e*
[Network]
DHCP=ipv4
Next we’ll ensure that dracut actually sets the networking stack up at early boot. To do that, we simply create a file like /etc/dracut.d/sshd.conf with the following contents to start systemd-networkd at early boot and include the network configuration from above:
+=" /etc/systemd/network/20-wired.network "
+=" systemd-networkd "
Setting up SSHD #
Next, we’ll need a dracut module that includes the OpenSSH daemon in the initramfs and loads it at early boot. Luckily, a person named Georg Sauthoff did the work and made a publicly available dracut-sshd module on GitHub. It is very simple and easy to configure. But first, we’ll start by cloning the repository and copying the module directory 46sshd to /usr/lib/dracut/modules.d/:
# also available as a copr repository
This module reuses the host’s sshd private keys and includes them in the initramfs image. Depending on your threat model, this might be a problem. After all, if one’s root partition is encrypted, one might not want its secrets to be stored in the boot image unencrypted. If one doesn’t want their keys at /etc/ssh/ssh_host_*_key{,.pub} to be used, simply add new keys at /etc/ssh/dracut_ssh_host_*_key{,.pub}. If these exist, the module will use them instead. But be aware: you might want to consider creating a separate SSH config host entry for the early-boot daemon, or you’ll be greated by an host-key-missmatch error everytime you connect.
Similarly for the authorized keys, the modules looks at different locations; in order of precedence:
/root/.ssh/dracut_authorized_keys/etc/dracut-sshd/authorized_keys/root/.ssh/authorized_keys
Remember to adjust the permissions of all these files accordingly, i.e. only read- and writeable by root:
By default, the early-boot SSH daemon listens on port 22. If you want to change this, create a file /etc/sysconfig/dracut-sshd that defines SSHD_OPTS with the option -p 22 (and possible other sshd-options as desired). And that should be it. All that’s left now, is to rebuild the initramfs image using dracut (and possible other arguments depending on your setup):
And if everything went well, on the next boot, while the passphrase prompt is displayed on screen, an OpenSSH daemon should be running and listening for connections. Simply connect to it, and it should give you the instruction to run systemd-tty-ask-password-agent and supply it with your respective encryption passphrase. You can also use systemd-tty-ask-password-agent --list to display a list of all pending prompts.