18. Apr 2017

Arch Linux on an encrypted ZFS root system

linux zfs howto

Note as of October 2019🔗

The native ZFS encryption introduced in version 0.8 is working quite fine and you can use it for a root file system. With this in mind this guide is probably a bit outdated.

This blog entry is my personal documentation. If it helps others even better. I will try to keep it as up to date as possible. What helped me when I tried this the first time was the github gist by codedreality. Since then I modified and adapted his procedure a bit to fit my personal needs.

The goal is to have an encrypted Arch Linux running on ZFS as root file system with an additional swap partition to enable hibernation. As bootloader rEFInd will be used.

Please read this guide with caution and adjust device paths and partition sizes according to your system and needs!

This guide is written with a single disk in mind. If you want to use mirroring or a raidz approach you need to create the partitions and cryptcontainers on all disks.

In the end we want it to look something like this:

$ lsblk
NAME              MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda               259:0    0 238.5G  0 disk		# Disk
├─sda1            259:1    0   512M  0 part  /boot	# boot / efi
└─sda2            259:2    0 235.6G  0 part		# encrypted volume
  └─cryptoroot    254:0    0 235.6G  0 crypt
    ├─cryptoroot1 254:1    0   8.5G  0 part  [SWAP]	# encrypted swap
    └─cryptoroot2 254:2    0 227.1G  0 part		# encrypted ZFS

Need to troubleshoot? Check the troubleshooting section at the end.

Preparations🔗

The regular Archiso images don't come with the ZFS packages. While writing this guide I ran into problems when installing them during install. Therefore my advice is to prepare a custom iso to boot from. See John Ramsden's guide in combination with the official Arch Linux wiki entry Archiso.

Getting started🔗

Once the live CD is booted up we are logged in as root we need a network connection. If you are only able to use wifi, run wifi-menu. To begin the install process let's create partitons. The first is for the EFI bootloader and the initramfs. The second is for the LUKS container.

$ parted /dev/sda
(parted) mklabel gpt
(parted) mkpart ESP fat32 1MiB 513MiB
(parted) set 1 boot on
(parted) mkpart primary ext2 513MiB 99%

Once the partitions are created and the first is set as boot partition, let's check the alignment of the partitions:

(parted) align-check optimal 1
1 aligned
(parted) align-check optimal 2
2 aligned

Format the EFI partition with FAT32.

$ mkfs.fat -F32 /dev/sda1

Create the LUKS container🔗

Run the following command to create the LUKS container.

$ cryptsetup luksFormat /dev/sda2

Follow the instructions and remember the passphrase! It can be changed later. Once the encrypted container is created let's open it under the name cryptoroot.

$ cryptsetup luksOpen /dev/sda2 cryptoroot

Swap partition🔗

Now that we have the cryptoroot container open we can start to partition it as well. It is possible to have the swap in a ZVOL but currently you cannot resume from hibernation with that. Therefore we need to split up the crypto container into a partition that will contain ZFS and another that will be used for swap.

(parted) mklabel gpt
(parted) mkpart ext2 0% 512MiB
(parted) mkpart ext2 512MiB 100%

The 512MiB for the swap partition are just an example! For hibernation make sure that it is at least as big as your systems amount of RAM!

$ mkswap /dev/mapper/cryptoroot1
$ swapon /dev/mapper/cryptoroot1

Set up ZFS🔗

Create the zpool.cache file.

$ touch /etc/zfs/zpool.cache

Before creating the ZFS pool make sure which sector size your disk is using. You can use parted for this. For example

$ parted /dev/sda
(parted) print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 10.7GB
Sector size (logical/physical): 512B/512B <--- here
Partition Table: gpt
...

Should you have a 4k disk then add -o ashift=12 to the zpool create command.

$ zpool create -o cachefile=/etc/zfs/zpool.cache -m none -R /mnt zroot /dev/mapper/cryptoroot2

Create ZFS filesystems🔗

Let's create the ZFS filesystems in the new pool.

$ zfs create -o mountpoint=none -o compression=lz4 zroot/ROOT
$ zfs create -o mountpoint=/ zroot/ROOT/default
$ zfs create -o mountpoint=/opt zroot/opt
$ zfs create -o mountpoint=/home zroot/home
$ zfs create -o mountpoint=/root zroot/home/root

$ zpool set bootfs=zroot zroot

Export and reimport the pool:

$ zpool export zroot
$ zpool import -R /mnt zroot

Mount the boot partition to the correct location. The output of blkid /dev/sda1 can give you the UUID. Be careful not to mix it up with the PARTUUID.

$ mkdir /mnt/boot
$ mount /dev/disk/by-uuid/UUID_OF_DISK /mnt/boot

Install Base System🔗

$ pacstrap -i /mnt base base-devel

Generate fstab entries🔗

$ genfstab -U -p /mnt | grep boot >> /mnt/etc/fstab
$ genfstab -U -p /mnt | grep swap >> /mnt/etc/fstab

Chroot🔗

Now is the time to chroot into the new system and get it set up.

$ arch-chroot /mnt /bin/bash

Other stuff🔗

Let's get other stuff set up. For more infos on the steps take a look at the Arch Linux Installation guide.

Locale🔗

Uncomment needed locale in /etc/locale.gen and generate them with

$ locale-gen

Set LANG variable in /etc/locale.conf according to what has been uncomented.

# etc/locale.conf:
#-----------------
LANG=en_US.UTF-8

Time zone🔗

Check /usr/share/zoneinfo/* for suited time zones.

$ ln -s /usr/share/zoneinfo/Europe/Vienna /etc/localtime

Hardware Clock🔗

$ hwclock --systohc --utc

NTP Client🔗

Install ntp

$ pacman -S ntp

Add pools to /etc/ntp.conf and sync manually with

$ ntpd -q

Then save it to the HW clock with

$ hwclock -w

Install other software🔗

Install whatever basic software you need.

$ pacman -S tmux vim rsync ...

Finish ZFS🔗

First we need to install the ZFS repos in the fresh install as well. Therefore we need to add the repository to /etc/pacman.conf.

# /etc/pacman.conf:
#------------------
[archzfs]
Server = http://archzfs.com/$repo/x86_64

Also don't forget to add and sign the keys of the repo:

$ pacman-key -r F75D9D76
$ pacman-key --lsign-key 0 F75D9D76

Update the package database and install the zfs package.

$ pacman -Syy
$ pacman -S zfs-linux

Enable the ZFS services.

Update: As noted in the Archwiki on ZFS root we need to enable a few more services!

$ systemctl enable zfs.target
$ systemctl enable zfs-import-cache
$ systemctl enable zfs-mount
$ systemctl enable zfs-import.target

Bootup Hooks🔗

We need to create a small hook for bootup to run the partprobe command on the LUKS container. Install parted to get partprobe and create two files:

$ pacman -S parted

Create /etc/initcpio/install/load_part:

# /etc/initcpio/install/load_part:
#---------------------------------
#!/bin/bash

build() {
        add_binary 'partprobe'

        add_runscript
}

help() {
        cat <<HELPEOF
Probes mapped LUKS container for partitions.
HELPEOF
}

Create /etc/initcpio/hooks/load_part:

# /etc/initcpio/hooks/load_part:
#------------------------------
run_hook() {
        partprobe /dev/mapper/cryptoroot
}

MKINITCPIO🔗

Now let's edit /etc/mkinitcpio.conf. Look for the line with HOOKS=... and modifiy it to resemble

HOOKS="base udev autodetect modconf block keyboard encrypt load_part resume zfs filesystems"

Basically we need keyboard loaded before encrypt, after this our custom partprobe hook and then ZFS.

Let's create the new boot image by running

$ mkinitcpio -p linux

At the time of writing this guide there was an error about zlib_deflate wich can be ignored.

Bootloader🔗

We will be using rEFInd as bootloader. Let's install it and get it set up.

$ pacman -S refind-efi
$ refind-install

Now we need to change the boot parameters in /boot/refind_linux.conf.

# /boot/refind_linux.conf:
#-------------------------
"Boot with defaults" "cryptdevice=/dev/disk/by-uuid/<uuid>:cryptoroot zfs=zroot/ROOT/default rw resume=UUID=<swap UUID>"

To tell the kernel which blockdevice to decrypt on boot the cryptdevice parameter is used. Run blkid /dev/sda2 to get the UUID of the encrypted partition. For hibernating to work the kernel needs to know which block device is used as swap with the resume parameter. The output of blkid /dev/mapper/cryptoroot1 will return the UUID of the swap space.

Wrap up the install🔗

We are almost done. Just let us set the last important things.

Hostname🔗

Set the hostname in /etc/hostname and /etc/hosts.

# /etc/hostname:
#---------------
myhost

# /etc/hosts:
#------------
#
# /etc/hosts: static lookup table for host names
#

#<ip-address>	<hostname.domain.org>	<hostname>
127.0.0.1	localhost.localdomain	localhost
::1		localhost.localdomain	localhost
127.0.1.1	myhost.local		myhost

# End of file

The 127.0.1.1 line is there in case the computer is changing IPs or getting them via DHCP. If you have a manual IP address on that host use it instead of the 127.0.0.1.

Root password🔗

Set the root password.

$ passwd

Wifi🔗

In case your only network connection is via wifi install iw, wpa_supplicant and dialog in order to be able to use wifi-menu on the freshly installed system.

$ pacman -S iw wpa_supplicant dialog

Finish🔗

Exit the chroot environment. Copy over the zpool.cache file.

$ cp /etc/zfs/zpool.cache /mnt/etc/zfs/

Unmount the boot partition and export the pool.

$ umount /mnt/boot
$ zpool export zroot

Reboot!

Closing lines🔗

You should now be able to boot into your fresh encrypted Arch Linux running on ZFS.

To read up on how to proceed from here

check the General recommendations Arch wiki page
consider a nicer appearance of rEFInd by theming it
make use of automated ZFS snapshots
set up a scrub job to catch errors early

Troubleshooting🔗

Something has gone wrong? You forgot something in the install process? Need to access the system for some other reason without booting it?

Run the following steps after you boot from a USB drive or attach the disk to another system.

Open the encrypted partition. cryptsetup luksOpen /dev/sda2 cryptoroot.
Scan for the partitions within the LUKS container. partprobe /dev/mapper/cryptoroot.
Use the swap space (optional). swapon /dev/mapper/cryptoroot1
zpool import should find the zroot pool. Import it and mount everything relative to the mount path. zpool import -R /mnt zroot
Mount the boot partition accordingly before you chroot. mount /dev/sda1 /mnt/boot

Now you should be able to access your installed system and do whatever it is you need to.

Got any hints or questions? blog@aaronlauterer.com

Aaron Lauterer