Arch Linux on an encrypted ZFS root system
Note as of October 2019
The native ZFS encryption introduced in version 0.8 is working quite fine and you can use it for a root file system. With this in mind this guide is probably a bit outdated.
This blog entry is my personal documentation. If it helps others even better. I will try to keep it as up to date as possible. What helped me when I tried this the first time was the github gist by codedreality. Since then I modified and adapted his procedure a bit to fit my personal needs.
The goal is to have an encrypted Arch Linux running on ZFS as root file system with an additional swap partition to enable hibernation. As bootloader rEFInd will be used.
Please read this guide with caution and adjust device paths and partition sizes according to your system and needs!
This guide is written with a single disk in mind. If you want to use mirroring or a raidz approach you need to create the partitions and cryptcontainers on all disks.
In the end we want it to look something like this:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 259:0 0 238.5G 0 disk # Disk
├─sda1 259:1 0 512M 0 part /boot # boot / efi
└─sda2 259:2 0 235.6G 0 part # encrypted volume
└─cryptoroot 254:0 0 235.6G 0 crypt
├─cryptoroot1 254:1 0 8.5G 0 part [SWAP] # encrypted swap
└─cryptoroot2 254:2 0 227.1G 0 part # encrypted ZFS
Need to troubleshoot? Check the troubleshooting section at the end.
Preparations
The regular Archiso images don't come with the ZFS packages. While writing this guide I ran into problems when installing them during install. Therefore my advice is to prepare a custom iso to boot from. See John Ramsden's guide in combination with the official Arch Linux wiki entry Archiso.
Getting started
Once the live CD is booted up we are logged in as root we need a network connection. If you are only able to use wifi, run wifi-menu. To begin the install process let's create partitons. The first is for the EFI bootloader and the initramfs. The second is for the LUKS container.
$ parted /dev/sda
(parted) mklabel gpt
(parted) mkpart ESP fat32 1MiB 513MiB
(parted) set 1 boot on
(parted) mkpart primary ext2 513MiB 99%
Once the partitions are created and the first is set as boot partition, let's check the alignment of the partitions:
(parted) align-check optimal 1
1 aligned
(parted) align-check optimal 2
2 aligned
Format the EFI partition with FAT32.
$ mkfs.fat -F32 /dev/sda1
Create the LUKS container
Run the following command to create the LUKS container.
$ cryptsetup luksFormat /dev/sda2
Follow the instructions and remember the passphrase! It can be changed later.
Once the encrypted container is created let's open it under the name cryptoroot.
$ cryptsetup luksOpen /dev/sda2 cryptoroot
Swap partition
Now that we have the cryptoroot container open we can start to partition it as well. It is possible to have the swap in a ZVOL but currently you cannot resume from hibernation with that. Therefore we need to split up the crypto container into a partition that will contain ZFS and another that will be used for swap.
(parted) mklabel gpt
(parted) mkpart ext2 0% 512MiB
(parted) mkpart ext2 512MiB 100%
The 512MiB for the swap partition are just an example! For hibernation make sure that it is at least as big as your systems amount of RAM!
$ mkswap /dev/mapper/cryptoroot1
$ swapon /dev/mapper/cryptoroot1
Set up ZFS
Create the zpool.cache file.
$ touch /etc/zfs/zpool.cache
Before creating the ZFS pool make sure which sector size your disk is using. You can use parted for this. For example
$ parted /dev/sda
(parted) print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 10.7GB
Sector size (logical/physical): 512B/512B <--- here
Partition Table: gpt
...
Should you have a 4k disk then add -o ashift=12 to the zpool create command.
$ zpool create -o cachefile=/etc/zfs/zpool.cache -m none -R /mnt zroot /dev/mapper/cryptoroot2
Create ZFS filesystems
Let's create the ZFS filesystems in the new pool.
$ zfs create -o mountpoint=none -o compression=lz4 zroot/ROOT
$ zfs create -o mountpoint=/ zroot/ROOT/default
$ zfs create -o mountpoint=/opt zroot/opt
$ zfs create -o mountpoint=/home zroot/home
$ zfs create -o mountpoint=/root zroot/home/root
$ zpool set bootfs=zroot zroot
Export and reimport the pool:
$ zpool export zroot
$ zpool import -R /mnt zroot
Mount the boot partition to the correct location. The output of blkid /dev/sda1 can give you the UUID. Be careful not to mix it up with the PARTUUID.
$ mkdir /mnt/boot
$ mount /dev/disk/by-uuid/UUID_OF_DISK /mnt/boot
Install Base System
$ pacstrap -i /mnt base base-devel
Generate fstab entries
$ genfstab -U -p /mnt | grep boot >> /mnt/etc/fstab
$ genfstab -U -p /mnt | grep swap >> /mnt/etc/fstab
Chroot
Now is the time to chroot into the new system and get it set up.
$ arch-chroot /mnt /bin/bash
Other stuff
Let's get other stuff set up. For more infos on the steps take a look at the Arch Linux Installation guide.
Locale
Uncomment needed locale in /etc/locale.gen and generate them with
$ locale-gen
Set LANG variable in /etc/locale.conf according to what has been uncomented.
# etc/locale.conf:
#-----------------
LANG=en_US.UTF-8
Time zone
Check /usr/share/zoneinfo/* for suited time zones.
$ ln -s /usr/share/zoneinfo/Europe/Vienna /etc/localtime
Hardware Clock
$ hwclock --systohc --utc
NTP Client
Install ntp
$ pacman -S ntp
Add pools to /etc/ntp.conf and sync manually with
$ ntpd -q
Then save it to the HW clock with
$ hwclock -w
Install other software
Install whatever basic software you need.
$ pacman -S tmux vim rsync ...
Finish ZFS
First we need to install the ZFS repos in the fresh install as well. Therefore we need to add the repository to /etc/pacman.conf.
# /etc/pacman.conf:
#------------------
[archzfs]
Server = http://archzfs.com/$repo/x86_64
Also don't forget to add and sign the keys of the repo:
$ pacman-key -r F75D9D76
$ pacman-key --lsign-key 0 F75D9D76
Update the package database and install the zfs package.
$ pacman -Syy
$ pacman -S zfs-linux
Enable the ZFS services.
Update: As noted in the Archwiki on ZFS root we need to enable a few more services!
$ systemctl enable zfs.target
$ systemctl enable zfs-import-cache
$ systemctl enable zfs-mount
$ systemctl enable zfs-import.target
Bootup Hooks
We need to create a small hook for bootup to run the partprobe command on the LUKS container.
Install parted to get partprobe and create two files:
$ pacman -S parted
Create /etc/initcpio/install/load_part:
# /etc/initcpio/install/load_part:
#---------------------------------
#!/bin/bash
build() {
add_binary 'partprobe'
add_runscript
}
help() {
cat <<HELPEOF
Probes mapped LUKS container for partitions.
HELPEOF
}
Create /etc/initcpio/hooks/load_part:
# /etc/initcpio/hooks/load_part:
#------------------------------
run_hook() {
partprobe /dev/mapper/cryptoroot
}
MKINITCPIO
Now let's edit /etc/mkinitcpio.conf. Look for the line with HOOKS=... and modifiy it to resemble
HOOKS="base udev autodetect modconf block keyboard encrypt load_part resume zfs filesystems"
Basically we need keyboard loaded before encrypt, after this our custom partprobe hook and then ZFS.
Let's create the new boot image by running
$ mkinitcpio -p linux
At the time of writing this guide there was an error about
zlib_deflatewich can be ignored.
Bootloader
We will be using rEFInd as bootloader. Let's install it and get it set up.
$ pacman -S refind-efi
$ refind-install
Now we need to change the boot parameters in /boot/refind_linux.conf.
# /boot/refind_linux.conf:
#-------------------------
"Boot with defaults" "cryptdevice=/dev/disk/by-uuid/<uuid>:cryptoroot zfs=zroot/ROOT/default rw resume=UUID=<swap UUID>"
To tell the kernel which blockdevice to decrypt on boot the cryptdevice parameter is used. Run blkid /dev/sda2 to get the UUID of the encrypted partition. For hibernating to work the kernel needs to know which block device is used as swap with the resume parameter. The output of blkid /dev/mapper/cryptoroot1 will return the UUID of the swap space.
Wrap up the install
We are almost done. Just let us set the last important things.
Hostname
Set the hostname in /etc/hostname and /etc/hosts.
# /etc/hostname:
#---------------
myhost
# /etc/hosts:
#------------
#
# /etc/hosts: static lookup table for host names
#
#<ip-address> <hostname.domain.org> <hostname>
127.0.0.1 localhost.localdomain localhost
::1 localhost.localdomain localhost
127.0.1.1 myhost.local myhost
# End of file
The 127.0.1.1 line is there in case the computer is changing IPs or getting them via DHCP. If you have a manual IP address on that host use it instead of the 127.0.0.1.
Root password
Set the root password.
$ passwd
Wifi
In case your only network connection is via wifi install iw, wpa_supplicant and dialog in order to be able to use wifi-menu on the freshly installed system.
$ pacman -S iw wpa_supplicant dialog
Finish
Exit the chroot environment. Copy over the zpool.cache file.
$ cp /etc/zfs/zpool.cache /mnt/etc/zfs/
Unmount the boot partition and export the pool.
$ umount /mnt/boot
$ zpool export zroot
Reboot!
Closing lines
You should now be able to boot into your fresh encrypted Arch Linux running on ZFS.
To read up on how to proceed from here
- check the General recommendations Arch wiki page
- consider a nicer appearance of rEFInd by theming it
- make use of automated ZFS snapshots
- set up a scrub job to catch errors early
Troubleshooting
Something has gone wrong? You forgot something in the install process? Need to access the system for some other reason without booting it?
Run the following steps after you boot from a USB drive or attach the disk to another system.
- Open the encrypted partition.
cryptsetup luksOpen /dev/sda2 cryptoroot. - Scan for the partitions within the LUKS container.
partprobe /dev/mapper/cryptoroot. - Use the swap space (optional).
swapon /dev/mapper/cryptoroot1 zpool importshould find thezrootpool. Import it and mount everything relative to the mount path.zpool import -R /mnt zroot- Mount the boot partition accordingly before you chroot.
mount /dev/sda1 /mnt/boot
Now you should be able to access your installed system and do whatever it is you need to.