Debian 11 LUKS mod: hang/freeze when init is supposed to start

Question

I'm trying to modify a Debian 11 Raspi image to boot from a LUKS device. When booting, the last lines being logged are HID (input device) detection, and after a couple of seconds:

kernel: random: crng init done

After that boot hangs. No prompt. No reaction to keypresses except Ctrl + Alt + Del (halt).

I followed the following answer's recipe to modify the Debian image to boot from a LUKS device:

https://askubuntu.com/questions/1287837/luks-disk-encryption-on-raspberry-pi-4-and-ubuntu-desktop-20-10

What I made different:

• Pre modification

  • installed cryptsetup-initramfs
  • added CRYPTSETUP=y to /etc/cryptsetup-initramfs/conf-hook and update-initramfs

• From the recipe I deviated by

  • Storing the root partition to a .tar
  • Formatting a LUKS device on the root partition
  • Using another cipher (aes-xts-plain64)
  • Using LUKS type 1
  • Opening the LUKS device
  • Formatting EXT4 partition on the mapped device
  • Fixed the cryptdevice name after : to equal to the mapper name, not "sdcard"

Question

Apparently init does not execute or crashes early. Adding debug to cmdline.txt does not show much more infos, except for this:

[5.12345] Run /init as init process
[5.12345]   with arguments:
[5.12345]     /init
[5.12345]   with environment:
[5.12345]     HOME=/
[5.12345]     TERM=linux
[5.12345]     cryptdevice=/dev/mmcblk1p2:rpirootfs

Are there any obvious errors? How can I further debug at what point / for what reason the start stop?

My scripts

Pre modification:

apt-get update -y
apt-get upgrade -y

cat <<-EOF >>/etc/initramfs-tools/modules
    algif_skcipher
    xchacha12
    adiantum
    aes_arm
    sha256
    nhpoly1305
    dm-crypt
EOF

echo “CRYPTSETUP=y” >> /etc/cryptsetup-initramfs/conf-hook

apt-get install cryptsetup-initramfs

update-initramfs -c -u

Encryption script

LUKS_CIPHER="aes-xts-plain64"
LUKS_NAME="rpirootfs"

# Inside Raspberry Pi
RPI_ROOT_DISK_NAME="/dev/mmcblk1p2"

MOUNT_RPI="/mnt"
PACKED_RPI="/media/ramdisk/rpi.tar"

luks_mapper_device="/dev/mapper/${LUKS_NAME}"

mount ${disk} ${MOUNT_RPI}

tar -C ${MOUNT_RPI} -Pcf ${PACKED_RPI} .

umount ${disk}

cryptsetup luksFormat -c ${LUKS_CIPHER} -s 256 -h sha512 --use-urandom ${disk}

cryptsetup luksOpen ${disk} ${LUKS_NAME}

mkfs.ext4 ${luks_mapper_device}

mount ${luks_mapper_device} ${MOUNT_RPI}

tar -C ${MOUNT_RPI} -xf ${PACKED_RPI}

echo "${LUKS_NAME} ${RPI_ROOT_DISK_NAME} none luks" > ${MOUNT_RPI}/etc/crypttab
echo "${luks_mapper_device} / ext4 defaults,noatime 0 0" > ${MOUNT_RPI}/etc/fstab

umount ${luks_mapper_device}

cryptsetup close ${LUKS_NAME}

mount ${boot_disk} ${MOUNT_RPI}

sed -i -r "s;root=LABEL=[^ ]+ ;root=${luks_mapper_device} ;" ${MOUNT_RPI}/cmdline.txt
sed -i -r 's/splash//' ${MOUNT_RPI}/cmdline.txt
sed -i -r "s;cryptdevice=[^ ]+;;" ${MOUNT_RPI}/cmdline.txt
sed -i -r "s;$; cryptdevice=${RPI_ROOT_DISK_NAME}:${LUKS_NAME};" ${MOUNT_RPI}/cmdline.txt

umount ${boot_disk}

Answer 1

TL;DR

1. Configure cryptsetup via initramfs option in /etc/crypttab

   Setting the config variable in /etc/cryptsetup-initramfs/conf-hook is obsolete in Debian 11.

   The update-initramfs hooks will look for the option in the crypttab and trigger the appropriate initramfs preparation steps.

2. Configuring additional modules in /etc/initramfs-tools/modules is not required

   The initramfs-update hooks will automatically add all required modules.

3. Ensure the console option in cmdline.txt is tty0

   With only console=tty0 the unlock prompt is sent to the monitor + keyboard terminal.

   The default configuration was console=tty0 console=ttyS1,115200, obviously sending the prompt to the serial console.

Debugging process

When reading the /etc/cryptsetup-initramfs/conf-hook file, the configuration file contained a comment giving a hint on the proper configuration process:

# Matching key files from the crypttab(5) are included in the initramfs image.

Manpages helped configuring the crypttab properly.

However, even with this fixed, the boot process gets stuck at

[   6.812345] device-mapper: uevent: version 1.0.3
[   6.812345] device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialized: [email protected]

The good things is, that cryptsetup was obviously correctly configured because the device mapper module was added to initramfs.

Next troubleshooting steps involved

1. Adding rootdelay=N - only delayed this message
2. Removing rootwait - does not change anything
3. Adding debug does not show more infos
4. Adding break=premount does not drop to a shell
5. Adding break=top does not drop to a shell but it also does no longer load the dm-crypt modules - as expected

This indicated that the init script is working properly. And now it dawned on me: is the console not the monitor + keyboard?

For sure!

I looked at the cmdline.txt and found:

console=tty0 console=ttyS1,115200

Man!

After removing console=ttyS1,115200 initramfs now shows the shell with a break config.

So, afterall the unlock prompt was printed to the serial terminal.

References:

• https://manpages.debian.org/bullseye/cryptsetup/crypttab.5.en.html
• https://manpages.debian.org/bullseye/initramfs-tools-core/initramfs-tools.7.en.html

Further tips

Process optimizations

The process can and should be furhter optimized by not only creating the proper fstab and crypttab in the preparation step but also configuring the Rasperry Pi config update and kernel post-install scripts to regenerate cmdline.txt properly.

This outlines the process:

cat >/etc/default/raspi-extra-cmdline <<-EOF
    rootdelay=${INITRD_ROOTDELAY} cryptdevice=${RPI_ROOT_DISK_NAME}:${LUKS_NAME}
EOF

sed -i -r \
    -e "s;^#?(ROOTPART)=.*$;\1=${LUKS_MAPPER_DEVICE};" \
    -e "s/^#?(CONSOLES)=.*$/\1=${INITRD_CONSOLE}/" \
        /etc/default/raspi-firmware

The option rootdelay was added to give the kernel some time to end logging device detections to not printing into the password prompt.

Debugging

The init script will dump much more detailed runtime information with the debug option in cmdline.txt set to a random string (i.e. debug=foo).