3

It appears that after a kernel update happened, and rebooted for, Raspbian will no longer boot. (RPi4) After entering the (correct) encryption key when prompted, the mount fails with error:

Cannot initialize device-mapper. Is dm_mod kernel module loaded?

If I CTRL+C enough I can drop to a BusyBox shell and attempt to cryptsetup manually, but that has the same problem, even after trying modprobe dm_mod, so I don't think that the device mapper kernel module is being included in the initramfs at all.

I can remove the SD card and mount it without issue in a different Debian PC, and am able to read/write the file system, including installing packages via apt and rebuilding the initramfs. The process I'm using to do this is:

# replace /dev/sdf with your SD card block device
# make sure you do that for the two `sed` calls for "/etc/crypttab" too!

# mount
cryptsetup -v luksOpen /dev/sdf2 sdcard
mount /dev/mapper/sdcard /mnt; mount /dev/sdf1 /mnt/boot; mount -o bind /dev /mnt/dev; mount -o bind /dev/pts /mnt/dev/pts; mount -t sysfs none /mnt/sys; mount -t proc  none /mnt/proc

# comment out ld.so.preload
sed -i 's/^/#/g' /mnt/etc/ld.so.preload
# crypttab needs to point to the current physical device during mkinitramfs or cryptsetup won't deploy
sed -i 's/mmcblk0p/sdf/g' /mnt/etc/crypttab

# copy qemu binary
cp /usr/bin/qemu-arm-static /mnt/usr/bin/

# chroot to Raspbian to update and rebuild initramfs
chroot /mnt /bin/bash
    rm -rf /var/tmp/mkinitramfs*
    apt update && apt upgrade && apt dist-upgrade && apt autoremove
    test -L /sbin/fsck.luks || ln -s /sbin/e2fsck /sbin/fsck.luks
    mkinitramfs -o /boot/initramfs.gz -k $(ls -t /lib/modules | tail -1)
    exit

# undo damage
sed -i 's/^#//g' /mnt/etc/ld.so.preload
sed -i 's/sdf/mmcblk0p/g' /mnt/etc/crypttab

# unmount
sync
umount /mnt/{dev/pts,dev,sys,proc,boot} /mnt
cryptsetup -v luksClose sdcard

These instructions have gone through revisions to try and resolve this. Most notably the "crypttab" replacement from "mmcblk0p" to "sdf", which is only there so when mkinitramfs runs within a chroot the encrypted partition is found. Otherwise the process shows "cryptsetup: ERROR: Couldn't resolve device /dev/mmcblk0p2". (Either value give the same error when really trying to boot.)

The boot command line "/boot/cmdline.txt" refers to the encrypted partition correctly:

console=serial0,115200 console=tty1 root=/dev/mapper/sdcard cryptdevice=/dev/mmcblk0p2:sdcard rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet plymouth.ignore-serial-consoles

Kernel version going on is "4.19.97-v8+" though I've tried "4.19.97+", "4.19.97-v7+", and "4.19.97-v7l+" too.

Other than a reinstall, would you have any advice on building a good initramfs to resolve this encryption issue?

Related questions:

14
  • 1
    You could symlink or mknod /dev/mmcblk0p1 to /dev/sdf1 and so on for the other partitions if you wanted to avoid all the sed stuff. Basically you have to figure out how to get the missing module into the initramfs. Most distros have a config file you can edit to list additional modules that should be included, e.g. this one for Ubuntu which might give you some hints. If you can figure that out, regenerating the initramfs as you've done should be all you need to get the system going again.
    – Malvineous
    Feb 14 '20 at 14:04
  • 1
    "Other than a reinstall, would you have any advice on building a good initramfs to resolve this encryption issue?" -> It's implied here that you've had this working, but normally Raspbian does not use an initrd of any kind. As far as I'm aware it requires an option set on the kernel command-line, but I don't see one here (mebbe there is a hardcoded default to something in the first partition?). Have you had it working before?
    – goldilocks
    Feb 14 '20 at 15:23
  • 1
    @goldilocks initrd is deprecated but pretty sure it's impossible to build a modern kernel without initramfs. initrd was a separate file, initramfs is embedded into the kernel image itself.
    – Malvineous
    Feb 15 '20 at 2:10
  • 1
    @Malvineous WRT "is depecreciated": Yes, but initrd= is still the kernel parameter used to indicate the path of an external initramfs (which is why I wrote "any kind of" -- anyway, they're normally not compiled in, if you have a normal linux box, go look at the grub2 config, it will be using initrd=/path/to/initramfs). "it's impossible to build a modern kernel without initramfs": Enabling and building them is still an option; I'm on an x64 box with a custom 5.4 kernel built by me, no initramfs. The major reason for using an initrd/ramfs is to facilitate loading of drivers...
    – goldilocks
    Feb 15 '20 at 14:35
  • 2
    @goldilocks: Sorry you are right, it seems I only got as far in the docs as all kernels must have initramfs (even if it's just empty), but I didn't get to the part where it said you can pass an external one in using the initrd parameter and it will overwrite whatever is included in the built in (and possibly empty) initramfs. Of course now that I think about it it seems obvious, having to patch a precompiled kernel with a user-specific initramfs would be a huge headache!
    – Malvineous
    Feb 16 '20 at 9:17
1

Another solution for this problem (besides the comments under the question) which helped in my case on a raspberry 3b+ running on the raspbian OS:

sudo apt-get install --reinstall --yes raspberrypi-bootloader raspberrypi-kernel
sudo reboot

Taken from this thread: https://www.raspberrypi.org/forums/viewtopic.php?t=282802

my cryptsetup command worked afterwards, while having the same error like OP stated: Cannot initialize device-mapper. Is dm_mod kernel module loaded? previously.

0

This happened again today. Fortunately I had kept the script from previously, with the small addition of @ElectRocnic's apt install --reinstall --yes raspberrypi-bootloader raspberrypi-kernel.

VERY IMPORTANT: It mattered to ensure that the correct kernel module was given to mkinitramfs, because apparently this doesn't automatically happen during regular updates. Always look in "/lib/modules" yourself as $(ls -t /lib/modules | tail -1) doesn't always give you the right one. Not doing this will give you an error about "dm_mod" module not being loaded when entering your volume passphrase.

Full script is as follows:

# Replace /dev/sdg with your SD card block device.
# Use `lsblk` to help you find where this is.

# Link "mmcblk" block device so chroot later sees the expected device block names.
test -L /dev/mmcblk0p1 || ln -s /dev/sdg1 /dev/mmcblk0p1
test -L /dev/mmcblk0p2 || ln -s /dev/sdg2 /dev/mmcblk0p2

# Mount SD card.
cryptsetup -v luksOpen /dev/mmcblk0p2 sdcard
mount /dev/mapper/sdcard /mnt; mount /dev/mmcblk0p1 /mnt/boot; mount -o bind /dev /mnt/dev; mount -o bind /dev/pts /mnt/dev/pts; mount -t sysfs none /mnt/sys; mount -t proc  none /mnt/proc

# Comment out ld.so.preload.
sed -i 's/^/#/g' /mnt/etc/ld.so.preload

# Copy qemu binary.
cp /usr/bin/qemu-arm-static /mnt/usr/bin/

# chroot to Raspbian to update and rebuild initramfs.
chroot /mnt /bin/bash
    rm -rf /var/tmp/mkinitramfs*
    apt update && apt upgrade && apt dist-upgrade && apt autoremove
    apt install --reinstall raspberrypi-bootloader raspberrypi-kernel
    test -L /sbin/fsck.luks || ln -s /sbin/e2fsck /sbin/fsck.luks
    update-initramfs -u
    # IMPORTANT: Replace "5.10.52-v7l+" with the correct kernel from "/lib/modules".
    # Doing `$(ls -t /lib/modules | tail -1)` here doesn't always give you the right one!!!
    mkinitramfs -o /boot/initramfs.gz 5.10.52-v7l+
    exit

# Undo damage to our local recovery system.
sed -i 's/^#//g' /mnt/etc/ld.so.preload

# Force flushing write buffers and unmount SD card.
sync
umount /mnt/{dev/pts,dev,sys,proc,boot} /mnt
cryptsetup -v luksClose sdcard

# Unlink "mmcblk" block device.
test ! -L /dev/mmcblk0p1 || rm /dev/mmcblk0p1
test ! -L /dev/mmcblk0p2 || rm /dev/mmcblk0p2

# All done. Eject the SD card and attempt boot in your Pi.

Did this from another Linux system, worked as normal.

N.B. The step to forcibly reinstall "raspberrypi-bootloader" and "raspberrypi-kernel" takes sodding ages. :) Didn't time it exactly, but it was about half an hour.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.