0

My Raspberry Pi 4 SD card is completely unrecoverable. Not sure if it is due to power failure during write or some other reason but it has left the SD card unrecoverable and un-reformattable. Tried dosfsck (on the boot partition), e2fsck (on root and data partitions), fdisk, hdparm (to check and remove the software readonly lock), other tools, and fresh OS image flashes, but nothing works. I am unable to make any changes to the SD card. I can mount the SD card but whatever changes are made it's lost when I unmount and then remount it again.

My configuration:

  • Raspberry PI 4 Model B (4GB RAM)
  • SanDisk microSDHC UHS-I Card 32GB (Class 10, A1)
  • Raspberry Pi OS Lite 32-bit (Debian version 11 bullseye)
  • Three partitions: boot, root (5GB with overlay fs enabled), data (24.5GB normal rw ext4 partition mounted using fstab)

We have a small kiosk like product series where we have been using the Raspberry PI 4 boards and recently after the Debian 11 (bullseye) upgrade and overlayfs for root partition we have been noticing many SD card failures. The system won't boot because the data partition is dirty and the system is unable to clean/fix during the boot (the cmdline file contains fsck.repair=yes). The boot process hangs and goes into a emergency state but since the root account is locked there is nothing much else we are able to do or check.

I tried connecting the corrupted SD card using a card reader as a secondary disk and tried to run all kinds of diagnostics and repair tools but nothing seems to be working. I am not even able to remove the partitions and re-flash the SD card with a new Linux image.

The reason I enabled overlayfs was to ensure and protect the main root partition getting corrupt due to abrupt power recycle (the kiosk devices are directly turned on and off). Now the system is getting stuck due to the data partition. There is some diagnostics data and minimal logs being written to the data partition but nothing which explains the complete corruption and unrecoverable state of the SD cards. The more ironic thing is that many older PI 3 boards with class 4 SD cards that are still working for more than 2-3 years without any issues, event without overlayfs enabled.

I tried to Google many options and tried lots of solutions from the search results but nothing seems to work where I am able to reformat the SD card. I am not as much worried about recovery than the root cause of why the SD cards becoming corrupt.

I have the following questions:

  1. Is it because of the root partition has overlayfs enabled and the data is in normal rw mode?
  2. Could it be due to faulty power supply? The same power supply is working with other PI boards.
  3. The disk block size of the root partition is 4096 and the data partition is 1024. Could this cause some issues while writing or reading data between the two partitions? There are some files from the data partition linked (ln -s) in the root partition. One file is the wpa_supplicant.conf file for WiFi configuration, to make the configuration easier - without having to disable/enable the root overlayfs.

I have even tried to check/remote the software reaonly protection/flag from the SD card using the command hdparm -r0 /dev/sda but nothing changed. Still unable to format or delete the partitions.

vishnu@kiosk:~ $ sudo blkid 
/dev/sda1: LABEL_FATBOOT="boot" LABEL="boot" UUID="AE82-4BC1" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="b1214a26-01"
/dev/sda2: LABEL="rootfs" UUID="6d2ff93e-eacd-415c-96d5-4611ad21e05f" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="b1214a26-02"
/dev/sda3: UUID="be4e4b56-49aa-45b6-88cf-684242ac6ce5" BLOCK_SIZE="1024" TYPE="ext4" PARTUUID="b1214a26-03"
vishnu@LMG527:~ $ sudo fdisk /dev/sda

Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/sda: 29.72 GiB, 31914983424 bytes, 62333952 sectors
Disk model: Storage Device  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb1214a26

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sda1           8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/sda2         532480 11018239 10485760    5G 83 Linux
/dev/sda3       11018240 62333951 51315712 24.5G 83 Linux

Output of the df command from a working SD card:

vishnu@kiosk:~ $ df -ah
Filesystem      Size  Used Avail Use% Mounted on
sysfs              0     0     0    - /sys
proc               0     0     0    - /proc
udev            1.8G     0  1.8G   0% /dev
devpts             0     0     0    - /dev/pts
tmpfs           384M  1.2M  383M   1% /run
overlay         1.9G  179M  1.8G  10% /
securityfs         0     0     0    - /sys/kernel/security
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
cgroup2            0     0     0    - /sys/fs/cgroup
pstore             0     0     0    - /sys/fs/pstore
bpf                0     0     0    - /sys/fs/bpf
systemd-1          0     0     0    - /proc/sys/fs/binfmt_misc
mqueue             0     0     0    - /dev/mqueue
sunrpc             0     0     0    - /run/rpc_pipefs
debugfs            0     0     0    - /sys/kernel/debug
tracefs            0     0     0    - /sys/kernel/tracing
fusectl            0     0     0    - /sys/fs/fuse/connections
configfs           0     0     0    - /sys/kernel/config
/dev/mmcblk0p1  255M   59M  197M  23% /boot
/dev/mmcblk0p3   24G   20M   23G   1% /mnt/data
tmpfs           384M     0  384M   0% /run/user/1001

Booup error

Output of /etc/fstab file

vishnu@kiosk:~ $ cat /etc/fstab
proc            /proc           proc    defaults          0       0      
PARTUUID=b1214a26-01  /boot           vfat    defaults          0       2
PARTUUID=b1214a26-02  /               ext4    defaults,noatime  0       1
PARTUUID=b1214a26-03  /mnt/data       ext4    defaults,noatime  0       2
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
2
  • Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
    – Community Bot
    Jan 9 at 18:46
  • "protect the main root partition getting corrupt due to abrupt power recycle (the kiosk devices are directly turned on and off" -> Partially based on reports going back here for years, I don't believe a read-only file system is a bulletproof solution to this problem and regarding it as such is bound to bite you one day. The SD card reader (and the SD card itself) are really black boxes and I doubt there is anything in the protocol standard that says arbitrarily cylcling power is guaranteed not to cause arbitrary damage even if the operating system is not writing anything to it.
    – goldilocks
    Feb 8 at 14:47

2 Answers 2

0

Most of what you have posted is irrelevant and certainly not Pi specific.

It is unclear what you actually WANT to do.

There seems to be a religious fervour about overlayfs which is totally unrealistic and ignores the way SD Cards actually work - ANY write to the card risks failure.

If you can mount the SD Card copy your data.

If you want to reuse the card (despite possible failure) use the SD Association tool to reformat it (which can often recover failed cards).

1
  • I am just trying to understand why the SD cards are getting corrupt so fast, by fast I mean within 1-2 weeks of usage. This is the first time I am experiencing failure of SD cards so fast in multiple devices. This did not happen before (with PI 3 and without overlayfs) and like I mentioned, there are still many devices functioning properly for more than 2+ years.
    – Ram
    Jan 9 at 10:37
0

If a card is dead after only one week, I would put its authenticity into question. You can fill the customer support form of your manufacturer (for Sandisk that would be https://support-en.wd.com/app/askweb/), and provide:

  • Place and location of purchase including web site and URL
  • Proof of purchase
  • Clear pictures of both sides of the card

It doesn't make sense to spend time trying to understand why a fake card misbehaves.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.