Backup of SD image on Mac could not be written back to same-size SD: too large

Question

I've built a beautifully crafted SD card for my application using a Raspberry Pi Zero. Before shipping it to my customer I've made a backup.

What I did on my Mac BookPro is

1. Open DiskUtility
2. Menu `File/New Image.../Image from "APPLE SD Card Reader Media"
3. Choose "Compress image"

and wait. The original SD card was 16GB.
The resulting DMG file (if mounted) gives /dev/disk3 (disk image): #: TYPE NAME SIZE IDENTIFIER 0: FDisk_partition_scheme +16.1 GB disk3 1: Windows_FAT_32 boot 43.0 MB disk3s1 2: Linux 15.9 GB disk3s2

Then, using Etcher, I usually etch the resulting DMG to a new SD. But this time the resulting size is too big for the SD. Of course the original Card was a 16GB and I'm trying to copy it to another SD. I've tried changing several SD and the result is always the same. I don't know why.

I've done another backup of another SD card (different content) without compressing the image, and it seems to write back correctly.

Any idea on how to recover my data? Thanks Walter

Answer 1

As a rule of thumb, there aren't (m)SD cards of the exact same size. Each individual (m)SD card differs in the amount of storage sectors (blocks) offered, depending on the specific internal controller used as well as the specific memory chips and production yields. Even cards from the same manufacturer of the same model type and from the same batch/lot will show varying block counts, mSD cards of the exactly block count/size are good luck, and thus the exception.

There are basically two ways to deal with this situation:

1. Create a full 1:1 image on a smaller card, say, an 8GiB card. Later flash this image to larger cards, such as 16GiB cards. This is the fastest method if your image has a good ratio of used blocks/total blocks. For rather sparsely filled images, a plain 1:1 flash might not be the fastest method after all. After flashing, you may run a one-time script to expand the system partition to the full size of the mSD cards in order to make good use of the available storage. This is the way done with Raspbian, and other distros.
2. Clone your master card/master image logically onto your mSD card copies, with a cloning tool, such as rpi-clone. This can actually be even faster or on-par with plain 1:1 flashing in case your source image/card is only filled to a lesser extend.

Answer 2

Yes, you can recover your data. You're going to need a Linux system, preferably Debian, with enough free space to hold an image of your card, i.e. about 16GB. You can do this with a Raspberry Pi with a 64GB microSD card as the boot device. If you're building things with the Raspberry Pi, you need such an SD card anyway, just for things like this. Flash Raspbian onto it.

On the big card, you will also need a copy of PiShrink, which you get like this:

wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh
sudo mv pishrink.sh /usr/local/bin

Make an image of the card you want to recover, placing the image on the "big" card. Replace "myimage.img" in the example below with whatever you want to name your image file. Assuming the card is in an adapter that's /dev/sda, do this:

sudo umount /dev/sd*
sudo dd bs=4096 if=/dev/sda of=myimage.img
sudo sync

Now shrink the image with PiShrink.

sudo pishrink.sh myimage.img

Again replacing the image name with whatever name you used when you created the image.

You can now transfer the image to your Mac with VNC, SFTP, or by copying it as data to a USB stick. You will be able to flash that image with Etcher to any card that's nominally bigger than the image, and it'll automagically resize itself when first booted.

I've sometimes had trouble with the "resize" part of PiShrink. If you do, too, you can suppress it by running PiShrink with the -s option, e.g. sudo pishrink.sh -s myimage.img

If you do that, you'll need to expand the file system manually. There's an option in raspi-config to do that. It's the first thing in the "Advanced Options" menu. You might want to run the "Update" option in the first menu before expanding to be sure you have the latest software.