Piping dd with pishrink and ssh to remotely create a shrunk backup image of a raspberry pi

Question

I usually create remote backup image of my raspberrypi from my macbook by using:

ssh [email protected] "sudo dd if=/dev/mmcblk0 bs=1M” | pv | gzip - | dd of=/Users/USER/Downloads/pibackup.gz

There are drawbacks but It works quite fine and i'm ok with it. Recently i found pishrink and i would like to include it in the aforementioned pipe maybe removing gzip since pishrink can do it itself. So first think i installed pyshrink on my raspberry and then i tried to run something like

ssh [email protected] "sudo dd if=/dev/mmcblk0 bs=1M status=progress | pishrink.sh -az -"| dd of=/Users/USER/Downloads/pibackup.img.gz

But it doesn't work. Does anyone got some suggestion?

Answer 1

But it doesn't work

While some CLI apps will accept input from either a filepath or standard input, this is something that's implemented in the application, not the shell or the pipe mechanism. So if such is not implemented in the command, then trying to do this won't work.

Generally speaking it is behaviour which will be documented if it exists, and if not, it probably doesn't exist.

In this case, there are some logical reasons for why such behaviour is not possible -- or at least, not likely to be useful, because it would require you run it on a machine that had more RAM available than the size of the image you want to shrink. This is because the process must be applied to an entire filesystem (the second partition in the image), not a streaming window on an image of such. Hence, the tool would have to read the entire image into memory before it can do the shrink.

If you thought it was just skipping unused space or compressing it inline, that's not the case.

Answer 2

Now that you know why pishrink doesn't work, you may be interested in a solution that does work: image-utils. I've found its compression (via image-shrink) substantially reduces the image size.

You can download image-utils from the forum where it was posted; simply download the compressed package of scripts & place them wherever you think convenient. Alternatively, you may install from GitHub:

$ git clone https://github.com/seamusdemora/RonR-RPi-image-utils.git

Changes are infrequent, but easily incorporated:

git -C ~/RonR-RPi-image-utils pull https://github.com/seamusdemora/RonR-RPi-image-utils.git

---

NOTE: I am not the author of image-utils, nor do I make any changes to the source. I've put it on the GitHub site for my own convenience, and happy to share that if you prefer using git to keep your local sources updated (rather than dealing with the .zip file). I maintain sync with the master copy on the Raspberry Pi org's forum with a small script I've written to automate the process.