102

This question answers the question of how I use an external computer to create a backup of my RPi.

I'm wondering whether I can create a backup image of the SD card that is currently in use, and copying it to a file on a USB storage device. Is this possible? If not, is there any way to create a backup of a RPi without involving another computer?

6
  • 2
    Sure, but skip /tmp, /run, /proc, /sys, /dev, and /mnt. You don't need to create an image, you need a backup you can create or update an image from. So don't use dd, look into rsync.
    – goldilocks
    Commented Mar 14, 2013 at 2:13
  • 2
    @goldilocks I would love it if you would flesh this comment out into a more complete answer, explaining the backup & restore process you have in mind. Commented Mar 14, 2013 at 9:31
  • Done -- sorry it took me a few days to find the time.
    – goldilocks
    Commented Mar 17, 2013 at 15:33
  • 1
    If your destination volume is big enough, remounting the filesystem read-only and doing a dd copy with an appropriate block size will probably be fastest for a "new" copy. Doing a file-by-file copy to flash/SD media is probably a bad idea. Commented Dec 11, 2015 at 1:41
  • Here a tutorial that does exactly what you want to do. danran.rocks/2022/09/…
    – DanRan
    Commented Aug 16, 2023 at 23:31

9 Answers 9

107

Here's an intro to using rsync for back-up on the Pi. Once the initial back-up is created, keeping it up to date this way is much much faster than constantly ripping the entire image. You can do this to a local hard drive or over a network.

You actually do not want a complete copy of a running system as a back-up, since some of the stuff ostensibly in the filesystem exists only at runtime. Including that in a backup and then using it to recreate an image later may create problems for you.

There are some other exceptions too. rsync can accept a list of (glob) patterns to exclude, and those can be read from a file, so let's first go thru what should be in such a file. Note that the entries are of the form /directory/* and not /directory. This is because we want them to exist, but we don't want to copy anything in them.

/proc/*
/sys/*

These do not really exist on disk. They're an interface to the kernel, which creates and maintains them in memory. If you copy these and then copy them back into a system and boot it, it will be (at best) meaningless, since the kernel uses those as mount points for the interfaces [If you want to see what happens when you mount a filesystem partition on a directory with data in it, try. It works and won't do any harm, but the stuff that was in the directory is now inaccessible.]

Note that it is critical that the /sys and /proc mount points exist. But they should not contain anything. Next:

/dev/*

The dev directory is not quite the same thing as proc and sys but for our purposes it is. If you believe that you should save this so you can have the same device nodes in your backup or something, you're wrong. Don't bother. Do not copy dev. Once upon a long time ago Linux did work that way, but it doesn't anymore.

/boot/*

This is sort of a special case with the most (perhaps all) of the Pi specific distros such as Raspbian. It's actually a mount point for the first, vfat, partition. We are going to deal with that separately. Whatever you do, don't bother including it here, because again, it's a mount point.

/tmp/*
/run/*

/run is generally not on disk either, it's in memory. Perhaps /tmp could be too (this would save a bit of SD card action), but in any case, as the names imply, these are not places for storing persistent data. Applications which use them expect that they may be deleted at each boot.

/mnt/*
/media/*

These are important particularly if you are planning to back up to a hard drive or USB stick and the device is in /mnt or /media (automounting tends to use the latter), because if you don't exclude the location of those devices in the filesystem you will create a loop backing up the content of the drive to itself, until it runs out of space. I think rsync might be smart enough to spot something that dumb but try to avoid testing the premise.

On to the actual backing up: Create a directory to back up to on the locally mounted harddrive, USB thing, etc. -- e.g. "pi_backup". You can alternately backup to a remote location via ssh (see below) or using a network mounted filesystem, but this will probably take a while the first time.

If the file containing the list to exclude is /rsync-exclude.txt1 and your drive is /mnt/usbhd, to do the actual backup:

rsync -aHlAXNv --delete --exclude-from=/rsync-exclude.txt / /mnt/usbhd/pi_backup/

Notice that there is a trailing slash on pi_backup/. You may want to have a look at what all the switches mean in man rsync.

This will take a while and produce a lot of output (if you want to examine that in a log instead, append > rsync.log). --delete is meaningless the first time, but for keeping the backup updated use it. This ensures that stuff you've later deleted on the Pi also gets removed from your backup. The a sets recursion into directories and makes sure all the file attributes match. -H is to preserve hard links2, v is for verbose which is why you get some output (otherwise rsync is quiet). See man rsync for more.

There is a shortcut whereby you can skip the --exclude-from file. If you are sure that all of the things you don't want to copy (/tmp etc.) are on separate filesystems, you can just use:

rsync -axHlAXNv --delete-during / /mnt/usbhd/pi_backup/

-x has been inserted. This is the short form of --one-file-system, which tells rsync not to cross filesystem boundaries. Personally I prefer the --exclude-from, but on e.g., default Raspbian, --one-file-system will work fine. You can use both if you want to be -xtra careful :D

That's not quite a complete backup. It's enough if you haven't put anything in boot and you are fine with using the back up to just restore the system by sticking the card in a computer and running:

rsync -av --delete-during /mnt/usbhd/pi_backup/ /mnt/sdcard_partition2/

You could also do this with a card with a new image on it (presuming it's the same as your base image) although that's a little inefficient if you have to create the image (because you're then going to overwrite most of it). You could also connect another SD card via a USB adapter with such an image on it, and use the above method to maintain a duplicate card.

If you've put stuff in /boot (e.g., a custom kernel), including /boot/config.txt, you'll want to back that up too (pretty simple -- there's not much to it). Just do it separately, and when you restore, that stuff goes in the first partition.

See here if you want to create a blank Raspbian style image which you could then backup into. You can use a similar methodology to create an empty Raspbian style card -- just rather than dealing with an .img file, you'd be dealing with a real device (e.g. /dev/sdb), meaning all you have to do is create the partition table with fdisk and then format /dev/sdb1 and sdb2 (or whatever) with mkfs.

But copying the whole image is easier! Why bother with this?

It's not that hard; I restored to a blank card (formatted as per that last link) in 10 minutes. Yes, just using dd on the whole thing is simpler (if you find stuff like words confusing...), BUT then it takes quite a while every time you want to update your backup because you must do 100% of it every time. Using rsync, once a backup exists, updating it is much much faster, so you can set this up to happen painlessly everyday via cron. Over a network even. Every six hours. The more often you do it, the less time it will take.

rsync via ssh

Here's an example:

rsync [options] --rsh="ssh [ssh options]" root@[the pi ip]:/ /backup/rpi/

"Options" would be, eg, -av --delete --exclude-from=/rsync-exclude.txt and "ssh options" is whatever you normally use (if anything). You must have root access via ssh to do this for the purposes of a system backup (set PermitRootLogin=yes in /etc/ssh/sshd_config and restart the server).


1 You should keep this file. You can put comments in it on lines beginning with # or ;. This could include the actual rsync command, which can be copy pasted later so you don't have to remember it each time.

2 Thanks to Kris for pointing out rsync doesn't do this automatically.

20
  • 1
    Goldilocks. That looks like a great use of rysync. Any chance of rolling it into a script for us? Commented Aug 27, 2013 at 7:47
  • Instead of manually excluding all mountpoints, why not mkdir /tmp/backupable && mount --bind / /tmp/backupable and rsync that? That also has the advantage of backing up any data stored in places that are "shadowed" by something mounted there.
    – n.st
    Commented Dec 18, 2014 at 5:11
  • 1
    Great answer. I've used it setup my rpi automated backup and it works great. I noticed though that when I restored my 'clone' it actually took up more space on the card than the original. So I wonder if it maybe a good idea to include the -H option in the rsynch command. (rsynch manual says -a doesn't imply -H). The -H options tries to preserve 'hardlinks'.
    – Kris
    Commented Feb 1, 2015 at 20:17
  • 2
    @IgorGanapolsky The intention is to not create an image (read the "But copying the whole image is easier! Why bother with this?" part). In addition to being easier and faster to maintain once created, this method is generally more flexible. If you want to use it later to create an .img you can; this and this should help to explain how they are structured and can be created.
    – goldilocks
    Commented Jan 25, 2017 at 15:23
  • 1
    See the paragraph that begins, "That's not quite a complete backup...". It's basically the exact same thing in reverse. This may help with some concepts that people are commonly confused by/about.
    – goldilocks
    Commented Jan 25, 2017 at 15:38
26

A working script from the Raspberry Community made by a member there.

You can reuse and tweak the code how ever you like.It is well documented and self explanatory.

#!/bin/bash

# Setting up directories
SUBDIR=raspberrypi_backups
DIR=/hdd/$SUBDIR

echo "Starting RaspberryPI backup process!"

# First check if pv package is installed, if not, install it first
PACKAGESTATUS=`dpkg -s pv | grep Status`;

if [[ $PACKAGESTATUS == S* ]]
   then
      echo "Package 'pv' is installed."
   else
      echo "Package 'pv' is NOT installed."
      echo "Installing package 'pv'. Please wait..."
      apt-get -y install pv
fi

# Check if backup directory exists
if [ ! -d "$DIR" ];
   then
      echo "Backup directory $DIR doesn't exist, creating it now!"
      mkdir $DIR
fi

# Create a filename with datestamp for our current backup (without .img suffix)
OFILE="$DIR/backup_$(date +%Y%m%d_%H%M%S)"

# Create final filename, with suffix
OFILEFINAL=$OFILE.img

# First sync disks
sync; sync

# Shut down some services before starting backup process
echo "Stopping some services before backup."
service apache2 stop
service mysql stop
service cron stop

# Begin the backup process, should take about 1 hour from 8Gb SD card to HDD
echo "Backing up SD card to USB HDD."
echo "This will take some time depending on your SD card size and read performance. Please wait..."
SDSIZE=`blockdev --getsize64 /dev/mmcblk0`;
pv -tpreb /dev/mmcblk0 -s $SDSIZE | dd of=$OFILE bs=1M conv=sync,noerror iflag=fullblock

# Wait for DD to finish and catch result
RESULT=$?

# Start services again that where shutdown before backup process
echo "Start the stopped services again."
service apache2 start
service mysql start
service cron start

# If command has completed successfully, delete previous backups and exit
if [ $RESULT = 0 ];
   then
      echo "Successful backup, previous backup files will be deleted."
      rm -f $DIR/backup_*.tar.gz
      mv $OFILE $OFILEFINAL
      echo "Backup is being tarred. Please wait..."
      tar zcf $OFILEFINAL.tar.gz $OFILEFINAL
      rm -rf $OFILEFINAL
      echo "RaspberryPI backup process completed! FILE: $OFILEFINAL.tar.gz"
      exit 0
# Else remove attempted backup file
   else
      echo "Backup failed! Previous backup files untouched."
      echo "Please check there is sufficient space on the HDD."
      rm -f $OFILE
      echo "RaspberryPI backup process failed!"
      exit 1
fi

Consider adding comments to the original forum or post your own version to help mature the content. Take a little give a little.

*And thank you for giving back AndersW (Click for GIT script)

8
  • 2
    What if the filesystem (file deletion, new files added) changes in the time while the pi is backuping?
    – keiki
    Commented Mar 14, 2013 at 13:47
  • 2
    I back up several disks while they are running with rsync, and I have frequently been able to get exactly what I need from these file backups. However, in general, a unix file system can not be copied perfectly (with every bit in place and correct) while the file system is mounted (*). A copy made while the system was mounted is sometimes called a "dirty copy". Several measures can be taken to improve the quality of a dirty copy (as the script above does, shutting off cron and mysql) but it can't be perfect. Cheers! * -- I am wrong about this, it depends on the file system. Commented Mar 14, 2013 at 14:28
  • 1
    You can look at the Debian recommended backup utilities and see if Pi has a port of them. rsnapshot sounds promoising
    – Piotr Kula
    Commented Mar 14, 2013 at 14:29
  • 1
    @TaiViinikka You don't need a perfect copy. You need a partial copy that can be (quickly and easily) re-imposed on the original base image. rsync is the way to go; when I have time tomorrow I'll add an answer. rsnapshot is also worth investigating.
    – goldilocks
    Commented Mar 14, 2013 at 18:12
  • 3
    Based on ppumkins answer above, I synced up the 'dd' script with the latest comments in the original thread and added some minor improvements myself. The end result is available here: <github.com/aweijnitz/pi_backup>. Please don't hesitate to add improvements and sending me pull requests.
    – AndersW
    Commented Sep 4, 2013 at 21:36
14

I have adapted @goldilocks answer on rsync for back-up on the pi. I backup to an ext4 partition on a HDD mounted on the Pi. If the HDD is not mounted, rsync would copy to the mount directory (until the SD Card is full). If the HDD is not mounted in rw mode copious error messages are produced. Neither of these is desirable, so I check that my partition is is mounted in rw mode before proceeding.

NOTE 2015-03-03 I modified my answer to accurately copy hardlinks. The original worked, but converted many hardlinks into files. In addition to wasting space, this compromises many uses which assume the hardlinks are in place. (My current image has 869 links, many in Raspbian itself.)

My script to do this follows. (My partition is PiData, mounted on /mnt/PiData

#!/bin/bash
# script to synchronise Pi files to backup
BACKUP_MOUNTED=$(mount | awk '/PiData/ {print $6}' | grep "rw")
if [ $BACKUP_MOUNTED ]; then
    echo $BACKUP_MOUNTED
    echo "Commencing Backup"
    rsync -avH --delete-during --delete-excluded --exclude-from=/usr/bin/rsync-exclude.txt / /mnt/PiData/PiBackup/
else
    echo "Backup drive not available or not writable"
fi

Restore (or update another Pi) with the following:-

sudo rsync -avH /mnt/PiData/PiBackup/ /

I have enhanced the rsync-exclude.txt to eliminate unnecessary files.

The first group are the directories documented by @goldilocks https://raspberrypi.stackexchange.com/users/5538/

The second group are the files and directories created by OS X when I access my Pi using AFP (Apple Filing Protocol). (These are normally invisible on OS X, but not on Raspbian. In any event, there is no need to backup.) Even if you never use AFP, these will do no harm.

The third group are files which do not need to be backed up (and certainly not copied to another Pi). Examples fake-hwclock.data, RPi-Monitor reports. You will probably have others.

/proc/*
/sys/*
/dev/*
/boot/*
/tmp/*
/run/*
/mnt/*

.Trashes
._.Trashes
.fseventsd
.Spotlight-V100
.DS_Store
.AppleDesktop
.AppleDB
Network Trash Folder
Temporary Items

.bash_history
/etc/fake-hwclock.data
/var/lib/rpimonitor/stat/
3
  • 1
    Is there a way to make that output an .img file? Commented Jan 25, 2017 at 15:24
  • @IgorGanapolsky Well, seeing as all of the essential files are there (excepting boot files), it is obviously possible, but if you want an image make an image. You should ask any new question in a new post, not comments.
    – Milliways
    Commented Jan 26, 2017 at 3:42
  • @Milliways why not should we use "sudo rsync ..." ? There will be some files that may can't be synchonized ?
    – Smilia
    Commented Nov 4, 2018 at 9:45
9

It's now 2022; the original question here, and some of the answers, are about 9 years old now. I'm adding this answer to provide an update, and to connect this question, and its answers, with a more recent question asked (and answered) here. I've also included a section on Restoration, which is of course the end objective; the Backup is only the means to that end.

Backup:

The image-backup script provides an answer to both questions:

  • RPi can make a backup of itself while running,
  • the backup is contained in an image file (.img) for quick and easy restoration,
  • the image file does not contain unused space, and typically restores to a smaller SD card,
  • the image file is created quickly, (ref. details)
  • the script is extremely easy-to-use , prompting for all required options
  • the image file contains the entire SD card, incl. the /boot partition
  • the backup image file may be updated to maintain a current snapshot of your system
  • the script uses rsync, and therefore avoids most potential issues associated with dd in creating image backups
  • all of the rsync features may be applied to the backup: links, include &/or exclude files & directories, etc.

The image-backup script is part of a set of scripts called image-utils that was posted under the heading of Image File Utilities created by "RonR". You will find a link to download a .zip file there containing all the scripts. The author (RonR) provides technical support through his forum postings.

For those who prefer git, I've forked a GitHub repo of image-backup. I'm making an effort to keep it current with a small script I've written. If you'd like to use the git repo to download direct to your RPi, here's a recipe for that:

  1. If you haven't installed git, you'll need to do so from your RPi CLI:

    $ sudo apt install git
    
  2. Clone the GitHub repo of image-utils:

    $ cd
    $ git clone https://github.com/seamusdemora/RonR-RPi-image-utils
    
  3. Make the scripts executable:

    You now have a folder which is also a local git repo: ~/RonR-RPi-image-utils. And you have a decision to make: preserve your local git repo, or use it as your "working" folder.

    The simplest thing to do is use this folder as a working folder; only slightly more complicated is preserving your local repo & creating a working folder. Here's how I preserve my local repo; you can simply apply the chmod against your repo folder to use it as your working folder:

    $ mkdir ~/imgbkup && cp ~/RonR-RPi-image-utils/image-* ~/imgbkup/
    $ chmod 755 ~/imgbkup/image-*
    

    As an alternative, you could install the script files in /usr/local/sbin & make then executable in a single step:

    $ sudo install --mode=755 --group=pi --owner=pi /home/pi/RonR-RPi-image-utils/image-* /usr/local/sbin/
    # see 'man install' for details
    

    This (usr/local/sbin location) has the advantage of being on your (user pi) PATH; refs. Also note that (e.g.) image-backup must still be run w/ sudo - this is a requirement imposed in the script - not by permissions or ownership.

Restoration:

We create and maintain backups only because they enable us to restore our system to a prior state. Perhaps that prior state is one that existed before our SD card failed, or more frequently, a state that existed before an operator error occurred which accidentally deleted the ~/.ssh folder in our home directory, or /etc, /boot/config.txt, etc, etc. Mistakes do happen, and maintaining a current backup is the key to quickly restoring our system to its pre-disaster machine state.

In choosing a backup solution, one must consider not only the accuracy and efficiency of the backup process, but equally important is the question: What is the restoration process?

The following attempts to answer that question under two scenarios:

Complete system restoration:

This is the simplest scenario as it involves only writing the complete .img backup file to a new SD card, removing the old SD card, inserting the new one in its place and re-booting. The entire restoration process may require only a few minutes. If the backup .img file is up-to-date there should be little or no recovery time required to update configuration files, re-install packages, etc. The key is having a current image.

Partial restoration:

But perhaps you've made an error performing some administrative task; for example, I recently deleted my ~/.ssh folder... I can't replace that! Fortunately, this was also an easy restortion - though involving a bit more work than a full restore:

You can mount an .img file as a "loop device", effectively treating it as another block device - just as you do with your SD, HDD or USB. Only the mounting syntax is different. Without belaboring the point, here's how that might go:

  1. In this case, I was able to restore my RPi while still connected via SSH. The balance of this example is based on that scenario.

  2. However, the same basic procedure applies if you've lost access to your RPi; the difference being that you'll need another Linux machine - a restoration system. The restoration system can be any system that can mount ext4 & FAT filesystems; e.g. another RPi, Ubuntu laptop, etc. The restoration system must have access to the SD card of the "broken" system, and the *.img file created by image-backup.

  3. From the damaged RPi: Mount the *.img file (20220530_Pi4B_imagebackup.img) as follows:

$ sudo losetup -Pf 20220530_Pi4B_imagebackup.img
  1. Find the loop device(s) used:
$ ls -l /dev | grep loop
brw-rw---- 1 root disk      7,   0 Jun 18 04:32 loop0
brw-rw---- 1 root disk    259,   0 Jun 18 04:32 loop0p1
brw-rw---- 1 root disk    259,   1 Jun 18 04:32 loop0p2
... 
  1. Create mount point(s) for the partitions you need (i.e. /, /boot, or both) & mount the loop device/backup image:
$ sudo mkdir /mnt/part1 /mnt/part2     # create mount point
$ sudo mount /dev/loop0p2 /mnt/part2   # mount partition 2
$ sudo mount /dev/loop0p1 /mnt/part1   # mount partition 1
  1. At this point you can browse through the backup image file (20220530_Pi4B_imagebackup.img) partitions, locate the ~/.ssh folder, and verify the contents if you wish.
$ cd /mnt/part2/home/pi
$ ls -lR .ssh
  1. And finally - Perform the restoration:, and you are back in business:
$ cp -ipr .ssh /home/pi/.ssh

Partial restoration using a restoration system

  1. If you are unable to connect to your damaged RPi, you will need a restoration system to perform the restoration. As explained above, the procedure is similar. The only real difference is that in addition to mounting the backup *.img file, you will also need to mount the SD card from the damaged RPi. You will restore from the loop-mounted *.img file to the SD card.

  2. The SD card from the damaged RPi may be mounted via any available USB port using a USB-to-SD card adapter similar to the one shown below. Once the damaged/missing files & folders are copied/rsync'd from the loop-mounted backup to the SD card, simply re-insert the SD card in the RPi and boot.

SD-USB adapter

5

I have three Pis running in my local net and need to backup them on a regular base with cron when they are up and running. That's why I created a script which is able to create dd, tar and rsync backups and to restore them. I prefer to use rsync for my backups but other folks prefer dd or tar. It's used by a lot of people already. Hope it's useful for others also :-) raspibackup - Raspberry creates backups of itself

10
  • 1
    Nope, sorry: asking user to run (as root!) a script downloaded over HTTP is irresponsible. Please distribute this script over a safe channel.
    – Clément
    Commented May 1, 2016 at 20:38
  • 1
    I don't think it's off topic, and root or not doesn't matter much. The point is that software should be distributed over a secure channel, and your answer is encouraging bad security practices.
    – Clément
    Commented May 7, 2016 at 21:44
  • 1
    That would be a great step forward, yes :)
    – Clément
    Commented May 9, 2016 at 3:06
  • 2
    Just to note that delivery over HTTPS does not in any way add any security in this instance! You are still downloading and running a script from the Internet. The secure process is to download the script (http/https is irrelevant), open the script in an editor and read it top to bottom, check it for oddities and insecurities. Only when you are satisfied should you run it. Framp might be a hacker for all any of us know and delivery over https would only make him smile in that case :) (BTW, that isn't an accusation Framp!) Commented Jun 15, 2016 at 15:02
  • 2
    I agree with you. That's why there are two ways described how to install the script: 1. Use the installerScript 2. Download it manually, check the code and then install it manually
    – framp
    Commented Jun 16, 2016 at 17:40
3

Here is our stable tool for such purposes: https://github.com/aktos-io/aktos-dcs-tools

This tool is written to make ssh connections, make backup-root, make mount-root from remote places in mind at first, and then local sessions are added. So it supports local backups, direct remote backups, proxy remote backups. Backups are taken incrementally (only diffs are transferred) and backup directories are standalone (just pick a directory/version to restore, any directory has a full backup). Of course, you have versions (backup.last-0 is the newest one). You can interrupt backup process at anytime and continue later.

Here are the instructions for your specific problem:

 ssh to-your-raspberry
 cd /mnt/usb0/my-rpi-backups
 git clone https://github.com/ceremcem/aktos-dcs-tools backup-tools
 ln -s backup-tools/Makefile .

 ./backup-tools/configure # you do not need to make any settings for local sessions, just save the default 

 # just for the first time
 make set-local-session  # a flag file is created
 make init               # snapshots directory is created

 # anytime you want to back up
 make backup-root        # backup with rsync

EDIT

Now there is a new target added: You can create a physical SD Card from your backups with one command:

make create-disk-from-last-backup

Follow the instructions, create your SD Card, boot RaspberryPi with this newly created SD Card.

2

Here is a complete different approach. You can use LVM (Logical Volume Manager) to make consistent backups. Beside other improvements like easy adding, expanding and reducing storage or restore the operating system to an earlier status from a snapshot you can also make backups. You don't have do worry about dynamic changed files during backup, setting file systems read only, exclude specific directories or something else. With LVM you simply create a snapshot, mount this snapshot and backup it with the method you prefer. You can make a copy with cp -a, make a mirror with rsync, make an archive with tar or make an image with dd. Assuming you have mounted a backup device on /mnt/usbhd/pi_backup/ you can do for example:

rpi ~$ sudo lvcreate --snapshot --name rpi_snap --size 1G rpi_vg/root_lv
rpi ~$ sudo mkdir /mnt/snapshot
rpi ~$ sudo mount /dev/mapper/rpi_vg-rpi_snap /mnt/snapshot

# make backups
rpi ~$ sudo cp -a /mnt/snapshot/ /mnt/usbhd/pi_backup/
rpi ~$ sudo rsync -aH --delete /mnt/snapshot/ /mnt/usbhd/pi_backup/
rpi ~$ sudo tar -czf /mnt/usbhd/pi_backup/backup.tar.gz -V "Backup of my Raspberry Pi" -C /mnt/snapshot/ ./
rpi ~$ sudo dd if=/mnt/snapshot/ of=/mnt/usbhd/pi_backup/backup.img bs=4M

rpi ~$ sudo umount /mnt/snapshot/
rpi ~$ sudo lvremove rpi_vg/rpi_snap

It takes only one time a little effort to setup LVM. How to do that you can look at Easy backups and snapshots of a running system with LVM.

3
  • How would you compare the LVM approach to rpi-clone, a Raspberry Pi backup script on github?
    – Seamus
    Commented Jan 27, 2020 at 19:50
  • 2
    @Seamus LVM isn't a script doing things like any other solution I read here and also rpi-clone. It is part of the operating system to manage storage devices, even more it's part of the kernel, It must only invoked. Beside easy taking backups of running systems you can do many other nice things with your storage. E.g. you can boot into different snapshots with different stages of software installation. You can just revert to a snapshot and boot. It runs immediately, no need to fiddle with backup files and with long restore time. I use it for testing; always having a fresh flashed install.
    – Ingo
    Commented Jan 27, 2020 at 21:48
  • 1
    @Seamus Beside this you can do many other nice things with managing your storage, e.g. improve or shrink space, add additional physical volumes (disks, SSD, etc.) and so on.
    – Ingo
    Commented Jan 27, 2020 at 21:51
0

I have found a backup tool which makes installable images.

It also has utilities to mount and shrink images.

This may be useful to others

The documentation that comes with it is very brief so I note the following:-

  1. Extract the utilities into any directory and make scripts executable.
  2. Mount an ext4 formatted partition on your Pi in /mnt or /media (any format which allows large files and is supported by Pi e.g. exFAT or a network drive can be used).
  3. For the initial run you will be prompted for a Backup Image name e.g. /mnt/Image/BusterBackup.img
  4. You will be prompted for a Image ROOT filesystem size (in MB), this can be 0 for smallest possible or blank for full backup.
  5. On subsequent runs enter the path of the Backup Image to incrementally update.
An example of the commands I used:-
# Mount USB
sudo mount /dev/sda1 /mnt/Image/
# Update backup
sudo image-utils/image-backup /mnt/Image/BusterBackup.img
# Mount backup
sudo image-utils/image-mount /mnt/Image/BusterBackup.img  MountedImages
When done, run:
sudo umount MountedImages; sudo losetup -d /dev/loop0
# Compress backup
sudo sh -c "gzip -9c /mnt/Image/BusterBackup.img  > Images/BusterBackup.img.gz"

I have slightly modified the original (to copy mountpoints), to correctly calculate partition offsets and sizes and added a couple of comments.

#!/bin/bash
# Original https://raspberrypi.org/forums/viewtopic.php?p=1528736
# 2019-09-26    Modified to set size of boot sector

trap '{ stty sane; echo ""; errexit "Aborted"; }' SIGINT SIGTERM

ADDBLK=0

# Set BOOT_SIZE_MB to the Desired boot sector size (in MB) - should be multiple of 4MB
BOOT_SIZE_MB=256
BOOTSIZEM=$BOOT_SIZE_MB'M'

BOOTBEG=8192
BOOT_SIZE="$((BOOT_SIZE_MB * 1024 * 1024))"
ROUND_SIZE="$((4 * 1024 * 1024))"
# Ensure root sector starts on an Erase Block Boundary (4MB)
ROOTBEG=$(((BOOT_SIZE + ROUND_SIZE -1) / ROUND_SIZE * ROUND_SIZE / 512 + BOOTBEG))

MNTPATH="/tmp/img-backup-mnt"

ONEMB=$((1024 * 1024))

# create BOOT loop device
mkloop1()
{
  local INFO1=""
  local SIZE1=0
  local START1=0

  sync
  INFO1="$(sfdisk -d "${IMGFILE}")"
  START1=$(grep type=c <<< "${INFO1}" | sed -n 's|^.*start=\s\+\([0-9]\+\).*$|\1|p')
  SIZE1=$(grep type=c <<< "${INFO1}" | sed -n 's|^.*size=\s\+\([0-9]\+\).*$|\1|p')
  LOOP1="$(losetup -f --show -o $((${START1} * 512)) --sizelimit $((${SIZE1} * 512)) "${IMGFILE}")"
  if [ $? -ne 0 ]; then
    errexit "Unable to create BOOT loop device"
  fi
}

rmloop1()
{
  if [ "${LOOP1}" != "" ]; then
    sync
    losetup -d "${LOOP1}"
    LOOP1=""
 fi
}

# create ROOT loop device
mkloop2()
{
  local INFO2=""
  local SIZE2=0
  local START2=0

  sync
  INFO2="$(sfdisk -d "${IMGFILE}")"
  START2=$(grep type=83 <<< "${INFO2}" | sed -n 's|^.*start=\s\+\([0-9]\+\).*$|\1|p')
  SIZE2=$(grep type=83 <<< "${INFO2}" | sed -n 's|^.*size=\s\+\([0-9]\+\).*$|\1|p')
  LOOP2="$(losetup -f --show -o $((${START2} * 512)) --sizelimit $((${SIZE2} * 512)) "${IMGFILE}")"
  if [ $? -ne 0 ]; then
    errexit "Unable to create ROOT loop device"
  fi
}

rmloop2()
{
  if [ "${LOOP2}" != "" ]; then
    sync
    losetup -d "${LOOP2}"
    LOOP2=""
  fi
}

# Mount Image partitions
mntimg()
{
  MNTED=TRUE
  if [ ! -d "${MNTPATH}/" ]; then
    mkdir "${MNTPATH}/"
    if [ $? -ne 0 ]; then
      errexit "Unable to make ROOT partition mount point"
    fi
  fi
  mkloop2
  mount "${LOOP2}" "${MNTPATH}/"
  if [ $? -ne 0 ]; then
    errexit "Unable to mount image ROOT partition"
  fi
  if [ ! -d "${MNTPATH}/boot/" ]; then
    mkdir -p "${MNTPATH}/boot/"
    if [ $? -ne 0 ]; then
      errexit "Unable to make BOOT partition mount point"
    fi
  fi
  mkloop1
  mount "${LOOP1}" "${MNTPATH}/boot/"
  if [ $? -ne 0 ]; then
    errexit "Unable to mount image BOOT partition"
  fi
}

umntimg()
{
  umount "${MNTPATH}/boot/"
  if [ $? -ne 0 ]; then
    errexit "Unable to unmount image BOOT partition"
  fi
  rmloop1
  umount "${MNTPATH}/"
  if [ $? -ne 0 ]; then
    errexit "Unable to unmount image ROOT partition"
  fi
  rmloop2
  rm -r "${MNTPATH}/"
  MNTED=FALSE
}

errexit()
{
  echo ""
  echo "$1"
  echo ""
  if [ "${MNTED}" = "TRUE" ]; then
    umount "${MNTPATH}/boot/" &> /dev/null
    umount "${MNTPATH}/" &> /dev/null
    rm -rf "${MNTPATH}/" &> /dev/null
  fi
  rmloop1
  rmloop2
  exit 1
}

LOOP1=""
LOOP2=""
MNTED=FALSE

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

if [ $(id -u) -ne 0 ]; then
  errexit "$0 must be run as root user"
fi

PGMNAME="$(basename $0)"
for PID in $(pidof -x -o %PPID "${PGMNAME}"); do
  if [ ${PID} -ne $$ ]; then
    errexit "${PGMNAME} is already running"
  fi
done

rsync --version &> /dev/null
if [ $? -ne 0 ]; then
  errexit "rsync not installed (run: apt-get install rsync)"
fi

if command -v systemctl > /dev/null && systemctl | grep -q '\-\.mount'; then
  SYSTEMD=1
elif [ -f /etc/init.d/cron ] && [ ! -h /etc/init.d/cron ]; then
  SYSTEMD=0
else
  errexit "Unrecognized init system"
fi

if [ ${SYSTEMD} -eq 1 ]; then
  ROOT_PART="$(mount | sed -n 's|^/dev/\(.*\) on / .*|\1|p')"
else
  if [ ! -h /dev/root ]; then
    errexit "/dev/root does not exist or is not a symlink"
  fi
  ROOT_PART="$(readlink /dev/root)"
fi

ROOT_TYPE=$(blkid "/dev/${ROOT_PART}" | sed -n 's|^.*TYPE="\(\S\+\)".*|\1|p')

ROOT_DEV="${ROOT_PART:0:(${#ROOT_PART} - 1)}"
if [ "${ROOT_DEV}" = "mmcblk0p" ]; then
  ROOT_DEV="${ROOT_DEV:0:(${#ROOT_DEV} - 1)}"
fi

PTUUID="$(blkid "/dev/${ROOT_DEV}" | sed -n 's|^.*PTUUID="\(\S\+\)".*|\1|p')"

DEVSIZE=$(blockdev --getsize64 "/dev/${ROOT_PART}")
BLKSIZE=$(blockdev --getbsz "/dev/${ROOT_PART}")
BLKCNT=$((${DEVSIZE} / ${BLKSIZE}))
INFO="$(df | grep /dev/root)"
DFKSIZE=$(awk '{print $2}' <<< "${INFO}")
DFKFREE=$(awk '{print $4}' <<< "${INFO}")
ROOTSIZE=$((${BLKCNT} * ${BLKSIZE}))
ROOTUSED=$(((${DFKSIZE} - ${DFKFREE}) * 1024))
IRFSMIN=$(((${ROOTUSED} + (${ADDBLK} * ${BLKSIZE}) + (${ONEMB} - 1)) / ${ONEMB}))
IRFSMAX=$(((${ROOTSIZE} + (${ONEMB} - 1)) / ${ONEMB}))

IMGFILE="$1"
if [ "${IMGFILE}" = "" ]; then
# Create Image file
  while :
  do
    echo ""
    read -r -e -i "${IMGFILE}" -p "Image file to create? " IMGFILE
    if [ "${IMGFILE}" = "" ]; then
      continue
    elif [[ ! "${IMGFILE}" =~ ^/mnt/.*$ && ! "${IMGFILE}" =~ ^/media/.*$ ]]; then
      echo ""
      echo "${IMGFILE} does not begin with /mnt/ or /media/"
      continue
    fi
    if [ -d "${IMGFILE}" ]; then
      echo ""
      echo "${IMGFILE} is a directory"
    elif [ -f "${IMGFILE}" ]; then
      echo ""
      echo -n "${IMGFILE} already exists, Ok to delete (y/n)? "
      while read -r -n 1 -s answer; do
        if [[ "${answer}" = [yYnN] ]]; then
          echo "${answer}"
          if [[ "${answer}" = [yY] ]]; then
            break 2
          else
            break 1
          fi
        fi
      done
    else
      break
    fi
  done
  IRFSSIZE=""
  while :
  do
    echo ""
    read -r -e -i "${IRFSSIZE}" -p "Image ROOT filesystem size (MB) [${IRFSMAX}]? " IRFSSIZE
    if [ "${IRFSSIZE}" = "" ]; then
      IRFSSIZE=${IRFSMAX}
      break
    elif [ ${IRFSSIZE} -ge ${IRFSMIN} ]; then
      break
    else
      echo ""
      echo "Requested image ROOT filesystem size (${IRFSSIZE}) is too small (Minimum = ${IRFSMIN})"
      IRFSSIZE=${IRFSMIN}
    fi
  done
  echo ""
  echo -n "Create ${IMGFILE} [${IRFSSIZE} MB] (y/n)? "
  while read -r -n 1 -s answer; do
    if [[ "${answer}" = [yYnN] ]]; then
      echo "${answer}"
      if [[ "${answer}" = [yY] ]]; then
        break
      else
        errexit "Aborted"
      fi
    fi
  done
  if [ -f "${IMGFILE}" ]; then
    rm "${IMGFILE}"
    if [ $? -ne 0 ]; then
      errexit "Unable to delete existing image file"
    fi
  fi
  ROOTEND=$((${ROOTBEG} + ((${IRFSSIZE} * ${ONEMB}) / 512) - 1))
  truncate -s $(((${ROOTEND} + 1) * 512)) "${IMGFILE}"
  if [ $? -ne 0 ]; then
    errexit "Unable to create image file"
  fi
# create image/partitions
  sync
  fdisk "${IMGFILE}" <<EOF > /dev/null
p
n
p
1
${BOOTBEG}
+${BOOTSIZEM}
t
c
p
n
p
2
${ROOTBEG}
${ROOTEND}
p
w
EOF

  mkloop1
  mkloop2
  mkfs.vfat "${LOOP1}" > /dev/null
  if [ $? -ne 0 ]; then
    errexit "Unable to create image BOOT filesystem"
  fi
  dosfsck "${LOOP1}" > /dev/null
  if [ $? -ne 0 ]; then
    errexit "Image BOOT filesystem appears corrupted"
  fi
  if [ "${ROOT_TYPE}" = "f2fs" ]; then
    mkfs.f2fs "${LOOP2}" > /dev/null
  else
    mkfs.ext4 -q -b ${BLKSIZE} "${LOOP2}" > /dev/null
  fi
  if [ $? -ne 0 ]; then
    errexit "Unable to create image ROOT filesystem"
  fi
  rmloop2
  rmloop1
# Initialise image PARTUUID
  fdisk "${IMGFILE}" <<EOF > /dev/null
p
x
i
0x${PTUUID}
r
p
w
EOF
# Create empty directories in image root partition
  mntimg
  mkdir "${MNTPATH}/dev/" "${MNTPATH}/media/" "${MNTPATH}/mnt/" "${MNTPATH}/proc/" "${MNTPATH}/run/" "${MNTPATH}/sys/" "${MNTPATH}/tmp/"
  if [ $? -ne 0 ]; then
    errexit "Unable to create image directories"
  fi
  chmod a+rwxt "${MNTPATH}/tmp/"
  umntimg
  echo ""
  echo "Starting full backup (for incremental backups, run: $0 ${IMGFILE})"
# END of create image/partitions
else

# Check existing Image
  if [[ ! "${IMGFILE}" =~ ^/mnt/.*$ && ! "${IMGFILE}" =~ ^/media/.*$ ]]; then
    errexit "${IMGFILE} does not begin with /mnt/ or /media/"
  fi
  if [ -d "${IMGFILE}" ]; then
    errexit "${IMGFILE} is a directory"
  elif [ ! -f "${IMGFILE}" ]; then
    errexit "${IMGFILE} not found"
  fi
  echo "Starting incremental backup to ${IMGFILE}"
fi

# rsync root partition
mntimg
sync
rsync -aDH --partial --numeric-ids --delete --force --exclude "${MNTPATH}" --exclude '/dev' --exclude '/media' --exclude '/mnt/*/*' --exclude '/proc' --exclude '/run' --exclude '/sys' \
--exclude '/tmp' --exclude 'lost\+found' --exclude '/etc/udev/rules.d/70-persistent-net.rules' --exclude '/var/lib/asterisk/astdb.sqlite3-journal' / "${MNTPATH}/"
if [[ $? -ne 0 && $? -ne 24 ]]; then
  errexit "Unable to create backup"
fi
sync
umntimg
-1

Open terminal and type 'lsblk -f' .
This should show all connected storage devices.
Then type 'dd if=/dev/[the NAME of your sd card] bs=1M'.
This will take a while so you might want to run it in the background.
This is the same exact way you backup your sd card in Linux.

2
  • That backs up EVERYTHING, even unnecessary and undesirable files. Commented Jan 25, 2017 at 15:27
  • 5
    This will make an inconsistent backup because on a running system things changed during backup!
    – Ingo
    Commented Jul 12, 2018 at 9:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.