Is it possible to update, upgrade and install software before flashing an image?

Question

It seems silly to use our limited SD write cycles to upgrade the software shipped on the images. Can we upgrade the software and install new software before flashing an image to the SD card?

Answer 1

Yes

The answer's always yes, right, just takes a while to work out how!

The Hard Way

I will be running this on a VPS, provided by Brightbox.com. I used a Nano Server (2 CPUs, 512MB RAM, 20GB disk space) and a Ubuntu Precise 12.04 LTS server image. It should work on EC2's or Linode's equivalents, and of course, on your home Linux machine. I have now tested it on my (x86) Arch installation, but know it doesn't work on Ubuntu 10.04 LTS because some of the packages are too old.

Preparing your system - Debian/Ubuntu

Ensure your own system is up to date.

$ sudo apt-get update
$ sudo apt-get upgrade

Install some new software

$ sudo apt-get install binfmt-support qemu qemu-user-static unzip

qemu is an ARM emulator, and qemu-user-static and binfmt-support allows us to run ARM executables without emulating the ARM kernel. (How cool is that!?!)

Preparing your system - Arch

I can't find a statically linked qemu in the Arch repositories, so we will have to compile from source.

1. Download the latest release from http://git.savannah.gnu.org/cgit/qemu.git

2. Unzip and run

   ./configure --disable-kvm --target-list=arm-linux-user --static

3. Build using make and install using sudo make install.

4. Run the following as root

   echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/local/bin/qemu-arm:' > /proc/sys/fs/binfmt_misc/register

   echo ':armeb:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/local/bin/qemu-armeb:' > /proc/sys/fs/binfmt_misc/register

Warning You shouldn't run arbitrary commands you find online as root - these were taken from qemu-binfmt-conf.sh under the ARM cpu type. Please extract the commands from this file and run those.

Download and unzip the image

Go to raspberrypi.org and download the image you want. Unzip it and save the .img file somewhere useful.

$ sudo mkdir -p /images/debian-squeeze
$ sudo wget "http://files.velocix.com/c1410/images/debian/6/debian6-19-04-2012/debian6-19-04-2012.zip" -O "/images/debian-squeeze.zip"
$ sudo unzip "/images/debian-squeeze.zip" -d /images/debian-squeeze
$ sudo rm /images/debian-squeeze.zip

Find the correct partition

The .img will contain 3 partitions, including the boot partition.

$ cd /images/debian-squeeze/debian6-19-04-2012/
$ fdisk -lu debian6-19-04-2012.img
Disk debian6-19-04-2012.img: 1949 MB, 1949999616 bytes
4 heads, 32 sectors/track, 29754 cylinders, total 3808593 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000ee283

                 Device Boot      Start         End      Blocks   Id  System
debian6-19-04-2012.img1            2048      155647       76800    c  W95 FAT32 (LBA)
debian6-19-04-2012.img2          157696     3414015     1628160   83  Linux
debian6-19-04-2012.img3         3416064     3807231      195584   82  Linux swap / Solaris

We need to know the offset of the Linux partition, in this case it is 157696 sectors, and the boot partition, which is at 2048 sectors. Each sector is 512 bytes, so the root offset is 157696*512=80740352 bytes and the boot offset is 2048*512=1048576.

Mount the image as a loopback device

Next, we need to mount the image as a file system. This can be done using a loopback device. We use the offset from the previous section to tell mount which partitions to mount and where. The order of these commands is important.

$ sudo mount -o loop,offset=80740352 "/images/debian-squeeze/debian6-19-04-2012/debian6-19-04-2012.img" /mnt
$ sudo mount -o loop,offset=1048576 "/images/debian-squeeze/debian6-19-04-2012/debian6-19-04-2012.img" /mnt/boot

Preparing the filesystem.

We're nearly ready to chroot into our file system and start installing new software. First, we must install the emulator into our image, as it won't be available once we use chroot.

Debian/Ubuntu

$ sudo cp /usr/bin/qemu-arm-static /mnt/usr/bin/

Arch Linux

$ sudo cp /usr/local/bin/qemu-arm /mnt/usr/local/bin/

All host systems

We also need to provide access to certain other parts of the system.

$ sudo mount --rbind /dev     /mnt/dev
$ sudo mount -t proc none     /mnt/proc
$ sudo mount -o bind /sys     /mnt/sys

chroot

We are done! chroot away...

$ sudo chroot /mnt

You are now in your Raspberry Pi, but the services aren't running etc. Be careful, you are root!

Update/Install software - Debian Image

To update the software, we use apt-get.

 # apt-get update
 # apt-get upgrade

You can also install software using apt-get install as per usual.

Update/Install software - Arch Image

To update the software, we use pacman.

 # pacman -Syu

You can also install software using pacman -S as per usual.

NOTE You can run pacman natively by following the instructions on How do I run my native pacman against a mounted image?.

Exiting

You can exit the chroot by using Ctrl+D and unmount the system by running sudo umount /mnt - you will have to unmount each mount point separately..

You should remove qemu-user-static from /usr/bin or qemu-arm from /usr/local/bin on the RPi, then the image is ready to be flashed.

Final Words

This is a little long and tedious, but do it once and you'll learn loads about how this all works!

Note on latest images

When trying to run do this on the latest images, you will get an error

qemu: uncaught target signal 4 (Illegal instruction) - core dumped
Illegal instruction (core dumped)

To fix this error, simply comment out the contents of the /etc/ld.so.preload file

The Easy Way - piimg

I've started work on a utility for doing a lot of this for you. It is called piimg and can be found at github.com/alexchamberlain/piimg.

So far, it can mount the SD card for you by running

piimg mount /images/debian-squeeze/debian6-19-04-2012/debian6-19-04-2012.img /mnt

and unmount them again by running

piimg umount /mnt

You just need to install qemu and chroot away.

DISCLAIMER I, Alex Chamberlain, am the lead developer of piimg. As such, I may be biased towards the use of piimg in relation to other methods.

References

1. Running ARM Linux on your desktop PC: The foreign chroot way

2. Getting 'illegal instruction' when trying to chroot

Answer 2

Here a quick script I put together as I was rolling it mounts the first Linux partition of an image file Use at your own risk. It has no error handling / input validation

#!/bin/bash

# This script is designed to mount
# the first Linux filesystem
# in the partition table of the
# image handed to it

USAGE="$0 <image name> <mount point>"
IMAGE=$1
MOUNTPT=$2

OFFSET=`fdisk -lu $IMAGE | grep -m 1 Linux$ | awk '{ print $2 *512 }'`

echo "Executing as $USER: mount -o loop,offset=$OFFSET $IMAGE $MOUNTPT"
mount -o loop,offset=$OFFSET $IMAGE $MOUNTPT

Answer 3

I wrote a set of scripts to help with this. Hopefully someone finds these useful:

https://gist.github.com/cinderblock/20952a653989e55f8a7770a0ca2348a8

These all work on non-arm systems too (by automatically installing QEMU if necessary).

• Grow: Grow an image file (so that you can preinstall more stuff)
• Chroot: Get a root shell inside the img (have fun!)
• Prepare: Run a setup script (example provided) in a chroot on the image

Answer 4

I followed the instructions looking for a method to just build stuff for the PI via my main PC, it turns out it was very easy, once you're chrooted you can treat the image as if it were a live system and use all of the native apps. on teh pi, so no need to install a cross compiler :)

I do have one question though, during the setup of the chroot, we have to mount some partitions for the chroot to function properly:

$ sudo mount --rbind /dev     /mnt/dev
$ sudo mount -t proc none     /mnt/proc
$ sudo mount -o bind /sys     /mnt/sys

How do we unmount these properly? leaving them mounted, which stops you from unmounting the .img file, any help here would be appreciated. I tried the -f option, not sure if there is something else underlying that is mounted.

answering my own question, https://bugzilla.redhat.com/show_bug.cgi?id=194342 the 2nd method at the end of the post worked for me with a sudo modification.

cat /proc/mounts | awk '{print $2}' | grep "^$MOUNTPOINT" | sort -r | xargs sudo umount

Obviously change $MOUNTPOINT to the path of your chroot mount point (2nd partition containing the rootfs on the image file, which is /mnt in the above tutorial). Using this method you completely unmount the img file from the loopback devices and any other devices that were mounted via the chroot.

cat /proc/mounts |awk '{print $2}'|grep $CHROOTMOUNT |sort -r|xargs sudo umount
sudo umount $CHROOTMOUNT