How do I configure the Raspberry Pi to boot with an NFS root?

Question

How can I make my Raspberry Pi use an NFS server for the root filesystem, instead of the SD card?

I'd like to be able to set up my Raspberry Pi to have its root file system on a remote machine via NFS for performance/backup/development purposes. How can I do this?

Answer 1

The short version

1. You need to prepare a root and export it from the NFS machine
2. You need to configure the early stage booting to use that

You'll still need a SD card for the early stage booting, but after that nothing except /boot will need the SD card.

Details

I've only done this on a Linux machine. In theory it's possible on any machine that can export NFS filesystems though.

1. The easiest way to prepare a root filesystem is to copy the one provided by the existing images. I used the official Debian 6 image as my starting point:

   1. Download the image, extract the .zip file. You should now have debian6-19-04-2012.img somewhere

   2. Mount the root partition from that image using a loopback device. You'll need the offset into the image for the start of the root partition, in this instance it's the second partition in the image:

      fdisk -l debian6-19-04-2012.img
      
      Disk debian6-19-04-2012.img: 1949 MB, 1949999616 bytes
      4 heads, 32 sectors/track, 29754 cylinders, total 3808593 sectors
      Units = sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disk identifier: 0x000ee283
      
      
      Device Boot      Start         End      Blocks   Id  System
      debian6-19-04-2012.img1            2048      155647       76800    c  W95 FAT32 (LBA)
      debian6-19-04-2012.img2          157696     3414015     1628160   83  Linux
      debian6-19-04-2012.img3         3416064     3807231      195584   82  Linux swap / Solaris
      root@rapunzel:/tmp/debian6-19-04-2012# 
      

      The second partition starts at block (or sector) 157696 so the offset is (512*157696) = 80740352:

      root@rapunzel:/tmp/debian6-19-04-2012# mkdir /mnt/pi-root-loop
      root@rapunzel:/tmp/debian6-19-04-2012# mount -o loop,offset=80740352 /tmp/debian6-19-04-2012/debian6-19-04-2012.img /mnt/pi-root-loop
      root@rapunzel:/tmp/debian6-19-04-2012# ls /mnt/pi-root-loop
      bin   Desktop  etc   lib    mnt  proc  sbin  selinux  sys  usr
      boot  dev      home  media  opt  root  sd    srv      tmp  var
      root@rapunzel:/tmp/debian6-19-04-2012#
      

   3. With the image mounted you probably want to copy this somewhere more permanent to avoid using the loopback device forever. I made an LVM2 volume and mounted it at /mnt/rpi-root, so I did:

      cp -rav /mnt/pi-root-loop/* /mnt/rpi-root/
      

   4. We need to export the filesystem now, so we need to add a line like:

      /mnt/rpi-root  192.168.1.2(rw,no_root_squash)
      

      into /etc/exports on the NFS server, where 192.168.1.2 is the address of the Raspberry Pi. After adding that run exportfs -rv to re-read the list of exports. (See exports(5) for more advanced options including wildcards/subnets rather than individual hosts)

   You could alternatively skip this step and build a root filesystem from scratch using something like multistrap.

2. Now we need to build an SD card that does the early stage booting and boots using this NFS share. I did this by taking the first partition of the official image and copying that to an SD card, where the SD card is /dev/sdx:

   dd if=/tmp/debian6-19-04-2012/debian6-19-04-2012.img of=/dev/sdx bs=512 count=155647
   

   and then since this only copied the first partition editing the partition table with fdisk to delete the two partitions we didn't copy.

   After that I mounted the partition from the SD card and edited the file cmdline.txt from it to read:

   dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/nfs nfsroot=192.168.1.1:/mnt/rpi-root ip=dhcp rootfstype=nfs
   

   where 192.168.1.1 is the address of the NFS server. If you don't want to use DHCP you can set the IP address alternative ways, see this document for further details.

   Once this is done you can unmount the SD card and boot your Rapsberry Pi with an NFS root. You can also manipulate the filesystem from the server side, or other NFS clients as you see fit (e.g. QEMU+binfmt_misc, LVM snapshots, backup systems etc.)

Optional things

• Setting RAMRUN and RAMLOCK to yes in /etc/default/rcS causes /var/run and /var/lock respectively to be mounted as tmpfs (i.e. ramdisk) - this is probably worthwhile, even if you only have the one device booting from the NFS filesystem.

• If you're using ip=dhcp you'll probably need to comment out the line:

  #iface eth0 inet dhcp
  

  in /etc/network/interfaces (/mnt/rpi-root/etc/network/interfaces) in order to avoid a situation where swapping between the early boot DHCP and the usual later one causes a message like:

  nfs: server 192.168.1.1 not responding, still trying

• If you want to boot lots of devices from the same root you'll need to take care of providing a "local" /tmp (probably tmpfs) and other subtleties to avoid problems with multiple machines trying to write to the same files.

Answer 2

I don't currently have enough reputation to comment on the answer above (in fact, I've signed up just to comment). Supremely helpful!

Two elaborations: I was installing the resulting file system on my Thecus 4100+ NAS box, and the current Wheezy Raspbian distribution is on an ext4 format .img file, which my NAS doesn't support (it's too old, and understands ext3). So I had to do the extraction on my current Linux server. I originally tried doing the extraction with the cp -rav command listed above, writting directly to an NFS mount from the NAS box, but couldn't boot the from that, due to issues over the file permissions. I could probably have dealt with that via NFS, but in the end, I created a tar file directly from the loopback file system, which I then untarred on the NAS box.

secondly (and very minor): according to the rcS man page: The RAMLOCK, RAMSHM and RAMTMP variables have been moved to /etc/default/tmpfs; RAMSHM and RAMTMP settings in rcS are used (if set) for backward compatibility, but will be overridden by settings enabled in /etc/default/tmpfs.

(I also had the rootdev issue, which originates in the /etc/init.d/checkroot.sh script. I resolved it by altering /etc/fstab as Klaudiusz suggests.)