Raspbian boot process and the partition table

Question

Any a good description of the Raspberry Pi boot process and the disk partition structure?

What is stored on each partition (and why)?

Does the RPI boot from the SD card (sector zero) or the firmware reads from the disk structure?

I assume that being NOOBS created copying a few files under any O/S, the RPI firmware knows about (high level) MS-DOS disk structure and find it's way into the SD memory.

I want to clone a good SD card to a larger SD card to extend the available space.
Can I create and partition a new SD and just DD each partition to the new device? Which partitions are needed?

The 'linux' partition on the new device be larger than the original and then extend the disk space?

Maybe clone into a new card and extend only the last partition (p6) to use the available space, but I don't want to use a try and error method, will like to know how and why.

Why my question?

Most RPI SD cards have two (fdisk) partitions, perhaps when installed from a .img

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        2048   155647   153600   75M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      157696 15351807 15194112  7.3G 83 Linux

However I have one with as many as 6 partitions, this is the structure inherited from NOOBS.

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192     1675781      833795    e  W95 FAT16 (LBA)
/dev/mmcblk0p2         1679360    15499263     6909952   85  Linux extended
/dev/mmcblk0p3        15499264    15564799       32768   83  Linux
/dev/mmcblk0p5         1687552     1810431       61440    c  W95 FAT32 (LBA)
/dev/mmcblk0p6         1818624    15499263     6840320   83  Linux

I think something is wrong with this picture. Two W95 partitions?

I will like to clone only the right partitions to a new card. I have to much work on this running SD to start all over again! (BTW I have a few good backups).

Some notes.

Partition p1

-rwxr-xr-x 1 root root  18K 2015-03-09 04:45 bootcode.bin
-rwxr-xr-x 1 root root 2.2K 2015-03-09 04:45 INSTRUCTIONS-README.txt
drwxr-xr-x 4 root root  32K 2015-03-09 04:45 os
...
-rwxr-xr-x 1 root root 542K 2015-03-09 04:45 recovery.elf
-rwxr-xr-x 1 root root    0 2015-03-09 04:45 RECOVERY_FILES_DO_NOT_EDIT
-rwxr-xr-x 1 root root 2.1M 2015-03-09 04:45 recovery.img
-rwxr-xr-x 1 root root  20M 2015-03-09 04:45 recovery.rfs
-rwxr-xr-x 1 root root 9.5K 2015-03-09 04:45 riscos-boot.bin
drwxr-xr-x 2 root root 8.0K 2015-12-05 18:13 System Volume Information

Partition p2

mount: /dev/mmcblk0p2: can't read superblock

Partition p3

-rw-r--r-- 1 root root 274 1969-12-31 19:18 installed_os.json
drwx------ 2 root root 12K 1969-12-31 19:00 lost+found
-rw-r--r-- 1 root root  86 1969-12-31 19:18 noobs.conf

Partition p4 does not exist.

Partition p5:

-rwxr-xr-x 1 root root 9.7K 2015-12-01 20:17 bcm2708-rpi-b.dtb
...
-rwxr-xr-x 1 root root  18K 2015-12-01 20:17 bootcode.bin
-rwxr-xr-x 1 root root  120 1979-12-31 19:00 cmdline.txt
-rwxr-xr-x 1 root root 1.5K 2015-11-23 09:01 config.txt
-rwxr-xr-x 1 root root  19K 2015-10-14 05:43 COPYING.linux
...
-rwxr-xr-x 1 root root 9.5K 2015-12-01 20:17 fixup_x.dat
-rwxr-xr-x 1 root root  137 2015-02-16 09:57 issue.txt
-rwxr-xr-x 1 root root 3.9M 2015-12-01 20:17 kernel7.img
...
-rwxr-xr-x 1 root root 591K 2015-12-01 20:17 start_cd.elf
...

Partition p6 (the real raspbian partition)

drwxr-xr-x   2 root     root     4.0K 2015-07-28 21:40 bin
drwxr-xr-x   2 root     root     4.0K 2015-02-15 09:09 boot
drwxr-xr-x 119 root     root     4.0K 2015-12-23 15:36 etc
drwxr-xr-x   4 root     root     4.0K 2015-12-19 20:53 home
drwxr-xr-x  15 root     root     4.0K 2015-12-20 22:08 lib
drwx------   2 root     root      16K 2015-02-15 06:21 lost+found
....
drwxr-xr-x  12 root     root     4.0K 2015-11-04 15:18 var

Answer 1

My final outcome to migrate to a larger SD card on a noobs setting, is as follows.

I backup the original live image on a temporary external drive, shared with a windows box, an spare memory stick will do it.

dd if=/dev/mmcblk0 of=~/backup.img

you can also backup with win32DiskImager, then, I burned the image into a larger SD card (I went from 8 to 16GB) this time, using win32DiskImager.

I booted the newly created SD. At this point, the SD still kept the same partitions, primaries, extended and logical. So boot should works smoothly.

Using

fdisk /dev/mmcblk0

Take a live screenshot of the current partition table using the p command.

Delete the unused and unneeded partition #3 (p3) located at the end of the physical 'disk' (after the extended partition #2) with d.

Next, bravely delete partition #2 (p2), the nested logical partition will also vanish.

Recreate extended partition #2, but now, using all the available space.
To recreate #2, use the very exact same start original sector and let fdisk to give you the ending sector. Use p verify and that the type is extended type '85' and that the new ending sector is indeed larger than the original.

Recreate the two previous logical partitions: start with logical partition #5 (p5); for the start sector use the exact sector from the previous screenshot, same with the ending sector (this logical disk will not grow), use t to set the id to c (MS-DOS). Then, recreate partition #6 (p6) with the same start sector than before but with the end sector selected by fdisk and p' id83`.

Verify your entries with p and if and only if, looks ok just write the partition table with w otherwise just ^C and start again.

Reboot and cross your fingers. Raspbian may reboot twice.

Then just expand the file system with

resize2fs /dev/mmcblk0p6

This is the end result, now with extra 8gb!

 Device Boot             Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192     1675781      833795    e  W95 FAT16 (LBA)
/dev/mmcblk0p2         1679360    30703615    14512128   85  Linux extended
/dev/mmcblk0p5         1687552     1810431       61440    c  W95 FAT32 (LBA)
/dev/mmcblk0p6         1818624    30703615    14442496   83  Linux

So, now my ending sector is 14,442,496 from 6,840,320.

The Boot

I know that this topic is elsewhere, however this is good to know.

Raspberry have a two on board processors: the GPU (Graphic Processing Unit) and the ARM, each with a different task . The GPU share the memory with the ARM, a very efficient general purpose RISC (Reduced Instruction Set Computer) processor, running industry standard instruction set. All enclosed on a BCM2835 'SoC' (System on a Chip)

Upon power on the control is first given to the GPU, only the kernel, at the end of the boot process, will run on the ARM processor.

On the first stage, the GPU will run the on-chip ROM firmware to load bootcode.bin into the L2 cache memory and pass the control. bootcode.bin will enable SD card, enable SDRAM, and use the first MS-DOS partition, to find, load and run 'start.elf'.

At this point, the following files are considered by bootcode.bin from the first MS-DOS partition on the SD Card:

/boot/config.txt,  
/boot/cmdline.txt  (kernel parameters)
/boot/bcm????-rpi-*.dtb  (device dependent) 

There are a good number of options that can be customized on config.txt. You can edit the file mounting the SD on an external computer or just edit on a running unit:

nano /boot/config.txt

Typical changes on config.txt are related to overclock, HDMI, screen size, aspect, camera, set you MAC address etc.

nano cmdline.txt
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p6 rootfstype=ext4 elevator=deadline rootwait

As you can see, the designated boot partition, where kernel.img will boot is /dev/mmcblk0p6 and this is the second partition required on your SD card, formatted as 'ext4' (Linux or Id '83' in fdisk lingo).

Finally, start.elf will load and run kernel.img, your linux kernel.

You can (should?) update the kernel with rpi-update; if not there just apt-get install rpi-update).

note: the sudo prefix deleted for brevity.

Answer 2

If you google for Raspberry PI Boot process you will find lots, including What is the boot sequence? (there have been some minor changes).

What is stored on each partition (and why)? This varies, depending on how it was setup, and what distribution is used.

Does the RPI boot from the SD card (sector zero) or the firmware reads from the disk structure? No.

Can I create and partition a new SD and just DD each partition to the new device? Yes. Although you may be better to clone the card and expand the partition.

Most RPI SD cards have two (fdisk) partitions, perhaps when installed from a .img While Raspbian does, the FAT32 partition normally starts at 8192, which is ideal for SD Cards. (Partitions should be located on Erase Block Group 4M boundary.)

However I have one with as many as 6 partitions, some overlapping other sectors, perhaps this is the structure when started/inherited from NOOBS. You need to understand extended partitions. Try Wiki.