How do I setup a USB drive as a boot drive on Raspberry Pi 4B?

Question

I am somewhat new to Raspberry Pi.

I would like to make the 5 TB WD My Book USB drive the boot drive with the Raspberry Pi OS FULL (32-Bit) image. If I use the Raspberry Pi Imager to configure the USB Drive, I encounter an error on the firstboot because of insufficient privileges to create partitions. I have also noticed that the Raspberry Pi Imager converts the drive to MBR. This renders 3TB of the drive unusable.

I am going to be using some development tools that only work when installed to the boot drive.

Thus far, I have had no luck what-so-ever on using a 5 TB GPT drive as a boot drive. I can use it as a data drive.

Most of the problems are partition issues.

What I have tried

Imager

When the the `Imager` on the WD My_Book 5 TB drive (with no
no partitions); an attempt is made during the firstboot to 
resize the /dev/sda2. This fails and leaves the drive in an 
error state. The error that appears in GParted is:

Superblock checksum does not match the superblock while trying
to open /dev/sda2. Couldn't find valid filesystem superblock.

1. Issued the following command:

$sudo su - root

2. Removed all existing partitions from the USB drive in GParted.

3. Started the Imager.

4. Set the following options:

Operating System: RASPBERRY PI OS FULL (32-BiT)
Storage: WD MY_BOOK

5. Imager successfully writes and verifies to the USB Drive. The following message appears:

Write Successful

Raspberry Pi OS Full (32-bit) has 
been written to WD My_Book_25EE.

You can now remove the SD card from
the reader

6. Removed the SD Card and Clicked OK.

7. Re-booted the Raspberry Pi.

The Raspberry Pi endlessly cycles between: Trying Boot Mode USB-MSD and Trying Boot Mode SD

OR

Subsequent re-boot produces the following error:

Failed running firstboot:
Partition table resize of the root partition (/dev/sda2)
failed

Manually define Partitions

I have worked through partitioning and formatting the partitions of the 5 TB USB drive. My steps are below.

1. Unmounted the drive from the Raspberry Pi File Manager.
2. Opened a terminal and started a parted session.

$ sudo parted

3. Selected the /dev/sda drive to partition.

(parted) select /dev/sda

4. Created a GPT partition table.

(parted) mklabel gpt

5. Created new partitions on the drive.

(parted) mkpart os fat32 0gb 750gb
(parted) mkpart dev-ntfs 750gb 50%
(parted) mkpart dev-ext4 50% 100%

6. Exited parted.

(parted) q

7. Formatted the new partitions.

$ sudo mkfs.ntfs -L dev-ntfs -Q /dev/sda2
$ sudo mkfs.ext4 -L dev-ext4 /dev/sda3

8. Powered down the Raspberry Pi and restarted.
9. Temporarily remounted the partitions under /mnt.

$ cd /mnt
$ sudo mkdir dev-ntfs
$ sudo mkdir dev-ext4

10. Temporarily mount the partitions.

$ sudo mount /dev/sda2 /mnt/dev-ntfs
$ sudo mount /dev/sda3 /mnt/dev-ext4

11. Obtain the PARTUUID for the new partitions.

$ sudo blkid

12. At a stand-still until I can find a way to install the OS on the USB.

rpi-clone

1. Unmounted and deleted all partitions from the USB Drive in GParted.

2. Tried using rpi-clone as per https://github.com/billw2/rpi-clone

3. Set the boot-order in raspi-config.

4. This clones the SD-Card to the USB but renders the majority of the USB drive unallocated and unusable.

The Raspberry Pi will not boot from the USB drive.

Test Case with 128 GB USB (This Works)

As a test case I setup a 128 GB USB stick by using the Raspberry Pi Imager.

This works.

1. Boot the Raspberry Pi 4B from an SD Card.
2. Make sure the Raspberry Pi 4B is up-to-date.

$ sudo apt-get ugrade -y
$ sudo rpt-eeprom-update 

My Raspberry Pi 4B was already up-to-date from attempts with
the GPT drive.

3. Plug the 128 GB stick into a USB3 port.

4. Use the Raspberry Pi Imager to burn the OS onto the 128 GB USB stick.

5. When the imager completes the verify of the write to the 128 GB USB, shut down the Raspberry Pi 4B.

6. Take the SD-Card out of the Raspberry Pi 4B.

7. Power the Raspberry Pi 4B on.

Questions

1. I would like to setup a USB GPT drive with desired partitions.

 I have setup partitions on the USB GPT that ensures all 
 partitions are < 2 TB and fat32.

I would like to:

• Setup the desired partitions manually on a USB GPT drive with a predefined boot partition.

• Copy the boot from a working SD-Card to the predefined USB GPT drive.

• Boot from the GPT USB Drive and by-pass the automatic resize of the GPT USB Drive during firstboot (or any boot).

Answer 1

The imager takes a compressed .img file and overlays what is on your target drive. The .img file is the DOS MBR; some space; the VFAT32 partition (containing the kernel and cmdline.txt that will later be mounted at /boot); possibly some space; and then the root partition (mounted, of course, at '/').

Since the .img file is of an image of a DOS MBR drive, not a GPT drive, it can only support a ~2TB drive. And, if your drive was formatted (or more likely came formatted) as GPT, there is a backup partition table at the end of the drive. If you use the "imager", many things will then complain that the GPT drive is corrupted (it sees the backup partition table at the end of the drive, and sees the GPT partition table at the front of the drive has been "wiped out".

As GPT partition tables are checksummed, it is hard to use the old ways (imager) of creating a bootable drive. (Way out of my pay-grade.)

I have a cheap utility written in 'C' on github, https://github.com/tommylovell/raspberry-split, that breaks out the 'boot' and 'root' partitions as separate files so they can be mounted on loopback and rsynced to their destination filesystems. I manually (well, by script) create target filesystems as the first two partitions on the GPT drive; rsync files to where they should be; edit cmdline.txt and fstab so they are correct; and the drive is now bootable on a Pi4B.

Unfortunately, everything I have runs under Ubuntu Linux, so it could be run on the Pi -- slowly... But, it is actually a lot simpler than it sounds.

Sorry this answer is so long. If you want, I can post the script I use as an addendum to this answer.