How to get netboot debug info from start.elf

Question

I am attempting to netboot Ubuntu 20.04.1LTS arm64 on a Raspberry Pi 3B v1.2. To do this, I have replaced the Ubuntu u-boot boot loader with the Pi boot loader, as instructed in the Ubuntu documentation. From my tftp server logs I can see that all correct files are being loaded, but the Pi still does not progress past the rainbow splash screen.

I have set uart_2ndstage=1 in my config.txt, which has given me serial output of bootcode.bin, which shows config.txt, start.elf, and fixup.dat get loaded. According to the Raspberry Pi Documentation, the option should also enable serial output for start.elf, but that is not the case.

I have further set the option start_debug=1 in my config.txt and see that start_db.elf is now being loaded, but do not get any additional serial output.

As an additional test I have set the same options in the config.txt when used from an sdcard, so setting the serial options as well as using the adjusted boot loader and kernel. This leads to an identical hang.

Edit: I have found the cause of my issue. Setting arm_64bit=1 (or any non empty value) makes it so either the bootcode.bin or start.elf expect a non-compressed kernel image. Ubuntu gives compressed images by default. By manually decompressing the image, the Pi boots. My original question still stands, however.

My question is: what can I do to see serial output of what start.elf is doing and expecting?

Answer 1

I don't know if this fits to your installation but on Raspbian you can patch the bootcode.bin file to get debug output on the serial console. First look if you can find the flag with:

rpi ~$ strings /boot/bootcode.bin | grep BOOT_UART
BOOT_UART=0

Then patch it with:

rpi ~$ sudo sed -i -e "s/BOOT_UART=0/BOOT_UART=1/" bootcode.bin

I have taken this from Raspberry Pi boot modes. For further information look there.

Answer 2

According to this Github issue (this is for RPi4, but as long as you have recent versions of bootcode.bin & start.elf the same config.txt parameters should apply) you are correct and uart_2ndstage=1 should enable UART debugging for both Second Stage bootloader (bootcode.bin) and GPU firmware (start.elf):

Setting uart_2ndstage=1 causes the second-stage loader (bootcode.bin on devices prior to the Raspberry Pi 4, or the boot code in the EEPROM for Raspberry Pi 4 devices) and the main firmware (start*.elf) to output diagnostic information to UART0.

Be aware that output is likely to interfere with Bluetooth operation unless it is disabled (dtoverlay=disable-bt) or switched to the other UART (dtoverlay=miniuart-bt), and if the UART is accessed simultaneously to output from Linux then data loss can occur leading to corrupted output. This feature should only be required when trying to diagnose an early boot loading problem.

https://www.raspberrypi.com/documentation/computers/config_txt.html#uart_2ndstage