According to cursory explanations of the booting process and software such as this, the small second stage bootloader on the SD card enables the GPU to access its firmware on the SD card and load it.
I'm mostly ignorant on the topic of microprocessor design, but as far as I am aware, the code they run is always fetched from somewhere, eg. their own cache or external primary memory. For the built-in first stage bootloader, evidently this is a ROM chip internal to the SoC. For the small second stage loader, I can believe it is run directly from the SD card, but if the GPU
start.elf firmware stuff is resident, I don't believe the GPU has some kind of cache or something big enough to hold it, which implies it's actually kept in the board's RAM.
Is that correct? Or is this also code that's just run once directly from the card? Or something else completely?