Well, the answer seems to be that you can't switch from Raspberry Pi 3B+ to 4B. At least not with my specific installation.
I just got my new Pi 4 today, and booting up on various cards, USB flash drives and SSDs via USB, doesn't give me the LAN or WIFI.
Even going into raspi-config, I get an error, trying to set the country for my WIFI.
But the first error really shows up, during boot, where there's a counter for waiting for LAN to be connected. I'm sure, there is a really easy fix for this, like the Raspberry expect to have another IP or MAC adress, but since it's another unit, WIFI and LAN doesn't work.
But doing a ifconfig -a, reports the units correctly, showing each units's MAC adress. Just not an IP.