After enabling ssh, I could figure out that the xserver did not start.
"no screens found".
So I tried out all the ways to config the pi in headless mode
Editing the config.txt for fixed hdmi modes (see here), fiddeling with
raspi-config, updating the firmware - nothing worked.
Eventuall a look into the Xorg.0.log (in /var/log/) showed this:
"Failed to load module 'fbturbo'". With that info I stumbled upon this post. (I copy the essential information from there)
My root cause was, that I forgot to update the:
/etc/apt/sources.list.d/raspi.list from stretch to buster
After replacing 'stretch' in the raspi.list,
sudo apt update reported an additional ~ 200 packages that could be upgraded.
sudo apt full-upgrade completed without error.
But still no X server, still no display found in the Xorg.0.log , still the
"Failed to load module 'fbturbo'" error.
sudo apt-get install xserver-xorg-video-fbturbo but no success. Looking around with apt-cache I can see the currently install module is xorg-video-abi-24 and the demanded xorg-video-abi-23 does not exist for buster.
Downloaded the package here and installed it manually.
startx still did not work
Eventually found the solution here : Remove the fbturbo driver with
sudo mv /usr/share/X11/xorg.conf.d/99-fbturbo.conf ~
startx brings up the desktop... finally.
I do not know of the consequences kicking out the driver, but at least the desktop was back up and I could connect with VNC again.
BTW: On the same thread somebody mentions that one can get the fbturbo working with this tutorial, but I did not try (yet).
Sorry for going so much into the details - but I hope a rather long description helps someone who has the same issue as I did. (and makes it easier to find this issue on google)