I have a RPi connected to a small screen, and I can connect to the device using SSH. I want to be able to open applications/send keystrokes that will show up on the desktop.
Ideally I would like to host a flask server on the pi that can interact with the desktop when I make requests to it. I currently have a flask server on the device, and I have no issues starting processes, running background tasks, but opening GUI applications is eluding me.
Consider using a VNC (Virtual Network Computing) type application. There are commercial and open-source versions. It should be possible to run a VNC server on the Raspberry Pi and a client on any number of platforms. If configured correctly, VNC should allow the client to control mouse and keyboard inputs remotely while displaying the Raspberry Pi console on the remote client.
You could research xdotool, that allows you to send mouse controls and keystrokes to the X11-environment.
