That would have been a delightfully short answer, but "body must be at least 30 characters; you entered 1". :)
The X server is really usually called
X and you can just start it. You can set the DISPLAY number as an argument along with some other things.
You might want to have some sort of session, though, and still go through xinit or such and start X with startx. You can use
.xinitrc or such as a script and simply not start a wm there. You will need a "magic client" as the last command that stays running so X doesn't terminate immediately. In a traditional "failsafe" session, that was an xterm. If you know what to launch and launch once only, it could be that mplayer. Or any UI/wrapper that does the launching for you. When the last ("magic") client in the init script terminates, X terminates.
If all you want to do is play video, you might see if rpi can run mplayer with directfb instead and skip using X11 entirely.