It can run (slower) on 5v, which is fine for me Question is, is this possible / recommended ? I just want to be able to turn it on and off
All the Pi's GPIO are 3V3 and can only supply a few milliamps, not enough to power your motor.
Even if the GPIO could supply enough power it would be damaging to connect a motor direct to the Pi. The back EMF from the motor as it is switched off could destroy the Pi/GPIO.
The 5V pins on the expansion header are not GPIO, they are not switchable. They are always "on" while power is applied to the Pi.
You can buy a motor driver board for a few UK pounds from eBay, e.g. look for L9110S and L298N boards. That would be the "correct" way to control a motor from the Pi.
It would be sensible to use an external power supply to power the motor. In that case you still need to join the Pi ground with the power supply ground.