You said in a comment:
... No, there are no lit LEDs at all. Its an old computer psu, so theres 12V, 5V and 3V3...
That may be the case when the PSU is properly powered up but if there is only a tiny load on just one output (the 5V one) the Switch-Mode Power Supply is probably unstable and may be shutting down because the draw on it's outputs is so little. Have a look at the output ratings on the PSU and you will probably find that it is expecting to provide tens of amperes of current at 5V and probably several at 12V. Under conditions of underload the output voltages may start to rise and if they get too high it may be shutting down to prevent damage to the circuitry to which it is connected.
Also, depending on the era of the unit it may need a switching voltage to be provided to a control line (or a terminal to be grounded), see the connection labelled Power on here: https://en.wikipedia.org/wiki/Power_supply_unit_(computer)#Wiring_diagrams.
If you have a spare automotive direction indicator bulb you could wire up one or two of those between a 12V output and ground.
One thing to bear in mind is that: although those outputs are at a nice low, safe, voltage, they can provide a fair amount of current and if you short them out with say a ring or a metal watch strap that current is going to make those things get very hot very quickly - the standards seem to say that each output should be limited to providing no more than 240VA (effectively Watts) and at 5V that corresponds to nearly 50W - OTOH the heat from that should cauterise the stump should a red-hot wedding ring burn your finger off...
