There is no performance hit. The Pi components, like all electronics, have minimum and maximum voltage ratings. If the voltage is not high enough, it will not boot. If it is at a threshold level, you will get intermittent low voltage warnings.
An issue with voltage is that it can fluctuate, particularly under load (current draw), which widely varies for a running computer. If you suddenly max out all the cores, this is a demand for current. In an imperfect world, this will tend to mean a momentary slight voltage drop. If the voltage was already near the minimum threshold and this causes a drop below it, even for just an instant, the pi will shut down. Chances are the voltage will then rise again and the system will effectively reboot after an unclean shutdown.
However, the boot process itself is intense work, so if there is this potential at the outset, it will probably happen then. But if not, that's no guarantee it won't. Also, there may be loads from peripherals that aren't significant until after booting, in which case the potential will increase as they are put into use.
So performance wise it is an all or nothing thing.