This chart shows the Pi 3B+ drawing upwards of 5.6 W under load. I know they recommend a 2.5a power supply, but is that an exactly accurate number? Would 2.4a be enough? How powerful of a power supply do I need to provide this much power?
The under load Pi3B+ figure of 5.6 watts is 1.12 amps.
So you only need just over an amp to power a loaded Pi3B+.
My unloaded Pi3B+ is currently running powered from one of my laptop USB slots.
The rest of the 2.5 amp is an allowance for power hungry devices connected to the Pi (e.g. to the Pi's USB sockets). If you don't have such devices your power supply should be fine.