So I'm using a Lipo to power my Pi 3 and to be safe, I installed a 5v 3A UBEC. Upon checking the voltage output, I saw the BEC was only stepping the voltage down to around 5.25v, rather than 5v. I know this is pushing the recommended max voltage. Do you think it will be Ok? Thanks!
5.25 V should be acceptable.
As noted in Raspberry Pi Power Limitations:
Power sources SHOULD provide 5±0.25V and often list a current rating
I couldn't find any explicit mention of voltage in the Pi FAQ, but it turns out that the voltage constraint is part of the USB standard. Note that the tolerance listed there is 4.40 — 5.25 V (in the sidebar).
See also in the RetroPie overclocking documentation:
The Pi runs off 5V but there's nothing wrong with a power supply providing 5.1V or 5.25V. There's usually about 0.25V drop over a typical USB power cable so the extra voltage helps compensate.
Increasing the voltage too much further could cause damage, but you're not in that range yet:
One way to increase that reliability is to increase the voltage to the component. This has the tradeoff that more heat is generated by the component. A component may be damaged if too much voltage is supplied.
As long as the heat doesn't get excessive, and you don't go above 5.25 V, you'll be fine.