I've tried 5 different chargers, and none of them gets TP1>TP2 over 4.6 volts. I just plugged in a new charger (found here) which gives it 4.4 volts with only SD card attached, but 4.8 with SD card removed.
I tested the power before the polyfuse without SD card inserted: 4.94 volts
Voltage after polyfuse: 4.86 volts
So far, so good. Power off Pi, insert SD card, power up Pi and test again: 4.78 volts before polyfuse, 4.39 volts after polyfuse.
I tested the resistance of the F3 polyfuse, it seems to be a constant 1.2 Ohms. Is the fuse bad, or is the SD card somehow using that much power?