I have noticed that my Pi will not boot from a battery (supplying 5.26v Raspberry Pi 4 Model B) unless I first start it up from the USB-c power supply and then connect the battery power supply. My multimeter tells me I am giving it 5.26 volts and it draws 500 milliamps as per specification. Very confused, thanks.
it draws 500 milliamps as per specification
If you mean the Pi 4, you are referring to a minimum that it draws while idle. Electronic gadgets, especially computers, do not draw a steady fixed amount of current like a light bulb. Their draw depends on their activity. If your battery is failing at boot, it will fail at other times when the system is sufficiently stressed.
How do I let it draw more amps?
You attach a sufficient power supply. There's no special setting whereby you can tell to not draw the power it needs, and hence, change that to "let it draw more".
If the Pi does boot successfully using a normal wall supply but does not with a battery, it is probably not the Pi that is that is the problem, it's the battery.
It's the battery. Unless you've got a massive* battery power source, you are asking (actually begging) for trouble with your RPi. Your comment to @joan's answer suggests that you don't appreciate the fact that a battery's instantaneous capacity to supply current has very little to do with it's energy storage rating. Power is an instantaneous measure; energy is power over a period of time; 2.4 A-hr is a cumulative measure of energy storage - not instantaneous power delivery.
From your question, it seems you've already proven this to yourself, but you haven't acknowledged it yet:
"I first start it up from the usb c power supply and then connect the battery power supply." Your multimeter measurements contribute to your confusion because a multimeter is incapable of capturing the instantaneous - it is an averaging instrument. You'll need an oscilloscope to capture transients.