Putting two caps that are rated at 2.5V in series is NOT going to give you an equivalent 5V rated cap. You shouldn't do that. And think through the physics of holding up a (let's say) 5 Watt load with capacitors while you move wires around. There are better ways to do this:
Perhaps the simplest way to do what it sounds like you want to do is as follows:
simulate this circuit – Schematic created using CircuitLab
The diodes form a "wired-OR" power source that isolates one battery from the other. In this way, both batteries may be connected at the same time - effectively giving you a "make before break" power switch. The diodes are Schottky diodes chosen because they have a lower forward voltage drop than a p-n junction diode. In fact, you could extend this scheme to 3 or 4 or more batteries. This particular part (DSS 20-0015B) appears to have good availability & reasonable price.
One potential issue with this scheme is the batteries themselves, and the RPi's rather tight tolerances for input voltage. If these figures are still correct (4.75V - 5.25V), your batteries will need to be almost "spot on" to get reliable operation. If your application is "critical", all is not lost as we can still use the "wired-OR" scheme above with a slight twist:
simulate this circuit
Adding a buck regulator/converter will easily keep the RPi input voltage in the 4.75V - 5.25V range, using higher voltage batteries. And as before, the "wired-OR" configuration allows you to swap batteries without dropping power to the RPi. In this case, we don't need Schottky diodes, so a conventional diode/rectifier (e.g. 1N5626) may be used.
Buck regulators that meet your specs are readily available as ICs, but may require wiring up a few external components. If you prefer a more finished regulator, those are also available as an assembly from the usual sources. You should probably avoid linear regulators in this application as they are much less efficient than a switch-mode buck converter - but they'll certainly work.