I actually like the comment and answer that both say that without physical security, you're doomed. Your question doesn't tell us the use-case. However, I am challenged by the problem, so here goes.
If you are placing your Pis in a relatively benign environment, you may be able to pull this off. But only if the reward from hacking into one of them is small. If, however, you're bugging an embassy or storing bitcoins, you can't do what you want.
So, the first thing you must do is a risk analysis. There are two questions: How likely is a breach, i.e. how motivated are those with physical access, and how bad are the consequences of a breach?
If the risk analysis looks OK, consider a tamper-resistant case for the Pi. It covers the SD card and any ports you won't want accessible. The lid is attached with tamper-resistant screws, and there's a micro-switch that opens when the lid is removed. The micro-switch pulls a GPIO pin high. You can probably 3D-print such a thing. Add a sticker that says, "Opening voids your warranty, steals your significant other, and leaves leftover pizza under the sofa."
Once the Pi is safely in the case, install a watchdog program. The watchdog program starts when the Pi boots, waits perhaps a second for everything to settle down, then polls that GPIO pin, perhaps every 250 ms. If the pin isn't pulled high when sampled, the watchdog program 1) sends a tamper alert message, 2) overwrites the VPN key, and 3) deletes other sensitive data.
How safe is this? Not very. I could breach the case by cutting around the micro-switch, or just cutting an opening to allow access to the SD card. "Overwriting" on an SD card probably doesn't, because wear-leveling probably remaps sectors. Ditto for deleting confidential stuff.
So, take my hypothetical tamper-resistant case and your risk analysis and decide.