To the best of my knowledge it's absolutely fine to (carefully) de-solder the HDMI and USB sockets from the Pi. Several people have documented this procedure, including the exceedingly well respected Adafruit, and similar questions have come up on this board in the past.
Per scruss's comment above it may also be possible to disable the USB controller in software, but doing so might disable your ethernet controller. If you do this after removing your physical sockets you will no longer be able to provide any inputs to the Pi (no USB WiFi, no ethernet, no keyboard, etc.) so be careful. Plan out the steps required beforehand and make sure you've always left yourself a way to issue commands.
A common solution to the removable media issue is to use epoxy to either seal up a socket, glue something into a socket (USB stick, SD card, etc.), or encapsulate an entire board (also known as potting). This works well and is widely used in industry. If you plan on trying this yourself, be aware that epoxy is generally a two part formula which, when mixed, causes an exothermic reaction. It gets hot. If your ambient temperature is already high this additional heat can push your components over their tolerance limits. Further to this, the maximum temperature reached during curing is the epoxy's 'set' point. If the epoxy cures fully and then is reheated beyond the curing temperature it will de-solidify until it's cooled again. Test everything very carefully before trying to sell it to anyone.
Finally, epoxy shrinks and hardens as it cures. The force exerted by the epoxy on the components can be enough to pull them off their traces. As, at that point, they're under a layer of nigh-on-invincible plasticky stuff, that results in a dead board. It may be practical to apply and cure a layer of (non-conductive, non-corrosive) silicone before applying the epoxy, as the silicone isn't quite so grabby.
There are many formulations of epoxy which have been developed specifically to have low-shrinkage, low conductivity, and lower viscosity to allow them to flow more easily into small spaces on PCBs. Make sure to read up thoroughly on the properties of whatever you choose.