I have read in many places that increasing your swap file size, or making a big swap partition; can boost the memory of your system as temporary files can go there instead of in RAM. However, on Raspberry Pis, it is advised against creating a swap on the SD card as it can wear it out - SDs weren't designed for RAM-style usage. It is also advised not to create swaps on SSDs or flash storage you plug in, as it can likewise wear it out.
A very good option I thought of is to create the swap on a high-RPM HDD, as they can be written to many more times over than an SSD. However, when building a cluster of 16 or so Pis that I am doing, it is really not cost effective or practical to buy a HDD drive, and SATA-USB adapter, for every Pi. It would also use tons of power.
So my question is: Can I create a swap file (or swap partition) on another machine? If possible, I would like to do a few per machine so, with 16 Pis, I only need 4 "Swap machines" (and then 4 hard drives and adapters are fine). Connect them all via a Gigabit switch, and... bob's your uncle? ???
Is this even possible? Can it be done? I am willing to spend lots of time on this, so it doesn't need to be easy. Secondly, will it actually degrade the performance of my Pi if it takes, say, 20ms to get a few KB of data and a second to get a MB?