I'm thinking about building a computer cluster with a few Raspberry Pis. I've seen these directions: http://www.southampton.ac.uk/~sjc/raspberrypi/pi_supercomputer_southampton_web.pdf and one thing I'm wondering is how are the Raspberry Pis connected? My basic research has told me that I need an ethernet switch, but I was hoping someone could explain that a bit more.
Add a comment
|
2 Answers
An ethernet switch is indeed what you all need, besides a bunch of ethernet cables. If you are planning on a bigger cluster, like maybe with 20+ nodes, you should use those 8-port gigabit switches to form a subcluster for every 5-7 nodes; and then use a bigger gigabit switch to connect all those subclusters. This is to avoid network congestion caused by slow or congested links (A gigabit link cannot carry 20 100Mbps links at once.) The software is more straightforward as you can just preconfigure all software on one Pi and copy images over (don't forget to change the IP addresses!)
-
Thank you! I'm planning on 3-4 nodes for now. If I search "Ethernet switch" on amazon is that enough or is there anything to look out for? Apr 1, 2014 at 18:00
-
@ohblahitsme A good switch can give you good throughput and hence higher efficiency in message passing. For a 3-4 node cluster you can go with a Linksys 8-port gigabit Ethernet switch. This switch will allow you to observe your current cluster and expand it into a subcluster of your future bigger cluster. Apr 4, 2014 at 7:11
You do not need to use an Ethernet switch to create a RPi cluster -- you can use the wireless Ethernet connection. In fact, you can start with one Raspberry Pi3B+, since the ARMv7 processor has 4 cores. Node has a module called Cluster -- it works exactly the same way that MPI4Py works, except it works on a single processor, which MPI4Py also supports.
I followed the instructions for creating a RPi3B+ cluster -- it is 5 nodes and I have the same setup that everyone is referring to -- a 5 port switch and five RPi3B+ with MPI4Py. It is neat, however -- I use them for other things, too. They all run BOINC and process SETI@HOME data, as well as running Node based HTTP servers on the wired Ethernet connection.
I recommend starting with one Raspberry Pi3B+ because the basic process does not change when adding nodes. You will be money ahead and you can still learn at the same level required for supercomputing. If you write a RESTful style server -- you can have all your RPis and other devices using a standard method of command and control and you can expose that as a data acquisition system.
I don't use cloud or DDNS, my system sits in my bedroom and is a subdomain that I own. it is right here :
http://stations.jinzai-studio.net/index.html
If you scroll to the bottom of the page, you will see a 'sitrep' in XML from each of the 4 nodes that reports some information about them. Next, I will connect some sensors to them and provide a means to configure and get data. The master node is what is sending the HTML page. Again, all 5 are still processing BOINC data and I can spawn parallel tasks to them on the wired connection, or even the wireless one. The entire system is being developed in Visual Studio Code and using SSH to publish as well as monitor.
