38

I have seen a lot of posts asking how to build a "supercomputer" or cluster with Raspberry Pis, but I have found very little with the pros and cons of actually making a network of Pis. I am interested in building my own small cluster, but I have a few questions.

Once clustered, do the RPIs act as a single virtual computer, or do you have individual control over all RPIs? Does one node have control of the others?

Is there anything that a RPI can do that is limited in a cluster? For instance, will I still have control over all individual GPIO ports?

How much faster does it make the pi? Lets say I make a 2 node cluster, over clocked to 1GHz each, will I essentially have a "dual core 1GHz" processor? Does the cluster need to be in Binary multiples? (ex. 1,2,4,8,16,32,64)

Will this complicate my web hosting running via NGINX for pi? Can the processing power of certain nodes be assigned specific tasks they handle? Such as node 1 will handle web service requests, Node 2 handles cloud storage, etc..

1 Answer 1

51

Some interesting questions. I think you may be slightly misunderstanding how the "supercomputers" built with Raspberry Pis work. They do not function as an automatic load sharing system. They are designed for something called parallel programming, where a complex task is broken down into pieces that can be performed simultaneously. The main Pi in the cluster (head node) is in charge of organizing the overall task, and each Pi in the cluster (client nodes) performs its allocated work and reports results back to the head node. Libraries such as OpenMPI are key for this.

Additionally, the Raspberry Pi clusters that have been built haven't been built as efficient computing devices. They have been built as development platforms. It's significantly cheaper to buy 32 Raspberry Pis than 32 desktop PCs. You can develop and test your parallel programming software much cheaper that way. The only alternative if you don't have the Pis or PCs is to request CPU time on an actual super computer. Depending on the size of the computer in question, doing so can cost money or have long wait times before the resources are available. Hence why people have built their own personal clusters. It only makes sense to run your application on the real computer when you'll know it works!

So, keeping this in mind and moving on to your specific questions.

In a proper cluster Pis do not act as a single virtual computer. You have loose control over each Pi in the form of a task or resource allocation system like the Sun Grid Engine. Tasks are allocated out to nodes as needed, and when the tasks are finished the resources of that node are freed up for new tasks.

In a cluster, since each Pi is running its own copy of Linux, each Pi would retain local control of its GPIO ports. I'm not sure how software would really use that in a parallel computing environment, but there you are.

A cluster doesn't really make a Pi faster, because they're not a single computer. What a you gain is simply the ability to do more at once. You're not limited to multiples of two. There is however a real practical upper limit to clustered computers depending on the tasks you run. Imagine if you're running a parallel calculation on many Pis that requires 200 steps. However, each next step requires the information that every Pi calculated on its previous step. So each Pi needs to receive data from every other Pi each iteration. Depending on the amount of time each iteration takes to calculate, you can end up spending more time sending data around than calculating. This is why most super computers use InfiniBand network. It's very fast, so you can do more calculating. The solution to this is to use fewer Pis but have them do more work each iteration, but that may not be possible depending on your algorithm. Hence the practical upper limit. (On Pis this would be particularly bad because the USB Ethernet is quite slow.)

So, on to a practical application, distributed web-hosting! You can take advantage of multiple Pis here, it's just not a regular cluster. Say you have 5 Pis. We'll call them GatewayPi, WebPi1, WebPi2, WebPi3, and DataPi. GatewayPi faces the internet, running Nginix to handle web requests, but doesn't do any processing. Instead what it does is Load balancing. It uses proxying and randomly forwards the incoming request to WebPi1, WebPi2, or WebPi3. We've just tripled the power of our web infrastructure, because we can now handle more requests simultaneously. What about DataPi? DataPi has attached a hard drive storing all our web-data and is running an NFS server. WebPiX mounts that NFS share so it has access to the data for processing. This way we only have to run backups in one location and can save on disk space.

What I've just described is essentially the model that large companies like Google and Facebook use, but scaled down to Raspberry Pi size. The only catch is, because of the costs of Pis, that doing this is essentially creating the same development platform (but for webserving) as a cluster of Pis is for parallel programming. Using multiple Pis, because of their relatively low computational ability soon looses out in performance to energy consumption as you head towards handling large amounts of data. But for just learning? They're perfect.

6
  • So if I'm working on a home project, for cheap web hosting of 2 websites, controlling cooling fans via GPIO, SSH, FTP, cloud storage, an home automation, It would be worth making a small cluster to distribute the tasks correct?
    – SuperAdmin
    Commented Apr 5, 2014 at 15:28
  • 4
    +1 I came to this question curious, and left with a tiny bit of education. Great answer.
    – jrhorn424
    Commented Apr 6, 2014 at 4:45
  • @iOSNoob, if you're the only person using it and you're only ever doing any one of those things at a time, it may not be worth a cluster even then, because the resources are allocated as necessary. If there's simply not enough RAM or you've got more load than a single person then it's probably worth it. I'd put things like fans and home automation on separate Pis simply for security. You really want 0 chance that someone will be messing around with stuff in your house... If you have to hop through an extra Pi to get to those systems all the better. That's an extra layer of security.
    – Fred
    Commented Apr 6, 2014 at 17:20
  • @jrhorn424 I'm glad you liked it! I recently attended a high performance (super) computing event so this stuff was fresh in my mind. Perfect timing to write it out.
    – Fred
    Commented Apr 6, 2014 at 17:21
  • 1
    For a concrete example of almost the exact web hosting architecture Fred described: pidramble.com (links out to code/instructions on GitHub where you can build you own with any flavor of Pi). Commented Mar 27, 2016 at 2:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.