I have quite a complex python program, and i would like to run one instance of it, on an environment, that would be managed by several Raspberries altogether, in a transparent fashion. I wish to benefit from:

  • Load balancing
  • Handling of the death of a node (one of the Raspberries) without impacting the program

My question: is it possible? Does a framework exists for that? What kind of theory / paradigm am I talking about?

It looks like the closest paradigm is a local cloud. I definitely cannot cut my algorithm into small computable parts to load on a cluster. I would like the hardware layer to be transparent at the code level.

  • So basically you want to share your Raspberries' computing power so it looks like a big powerful computer to your python program? And also handle the suppression (and/or addition?) of Raspberries at any time without crashing your program? – Anthony Vanden Eede Dec 14 '16 at 0:18
  • Precisely Anthony ! In a layer between algo and hardware. – avermeir Dec 14 '16 at 23:18
  • Would something like XML-RPC be more to your liking ? You keep talking about passing opaque "objects" around ? – flakeshake Dec 18 '16 at 14:43

Load Balancing is the distribution of work over a series of devices in a client server model. It's totally unrelated to what you're trying to accomplish.

A local cloud, aka fog computing is simply the management of a web application that runs on your local intranet. It is also completely unrelated to what you're trying to accomplish.

That being said, what you're talking about is nonsense. Cluster computing isn't as simple as plugging multiple devices together to get a super computer. The concept you're looking for is parallel computing.

I definitely cannot cut my algo into small computable parts to load on a cluster

If you cannot parallelize your algorithm into discrete parts that can operate independently, you cannot even take advantage of multiple CPU cores, let alone a cluster.

| improve this answer | |
  • Hi Jacob, thank you for the answer ! Before asking the question about the existence of a paradigm of framework, I have been implementing a peer to peer network with those rasps, that is transparent for the algo. Objects interact without knowing if their reciever is on another rasp or on the same one. And one of the object actively looks for the CPU usage of raspberries, to maybe rearrange objects in a more optimal way. Isnt this like a primitive form of load balancing ? – avermeir Dec 13 '16 at 23:44
  • "If you cannot parallelize your algorithm into discrete parts that can operate independently" this is exactly what i want to avoid, i wondered if there was a paradigm, with maybe an existing library, that acts at a lower level, between algo and physical and does the trick – avermeir Dec 13 '16 at 23:47
  • 1
    @avermeir: It sounds like you're asking for magic. If you can't break up the algorithm to run in multiple places, then there is literally nothing that can help you make it run faster unless you want to write it in a faster language like C/C++/Go or maybe Java. – Jacobm001 Dec 13 '16 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.