0

I have quite a complex python program, and i would like to run one instance of it, on an environment, that would be managed by several Raspberries altogether, in a transparent fashion. I wish to benefit from:

  • Load balancing
  • Handling of the death of a node (one of the Raspberries) without impacting the program

My question: is it possible? Does a framework exists for that? What kind of theory / paradigm am I talking about?

It looks like the closest paradigm is a local cloud. I definitely cannot cut my algorithm into small computable parts to load on a cluster. I would like the hardware layer to be transparent at the code level.

Improve this question
3
  • So basically you want to share your Raspberries' computing power so it looks like a big powerful computer to your python program? And also handle the suppression (and/or addition?) of Raspberries at any time without crashing your program?
    – Anthony Vanden Eede
    Dec 14, 2016 at 0:18
  • Precisely Anthony ! In a layer between algo and hardware.
    – avermeir
    Dec 14, 2016 at 23:18
  • Would something like XML-RPC be more to your liking ? You keep talking about passing opaque "objects" around ?
    – flakeshake
    Dec 18, 2016 at 14:43

1 Answer 1

Reset to default
2

Load Balancing is the distribution of work over a series of devices in a client server model. It's totally unrelated to what you're trying to accomplish.

A local cloud, aka fog computing is simply the management of a web application that runs on your local intranet. It is also completely unrelated to what you're trying to accomplish.

That being said, what you're talking about is nonsense. Cluster computing isn't as simple as plugging multiple devices together to get a super computer. The concept you're looking for is parallel computing.

I definitely cannot cut my algo into small computable parts to load on a cluster

If you cannot parallelize your algorithm into discrete parts that can operate independently, you cannot even take advantage of multiple CPU cores, let alone a cluster.

Improve this answer
3
  • Hi Jacob, thank you for the answer ! Before asking the question about the existence of a paradigm of framework, I have been implementing a peer to peer network with those rasps, that is transparent for the algo. Objects interact without knowing if their reciever is on another rasp or on the same one. And one of the object actively looks for the CPU usage of raspberries, to maybe rearrange objects in a more optimal way. Isnt this like a primitive form of load balancing ?
    – avermeir
    Dec 13, 2016 at 23:44
  • "If you cannot parallelize your algorithm into discrete parts that can operate independently" this is exactly what i want to avoid, i wondered if there was a paradigm, with maybe an existing library, that acts at a lower level, between algo and physical and does the trick
    – avermeir
    Dec 13, 2016 at 23:47
  • 1
    @avermeir: It sounds like you're asking for magic. If you can't break up the algorithm to run in multiple places, then there is literally nothing that can help you make it run faster unless you want to write it in a faster language like C/C++/Go or maybe Java.
    – Jacobm001
    Dec 13, 2016 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.