0

I'm about to build a system that is using different types of programs like Python, php, Javascript and so on. I'm using a Raspberry Pi 3 with Raspbian Jessie.

Basically all those types of programs have to react on one common status(file?) which should be stored independently from those programs. There will not be more than 8 status.

My first thought was a textfile (.txt). But the status should be read every 2-5 seconds so that there will be a huge system load to open the file and reading it all the time.

My question is: What kind of construct would fit for this? Is there maybe a Linux functionality to use?

  • Have you thought about a messaging service like rabbitmq? – Steve Robillard Jul 13 '16 at 6:28
  • Thank you for this nice approach, Steve! I will give it a try. For my tiny solution with max 8 status it is maybe a little bit over engineered I guess. I'm looking for something streamlined like a status register to set/reset with interrupts when status is changing. – Kay Gladen Jul 13 '16 at 6:53
  • 2
    Your premise is wrong. There will not be a huge load in opening and reading a text file from multiple processes every few seconds. I doubt the load will be measurable. – joan Jul 13 '16 at 10:21
1

But the status should be read every 2-5 seconds so that there will be a huge system load to open the file and reading it all the time.

I think 8 processes doing this every two seconds will amount to something more like an "infinitesimal" load on the system. It depends a bit on how much they are reading from the file, but the bottleneck is (on the surface) I/O and so even 8 processes busy constantly reading from the SD card won't consume significant processor time; they all have to share the same bottleneck, and even at the theoretical maximum (25 MB/s) data coming from the SD card comes much slower than it does from, e.g., main memory, or could be run through the processor -- although this is also dependent on how much processing you have to do to it.

I'm guessing the volume of data is actually going to be small, so that won't matter much either. If it is say, < 1 kB, then that is 4 kB/s, and if you have a slow SD card (say 8 MB/s) that's still only 0.0005% of the available bandwidth. The SD card uses DMA, so the CPU will be mostly free to do other things (such as processing read data), or nothing during that time.

However, presuming you haven't used 100% of your free RAM, if this data is being read and written that often and it is not hundreds of MB in size, it will end up in the page cache, so you actually won't be reading and writing constantly from the SD card. That will just be synced periodically by the system.

Of course, if you want to ensure they are using RAM and not slower storage, and the information does not have to persist across reboots, put the file in /tmp or another tmpfs backed location (you can see the existing ones with mount | grep tmpfs). In case that is not clear, those are in RAM to start with.

Caveat

Your one much more real issue will be concurrent access, so if you use files you will should implement file locking. On linux this is voluntary, meaning all parties need to implement it to work. The native system call is flock() (see man 2 flock; the 2 is important) which will have a wrapper version in most higher level languages; certainly python and php, anyway.

What [other] kinds of construct would fit for this?

There are lots of mechanisms for IPC; in this case shared memory segments jump to mind which may provide a performance advantage over file in memory, although again, since the load here is so tiny I doubt that will be meaningful. You might as well just use files (again, preferably ones with a tmpfs store, and make sure to use appropriate locking). Depending on the nature of what you are doing, you could also use a client-server style system with UNIX local sockets (which are supposedly a bit faster than inet sockets, but again, the volume here is I suspect so small it doesn't matter).

If it is just to indication a binary true/false status, you could use SysV semaphores, see man sem_overview (I don't know what support there will be for this in js or php though).

The OS has various higher level messaging systems (e.g., d-bus) that may or may not be suited to what you are doing, but the more you get it that kind of thing the more complex it becomes and the sketchier support in some languages, I think particularly js, is going to become.

  • Thank you goldilocks for your huge answer. Very helpful. I think about another approach that fit's to my tiny solution: probably I can use the GPIO status to store my flags. It's easy to setup and Linux takes care about the locks. – Kay Gladen Jul 13 '16 at 11:38
  • 1
    If it's just flags, SysV semaphores is another option, see man sem_overview, although I dunno about js and php with that. – goldilocks Jul 13 '16 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.