Concurrent task handling on Raspberry Pi specific applications

Question

I'm hoping that this is more relevant to the RPi exchange as a lot of my key concerns relate to issues with GPIO/PWM/serial bus usage with concurrency.

I'm currently writing software for a robotics application that uses a Raspberry Pi Zero (single core CPU) and which will use Python to manage multiple tasks at once. Some of these tasks can be asynchronous and need to occur either regularly or constantly; whereas others will have very strict sequencing (robotics and imaging, mainly).

Examples of the asynchronous/continuous tasks would include:

• TCP socket constantly listening for commands from networked computer (vital this is always listening and available)

• UDP socket to send telemetry data

• Regularly communicating with and sending heartbeat data to connected microcontrollers via UART

• TCP again to send large volumes of imaging data on regular intervals without blocking other processes

Synchronized tasks that must occur in order would include:

• Polling serial and I2C sensor arrays and logging data

• Acting on sensor data in turn (e.g. commanding actuators)

• Following a sequence of commanded events

• Sending event telemetry via UDP if something specific occurs

• Using Pi camera for imaging/video capture (particularly worried about this blocking)

• Logging data to SD cards, both using logger (thread-safe, not sure about asyncio-safe) and camera data

So, my question is: are threads/asyncio/green threads appropriate solutions for implementing this, or am I going to run into major issues, especially with respect to using GPIO/PWM/I2C etc?

I was first thinking of using threads for all the asynchronous things that can occur simultaneously, especially for the always-listening TCP socket, but I'm wary of race conditions and communicating between threads. I imagine found thread-safe queues would alleviate most worries, though. ThreadPoolExecutor could also help here.

I've been exploring the asyncio library recently, too, which might suit my needs - especially if I can dynamically add and remove tasks to this in response to TCP commands - but I'm not sure if I'm getting far too complicated with this when a couple of threads might work? The asyncio library looks great and I'm keen to learn it, but I only have a few months for this project and it looks like complexity could spiral. I am not certain whether it is suitable for tasks involving GPIO/PWM especially - also nervous about implementation with UART communication to a microcontroller.

Finally, I've looked into green threads as well, though it looks as though they come with their own pitfalls and traps.

I greatly appreciate any advice on this, thank you.

Answer 1

Some of these tasks can be asynchronous and need to occur either regularly or constantly; whereas others will have very strict sequencing (robotics and imaging, mainly).

Regularly is not something belonging to time-shared systems. If you want to get regular, you don't want an OS - or you want to run an RTOS. The choice of language is then basically irrelevant; you will NOT get your accuracy by design.

Now, most of the stuff you mention can be driven in a way it introduces very minimal lag and it is possible to build a program which does all those things in a single thread.

It is extremely hard to do all those things without ever blocking anything as 'blocking' is the basic construct of thread synchronization and if you don't want to do it your only option is being "lock-free". Not a thing you would do for much more than a data structure such as a queue or a set.

Keep in mind for example listen isn't realtime by any means as you have no control on the WiFi you'll be using to connect to the Zero! Waiting half a second for listen to be processed (while you do something else) is not going to change a thing.

I am not certain whether it is suitable for tasks involving GPIO/PWM especially - also nervous about implementation with UART communication to a microcontroller. Similar applies to SPI and I2C - they are basically not an issue.

UARTs are asynchronous by design. Your micro depending on its firmware might time out (but it's more a theorical possibility than real issue); the Pi can guarantee at hardware level the signal timings for each UART frame will be just right. Similar guarantees hold for GPIOs and to a certain degree to PWM.