Fast image array processing with Pi NoIR camera in Python using picamera module

Question

I'm working on a project with a RasPi in which I have to find a bright spot in an image (the bright spot is fire). I've written a function (code below) which captures images to RGB arrays using the PiRGBArray method of the PiCamera module. I need to compare two images to perform a crude background subtract (to reduce any ambient light), so I keep three images in a queue and return the image difference of the first and last every time a new image is taken. I am currently able to capture at about 4-6 frames per second.

When a fire is detected (by an external circuit) a GPIO pin triggers a callback function which reads the global FireImage and finds the bright spot in it. Because of my current speed, I have to ask the callback to wait for one image cycle so that there will actually be flame in the frame. If I could speed up the frame capture, this wouldn't have to be the case.

My question is, is there a faster way to gather the images? Would it be possible to use PiCamera's CircularIO() method to store frames? If so, how would I extract the first and last frame from the io buffer for comparison? Is there anything glaringly stupid that I'm doing to slow myself down?

Thanks for any help!

Camera settings:

framerate: 30fps

resolution: 160X120

import picamera, picamera.array
from collections import deque
from scipy import average

def pictureQueue(res,bright,con,fps):
    global FireImage
    imqueue = deque([])
    with picamera.PiCamera() as cam:
        cam.resolution = res
        cam.brightness = bright
        cam.contrast = con
        cam.framerate = fps
        with picamera.array.PiRGBArray(cam) as output:
            while True:
                cam.capture(output,'rgb',use_video_port = True)
                imqueue.append(average(output.array,-1))
                if len(imqueue) > 2:
                    FireImage = abs(imqueue[-1] - imqueue[0])
                    imqueue.popleft()
                output.truncate(0)

UPDATE:

I don't know what I was thinking, but somehow I forgot about capture_continuous. I just got up to ~32 fps by reducing the resolution to 50X50 (probably all I need for the current application) and taking the full RGB array (instead of averaging my own grayscaled image). I also store the whole imqueue variable as a global and take the difference of the images in the callback function. I left the code at work, so will update in the morning. Any suggestions on further increasing the speed are still welcome, but 30 fps is good enough for me at the moment. I will post it as an answer if no comments by the end of tomorrow.

Follow up question for anyone that might see this: If I increase the camera's framerate to 90 fps, I see an increase in captures to ~37 fps (understandable, since there is probably no way I can cycle through that much information that quickly with the Pi). Where are the other 53 frames going every second? Are they filling up a buffer somewhere just waiting to explode my RAM? Am I only seeing the first 37 frames pass through my queue, or do I just miss some frames along the way?

Answer 1

Apparently, I got a little sidetracked and didn't post this "by the end of tomorrow". Anyway, here is the solution I ended up with. Still not perfect by any means, but does at least provide quick, on-demand access to the camera and its most recent captures.

import picamera, picamera.array
from collections import deque()
with picamera.PiCamera() as cam:
    global ImQueue
    ImQueue = deque()
    with picamera.array.PiRGBArray(cam) as output:
        t1 = time.time()
        count = 0
        for im in cam.capture_continuous(output,'rgb',use_video_port = True):
            ImQueue.append(output.array)
            if len(ImQueue) > 3:
                ImQueue.popleft()
            output.truncate(0)
            count += 1
            if time.time() - t1 > 10:
                break
        print 'Captured %d images in %f seconds' % (count,time.time()-t1)

Next steps:

• find a way to avoid global r/w
• test for speed on new RaspPi boards (multicore) with a core dedicated to picture acquisition
• explore other options for camera interface (C/C++)

Hope this helps someone.

UPDATE (October 2016)

If you are using OpenCV, drivers are available now (since beginning of this year) to directly access the Pi's camera module using the OpenCV API. In other words, just use cv2.VideoCapture() and be done with it. Tested on a Pi 3 I got upwards of 85 fps.