2

Although the camera framerate has been set to 24, 24 frames are being captured at 3 seconds. Wondering where I am going wrong?? I need to convert the stream to an OpenCV object to do further image processing. Should I be using other capturing methods?

with picamera.PiCamera() as camera:
        camera.resolution = (640, 480)
        camera.framerate = 24
        camera.start_recording('timed.h264')
        stream = io.BytesIO()
        framecount = 0
        camera.capture(stream, format='jpeg', use_video_port=True)
        starttime = time.time()
        while True:
            frame = np.fromstring(stream.getvalue(), dtype=np.uint8)
            stream.seek(0)
            if frame is not None:
                framecount += 1
            if framecount % 24 == 0:
                endtime = time.time()
                diff = endtime - starttime
                print diff
                starttime = endtime
2

The script you've posted is starting an H.264 recording (which will be running at about 24fps) and then goes on to perform simultaneous JPEG captures and load the raw data into a numpy array (without decoding, so it's not exactly image data at this point). The JPEG captures aren't going to run at anywhere near 24fps partly because Python will be quite busy writing out H.264 data to the SD card in a background thread. However, even removing the background recording, I strongly doubt you'll be able to capture and process frames in OpenCV at 24fps (in Python, or for that matter in any language). Remember that the processing is happening on the CPU, and Pi's CPU is tiny compared to its GPU (where all the camera capture and encoding occurs).

So, forget 24fps processing in OpenCV: it's not going to happen. However, you can manage a more realistic 1-2fps or so by avoiding the JPEG decoding and just giving OpenCV unencoded BGR data straight from the camera. The following script demonstrates capturing in this way and having OpenCV convert the resulting data to HSV before printing some average values (rather basic, but this is just for demo purposes):

import picamera
import picamera.array
import time
import cv2
import numpy as np

with picamera.PiCamera() as camera:
    camera.resolution = (640, 480)
    camera.framerate = 24
    time.sleep(2) # AGC warm-up time
    start = time.time()
    for i in range(24):
        with picamera.array.PiRGBArray(camera) as stream:
            camera.capture(stream, 'bgr', use_video_port=True)
            hsv = cv2.cvtColor(stream.array, cv2.COLOR_BGR2HSV)
            print 'Average H: %.2f, S: %.2f, V: %.2f' % (
                np.average(hsv[..., 0]),
                np.average(hsv[..., 1]),
                np.average(hsv[..., 2]),
                )
    finish = time.time()
    print 'Processed 24 frames in %d seconds at %.2ffps' % (
        finish - start, 24.0 / (finish - start))

With this script I get about 2.5fps on my overclocked Pi. A little faster than 1-2fps, but then it's not really doing much! If you really need 24fps processing in OpenCV your only realistic option is to pipe image or video data from the Pi over a network to a more powerful machine and do the processing there.

  • Any idea comments on more powerful machine, to get 24 or even 50 frame per second, like BBB (Beagle Bone Black), board using STM32 (may be series 3 or 4 higher end) or even Intel CPU? – EEd Aug 13 '14 at 3:24
  • 1
    I would assume a full-blown PC; I'd be surprised if a system like a BBB could handle 24fps, despite its slightly beefier CPU. Also depends on what you mean by "processing" - my example of an average above is trivial whilst things like object recognition are seriously computation intense. – Dave Jones Aug 15 '14 at 11:55
  • Would saving frames to array first then doing your opencv cvtColor function on each frame in the resulting array improve performance? – bakalolo Jun 27 '18 at 1:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.