1

I have been working on a project where I am tracking an infrared LED (blob) and obtaining its coordinates to move the mouse cursor in raspbian OS.

I'm using SimpleCV and raspberry pi csi camera with an additional visible light filter, so the camera only sees infrared light.

I have installed v4l2 for accessing camera in SimpleCV.

Here is my code (only till blob finding, I'm stuck with another problem, never mind)

from SimpleCV import*
cam = Camera(0, {"width":640, "height":480})
while True:
   img = cam.getImage().grayscale().threshold(45)
   blob = img.findBlobs()
   print blob
   img.show()

When I run this, the video is very very slow, like only 2 or 3 fps.

Is there any way to speed this up, like in obtaining the data from the camera?

3

I'm not familiar with SimpleCV, but have a fair amount of experience with the Pi's camera module and OpenCV so this answer is an "educated guess":

My suspicion is that the camera is perfectly capable of delivering more than 2 or 3fps via v4l2 to SimpleCV. The problem is everything after the delivery of the frame (i.e. the processing you're doing with SimpleCV). This is all being done on the Pi's CPU (as opposed to the capture which is running on the GPU). The Pi's CPU is very small and one can't expect a great deal from it, processing-wise. Unfortunately blob finding is a computationally expensive task and I suspect the vast majority of the time in your loop is being spent on this part (not the capture).

Firstly, let's find out if my suspicion is correct. The following script is a variant on yours which measures the time to perform each step in your processing:

from SimpleCV import *
from time import time

cam = Camera(0, {'width': 640, 'height': 480})
capture_times = []
grayscale_times = []
threshold_times = []
find_times = []
show_times = []
loop_times = []

for i in range(100):
    loop_start = start = time()
    img = cam.getImage()
    capture_times.append(time() - start)
    start = time()
    img = img.grayscale()
    grayscale_times.append(time() - start)
    start = time()
    img = img.threshold(45)
    threshold_times.append(time() - start)
    start = time()
    blob = img.findBlobs()
    find_times.append(time() - start)
    start = time()
    img.show()
    show_times.append(time() - start)
    loop_times.append(time() - loop_start)

def mean(l):
    return sum(l) / len(l)

mean_loop_time = mean(loop_times)
mean_capture_time = mean(capture_times)
mean_grayscale_time = mean(grayscale_times)
mean_threshold_time = mean(threshold_times)
mean_find_time = mean(find_times)
mean_show_time = mean(show_times)

print('Average loop time: %.3fs (%.2ffps)' % (mean_loop_time, 1 / mean_loop_time))
print('Average capture time: %.3fs (%.1f%%)' % (mean_capture_time, mean_capture_time * 100 / mean_loop_time))
print('Average grayscale time: %.3fs (%.1f%%)' % (mean_grayscale_time, mean_grayscale_time * 100 / mean_loop_time))
print('Average threshold time: %.3fs (%.1f%%)' % (mean_threshold_time, mean_threshold_time * 100 / mean_loop_time))
print('Average findBlobs time: %.3fs (%.1f%%)' % (mean_find_time, mean_find_time * 100 / mean_loop_time))
print('Average display time: %.3fs (%.1f%%)' % (mean_show_time, mean_show_time * 100 / mean_loop_time))

The output from this on my Pi (which is a Pi 1 model B overclocked to 900Mhz, "Medium") is as follows:

Average loop time: 0.895s (1.12fps)
Average capture time: 0.059s (6.6%)
Average grayscale time: 0.056s (6.3%)
Average threshold time: 0.068s (7.6%)
Average findBlobs time: 0.429s (47.9%)
Average display time: 0.283s (31.6%)

So, I'm getting a little over 1fps, but we can immediately see that the capture time is a tiny proportion of the loop's overall time, and the vast majority is being spent in findBlobs and displaying the preview. If we simply cut out the preview display I get the following results:

Average loop time: 0.463s (2.16fps)
Average capture time: 0.042s (9.0%)
Average grayscale time: 0.048s (10.4%)
Average threshold time: 0.047s (10.1%)
Average findBlobs time: 0.326s (70.4%)

Obviously now findBlobs takes up an even greater proportion of the loop's time but at least we're up to 2fps. So, as suspected, the problem isn't the capture time but the post processing time. The only realistic way to get this faster is run it on a faster machine (e.g. a Pi 2 or a full PC if you ship the frames over a network to it), write a version of findBlobs that utilizes the GPU (this is hard), or find another (faster) algorithm for achieving the result you're looking for (unfortunately I don't know enough about computer vision to point you in a good direction here; it may be worth asking this as a separate question in a more general forum).

| improve this answer | |
  • The pi camera module does not work the same as the USB camera and 'VideoCapture cap(0)' doesn't work with pi camera module. I want to ask that how do you use pi camera with OpenCV? I am using this wrapper to get frames from pi camera module to use them with OpenCV. github.com/samarth-robo/RPi-OpenCV – ffttyy Jan 2 '16 at 17:47
1

I would suspect that the Pi is choking on its processor speed. Try stream the video feed off the Pi as a H.264 stream (encoding done using GPU so it will not choke the CPU too much) and use a more powerful computer (preferably something with a multi-gigahertz processor, 64-bit OS and gigabytes of RAM) to run SimpleCV against it.

| improve this answer | |
0

Sadly not. There is nothing to do with the camera. In fact the Pi camera module is faster than a usb camera. The problem is the ARM core of RPi is not that powerful to handle the computing in object detection. If you are using RPi2 and openCV, then it's much better. You can just enable openMP feature to use the multi cores. That will speed up a lot.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.