Getting extremely low fps on Rpi 3 while image processing

Question

I am working on a project that requires image processing to detect a traffic light and take action based on its status (on or off). I initially developed the code on my PC, using OpenCV and Python, it works just fine.

However when I moved over my Raspberry Pi 3, I was using the Pi Camera module, I followed a tutorial on the PyImageSearch blog by Adrian Rosebrock, who publishes great tutorials, anyways, I was able to get a video stream with decent FPS on my RPi.

However when I moved my main image processing code over to the RPi, I had to modify it to work with a for loop instead of my original while loop. When I ran the code, it gave me an error

(Thresholding Control:1356): Glib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed

But the script still ran anyway, giving me moderately low FPS, and while all this was happening, the CPU usage on the Pi was only 25-28%!!
Considering that the Raspberry Pi 3 has a separate graphics processor, a Quad Core processor @1.2 GHz and enough RAM, I find this very strange.

I looked up google on the error, found nothing. I then found another PyImageSearch tutorial over here. It explains threading to increase the FPS of the for loop pipeline, but I have no idea how to use it in my script.

How can I increase my FPS on the Pi? Is it threading? Or something else altogether?

Here's my code, modified to work on the Raspberry Pi 3 Model B.

from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2 
import numpy as np
def f(x): return
cv2.namedWindow('Thresholding Control')
cv2.namedWindow('Precise Controls')

cv2.createTrackbar('High H','Thresholding Control',179,179, f)
cv2.createTrackbar('Low H','Thresholding Control',0,179, f)
cv2.createTrackbar('High S','Thresholding Control',255,255, f)
cv2.createTrackbar('Low S','Thresholding Control',0,255, f)
cv2.createTrackbar('High V','Thresholding Control',255,255, f)
cv2.createTrackbar('Low V','Thresholding Control',0,255, f)
cv2.createTrackbar('Guassian Blur','Precise Controls',0,99, f)
cv2.createTrackbar('Intensity Threshold','Precise Controls',0,255,f)

camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 32
rawCapture = PiRGBArray(camera, size=(640, 480))

time.sleep(0.1)

for frame in camera.capture_continuous(rawCapture, format="bgr",use_video_port=True):

   image = frame.array
   HSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
   highH = cv2.getTrackbarPos('High H','Thresholding Control')
   lowH = cv2.getTrackbarPos('Low H','Thresholding Control')
   highS = cv2.getTrackbarPos('High S','Thresholding Control')
   lowS = cv2.getTrackbarPos('Low S','Thresholding Control')
   highV = cv2.getTrackbarPos('High V','Thresholding Control')
   lowV = cv2.getTrackbarPos('Low V','Thresholding Control')
   thresh = cv2.inRange( HSV, (lowH, lowS, lowV), (highH, highS, highV))
   blurVal = cv2.getTrackbarPos('Guassian Blur','Precise Controls')
   intensityVal = cv2.getTrackbarPos('Intensity Threshold','Precise Controls')
   if(blurVal%2==0):
       blurVal=blurVal+1
   thresh_smooth = cv2.GaussianBlur(thresh, (blurVal, blurVal), 0)
   Defining the kernel to be used for Morphological ops.
   kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
   # Applying Opening and Closing.
   thresh_smooth = cv2.morphologyEx(thresh_smooth,cv2.MORPH_OPEN,kernel)
   thresh_smooth = cv2.morphologyEx(thresh_smooth, cv2.MORPH_CLOSE, kernel)

   (minVal, maxVal, minLoc, maxLoc) = cv2.minMaxLoc(thresh_smooth)
   nonZero = cv2.countNonZero(thresh_smooth)
   if(maxLoc !=(0,0)): #Light is on
   cv2.putText(image,"Light on", (320,240), cv2.FONT_HERSHEY_SIMPLEX, 2, 255)
   cv2.circle(thresh_smooth, maxLoc, 50, (255, 0, 0), 2)
   cv2.circle(image, maxLoc, 50, (255, 0, 0), 2)

   elif (maxVal<intensityVal and maxLoc!=(0,0)): #light is off     
        cv2.circle(image, maxLoc, 5, (0, 0, 255), 2)
        cv2.putText(image,"Light Off", (320,240), cv2.FONT_HERSHEY_SIMPLEX, 2, 255)

   elif(nonZero==0): #Light is off
       cv2.circle(image, (0,0), 50, (0, 0, 255), 2)
       cv2.putText(image,"Light Off", (320,240), cv2.FONT_HERSHEY_SIMPLEX, 2, 255)

   cv2.imshow("BGR", image)
   cv2.imshow("Thresholded", thresh_smooth)
   print maxLoc 

   rawCapture.truncate(0)

   if cv2.waitKey(1) & 0xFF == ord('q'):
      break

   cv2.destroyAllWindows()

EDIT:
Could this be happening because of insufficient power supply? I mean my usual 5v 2A PSU blew up on day, luckily the Pi survived, I found a mobile charger, 5v 2A and I am using it, but I see the rainbow box, signifying undervoltage coming up repeatedly.

Answer 1

First off, how are you determining that the CPU is only at 25-28%? htop is what I would use as it shows the usage of each core separately. sudo apt-get install htop, then htop to run it, q to quit. Second, tiny little devices like the Raspberry Pi have much less processing power than what we get used to on our desktops. Your desktop probably has more than enough processing power (3-4 GHz) to run your code decently without threading, whereas the Raspberry Pi (1.2 GHz) may get burdened down without threading your code. Threading will allow your code to dispatch different tasks within your code to different cores, whereas the absence of threading will have your code run on a single-core only. For heavy tasks, threading makes a HUGE difference, so learn how to master it.

Answer 2

Have you tried bumping up the GPU mem to your camera?

sudo nano /boot/config.txt

Increase whatever you have for 'gpu_mem'

http://elinux.org/RPiconfig