Very high camera latency with OpenCV

Question

I have an USB camera and the official RPI camera. Both cameras deliver images with a delay of around 5 seconds with OpenCV 3.0 (VideoCapture) on my Raspberry Pi 2. When I run the same application on my Windows computer, I do not have any notable latency.

Is there any way to get rid of this delay?

Update

I'm using OpenCV with Java. To get access to the Raspicam I compiled OpenCV with V4L2 drivers.

Here's a simplified version of my code.

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.videoio.VideoCapture;
import org.opencv.videoio.Videoio;

import javax.swing.*;
import java.awt.image.BufferedImage;

public class Test {

    static {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    }

    public static void main(String[] args) {
        VideoCapture capture = new VideoCapture(0);
        Mat camImage = new Mat();
        JFrame frame = new JFrame();
        JLabel label = new JLabel();
        frame.add(label);
        frame.setVisible(true);
        frame.pack();
        int frameWidth = 320;
        int frameHeight = 240;
        frame.setSize(frameWidth, frameHeight);
        capture.set(Videoio.CAP_PROP_FRAME_WIDTH, frameWidth);
        capture.set(Videoio.CAP_PROP_FRAME_HEIGHT, frameHeight);

        if (capture.isOpened()) {
            while (true) {
                capture.read(camImage);
                if (!camImage.empty()) {
                    label.setIcon(new ImageIcon(convertMatToBufferedImage(camImage)));
                } else {
                    System.out.println("-- Frame not captured --");
                    break;
                }
            }
        } else {
            System.out.println("Couldn't open capture.");
        }
    }

    public static BufferedImage convertMatToBufferedImage(Mat in) {
        int width = in.width();
        int height = in.height();
        BufferedImage out;
        byte[] data = new byte[width * height * (int) in.elemSize()];
        int type;
        in.get(0, 0, data);

        if (in.channels() == 1) {
            type = BufferedImage.TYPE_BYTE_GRAY;
        } else {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }

        out = new BufferedImage(width, height, type);
        out.getRaster().setDataElements(0, 0, width, height, data);

        return out;
    }

}

Answer 1

While you haven't said what the "Windows computer" is, presuming it's not a tiny low power arm box (like, say, the Raspberry Pi...), a very very significant difference in performance is hardly surprising. As in, several orders of magnitude, not just one, as people might guess by looking at processor Hz. This is in part an ARM vs. x86 issue. How does your phone honestly compare to your laptop?

The primary heavy lifting I do on the pi is compiling (C/C++), and something relatively light weight (which will compile instantaneous on my desktop) can easily take 5+ seconds on the pi using the exact same compiler and OS. I think number crunching and image processing falls into the same general category. I'm sure you've noticed this with the java bytecode compiler too.

Have you analyzed specifically where the latency is generated? If it's in here:

out.getRaster().setDataElements(0, 0, width, height, data);

as much as here:

capture.read(camImage);

Then it's really about the processor speed (I'm making a crude guess about what's up). If it is almost entirely in the capture, then it is more mysterious and could be something to do with hardware drivers.

The easiest way to check this would be to comment stuff out.