0

I have an USB camera and the official RPI camera. Both cameras deliver images with a delay of around 5 seconds with OpenCV 3.0 (VideoCapture) on my Raspberry Pi 2. When I run the same application on my Windows computer, I do not have any notable latency.

Is there any way to get rid of this delay?

Update

I'm using OpenCV with Java. To get access to the Raspicam I compiled OpenCV with V4L2 drivers.

Here's a simplified version of my code.

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.videoio.VideoCapture;
import org.opencv.videoio.Videoio;

import javax.swing.*;
import java.awt.image.BufferedImage;

public class Test {

    static {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    }

    public static void main(String[] args) {
        VideoCapture capture = new VideoCapture(0);
        Mat camImage = new Mat();
        JFrame frame = new JFrame();
        JLabel label = new JLabel();
        frame.add(label);
        frame.setVisible(true);
        frame.pack();
        int frameWidth = 320;
        int frameHeight = 240;
        frame.setSize(frameWidth, frameHeight);
        capture.set(Videoio.CAP_PROP_FRAME_WIDTH, frameWidth);
        capture.set(Videoio.CAP_PROP_FRAME_HEIGHT, frameHeight);

        if (capture.isOpened()) {
            while (true) {
                capture.read(camImage);
                if (!camImage.empty()) {
                    label.setIcon(new ImageIcon(convertMatToBufferedImage(camImage)));
                } else {
                    System.out.println("-- Frame not captured --");
                    break;
                }
            }
        } else {
            System.out.println("Couldn't open capture.");
        }
    }

    public static BufferedImage convertMatToBufferedImage(Mat in) {
        int width = in.width();
        int height = in.height();
        BufferedImage out;
        byte[] data = new byte[width * height * (int) in.elemSize()];
        int type;
        in.get(0, 0, data);

        if (in.channels() == 1) {
            type = BufferedImage.TYPE_BYTE_GRAY;
        } else {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }

        out = new BufferedImage(width, height, type);
        out.getRaster().setDataElements(0, 0, width, height, data);

        return out;
    }

}
7
  • What's the resolution you're dealing with ? If it's higher, try lowering it to 640x480 or 320x240. Oct 25, 2015 at 9:55
  • I have 640x480 and already tried 320x240. It makes no difference.
    – android
    Oct 25, 2015 at 9:59
  • Can you share how exactly you're using it ? Code etc. Oct 25, 2015 at 10:01
  • I've updated my question with a simplified version of my code. With the simpler version, the latency got better (around 2 seconds), but it's still way too much.
    – android
    Oct 25, 2015 at 11:31
  • 1
    Oh, I guess I just found my problem. I'm updating the JLabel with every new image received from the camera. The RPI just can't keep up with so many GUI updates. When I update the GUI only every 200 ms, then I don't have a very high latency.
    – android
    Oct 25, 2015 at 12:13

1 Answer 1

1

While you haven't said what the "Windows computer" is, presuming it's not a tiny low power arm box (like, say, the Raspberry Pi...), a very very significant difference in performance is hardly surprising. As in, several orders of magnitude, not just one, as people might guess by looking at processor Hz. This is in part an ARM vs. x86 issue. How does your phone honestly compare to your laptop?

The primary heavy lifting I do on the pi is compiling (C/C++), and something relatively light weight (which will compile instantaneous on my desktop) can easily take 5+ seconds on the pi using the exact same compiler and OS. I think number crunching and image processing falls into the same general category. I'm sure you've noticed this with the java bytecode compiler too.

Have you analyzed specifically where the latency is generated? If it's in here:

out.getRaster().setDataElements(0, 0, width, height, data);

as much as here:

capture.read(camImage);

Then it's really about the processor speed (I'm making a crude guess about what's up). If it is almost entirely in the capture, then it is more mysterious and could be something to do with hardware drivers.

The easiest way to check this would be to comment stuff out.

1
  • 1
    Yes, you're right. The problem wasn't the capture.read() but the many GUI updates with every new image. That led to very high CPU (100% on one of the cores) and so to the high latency.
    – android
    Oct 25, 2015 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.