Raspberry Pi 3 and V4L2 C++ Capture Code for JPEGs

Question

I have a Raspberry Pi 3 with the latest Raspbian Stretch, and the Sony 8MB official camera.

I can save a good looking JPEG with the following commands:

v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=somefile.jpg

I can list all formats with this command:

root@raspberrypi:/home/pi# v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YU12'
    Name        : Planar YUV 4:2:0
        Size: Stepwise 32x32 - 3280x2464 with step 2/2

    ...

    Index       : 3
    Type        : Video Capture
    Pixel Format: 'JPEG' (compressed)
    Name        : JFIF JPEG
        Size: Stepwise 32x32 - 3280x2464 with step 2/2

Now I want to translate the v4l2-ctl capture of a JPEG to C++ / Qt5 with libv4l2. My code is shown below and whenever I run it on the Pi, I get the following error:

root@raspberrypi:/home/pi# ./v4l2-test 
Begin Capture
VIDIOC_STREAMON: Operation not permitted

C++ Code:

#include <QCoreApplication>

// C++ / V4L2 Includes
#include <linux/videodev2.h>
#include <fcntl.h>              /* low-level i/o */
#include <unistd.h>
#include <errno.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <sys/ioctl.h>

// Debugging
#include <QDebug>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    qDebug() << "Begin Capture";

    int fd;
    if((fd = open("/dev/video0", O_RDWR)) < 0){
        perror("open");
        exit(1);
    }

    struct v4l2_capability cap;
    if(ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0){
        perror("VIDIOC_QUERYCAP");
        exit(1);
    }

    if(!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)){
        fprintf(stderr, "The device does not handle single-planar video capture.\n");
        exit(1);
    }

    struct v4l2_format format;
    format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    format.fmt.pix.pixelformat = V4L2_PIX_FMT_JPEG;
    format.fmt.pix.width = 2592;
    format.fmt.pix.height = 1944;

    if(ioctl(fd, VIDIOC_S_FMT, &format) < 0){
        perror("VIDIOC_S_FMT");
        exit(1);
    }

    struct v4l2_requestbuffers bufrequest;
    bufrequest.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    bufrequest.memory = V4L2_MEMORY_MMAP;
    bufrequest.count = 1;

    if(ioctl(fd, VIDIOC_REQBUFS, &bufrequest) < 0){
        perror("VIDIOC_REQBUFS");
        exit(1);
    }

    struct v4l2_buffer bufferinfo;
    memset(&bufferinfo, 0, sizeof(bufferinfo));

    bufferinfo.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    bufferinfo.memory = V4L2_MEMORY_MMAP;
    bufferinfo.index = 0;

    if(ioctl(fd, VIDIOC_QUERYBUF, &bufferinfo) < 0){
        perror("VIDIOC_QUERYBUF");
        exit(1);
    }

    void* buffer_start = mmap(
                NULL,
                bufferinfo.length,
                PROT_READ | PROT_WRITE,
                MAP_SHARED,
                fd,
                bufferinfo.m.offset
                );

    if(buffer_start == MAP_FAILED){
        perror("mmap");
        exit(1);
    }

    memset(buffer_start, 0, bufferinfo.length);

    // Activate streaming
    int type = bufferinfo.type;
    if(ioctl(fd, VIDIOC_STREAMON, &type) < 0){
        perror("VIDIOC_STREAMON");
        exit(1);
    }

    /* Here is where you typically start two loops:
     * - One which runs for as long as you want to
     *   capture frames (shoot the video).
     * - One which iterates over your buffers everytime. */

    bool capture_is_running = true;
    while(capture_is_running){
        for(int i = 0; i < bufrequest.count; i++){

            // Put the buffer in the incoming queue.
            if(ioctl(fd, VIDIOC_QBUF, &bufferinfo) < 0){
                perror("VIDIOC_QBUF");
                exit(1);
            }

            // The buffer's waiting in the outgoing queue.
            if(ioctl(fd, VIDIOC_DQBUF, &bufferinfo) < 0){
                perror("VIDIOC_QBUF");
                exit(1);
            }

        }

    }

    // Deactivate streaming
    if(ioctl(fd, VIDIOC_STREAMOFF, &type) < 0){
        perror("VIDIOC_STREAMOFF");
        exit(1);
    }

    close(fd);
    return EXIT_SUCCESS;

    return a.exec();
}

What am I doing wrong? The camera works and I run as root, but for some reason I get "Permission Denied". I appreciate any help - thanks.

UPDATE - The exact error I see is shown below as well as the terminal output showing that "root" is in the "video" group

root@raspberrypi:/home/pi# groups root
root : root video
root@raspberrypi:/home/pi# ./v4l2-test 
Begin Capture
VIDIOC_STREAMON: Operation not permitted

UPDATE: Running 'file' on the v4l2-ctl saved image shows the 2592x1944 sized frame:

root@raspberrypi:/tmp# file somefile.jpg 
somefile.jpg: JPEG image data, Exif standard: [TIFF image data, big-endian, direntries=9, height=0, model=imx219, xresolution=130, yresolution=138, resolutionunit=2, datetime=1970:01:01 00:00:00, width=0], baseline, precision 8, 2592x1944, frames 3

UPDATE: Here is a paste of the complete STRACE output of all commands:

https://zerobin.net/?84fee190a4a904a0#ZZOwDe7ftBKjpqpP/lPfnTMPrUlILyJaCCCyunO6aug=

UPDATE: Linking step of application compilation

/home/me/Desktop/buildroot/output/host/bin/arm-buildroot-linux-gnueabihf-g++ --sysroot=/home/me/Desktop/buildroot/output/host/arm-buildroot-linux-gnueabihf/sysroot -o v4l2-test main.o   -lrt -lpthread -ldl -latomic 

UPDATE: I removed my S_FMT ioctl, but the issue still continues:

    struct v4l2_format format;
    format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    format.fmt.pix.pixelformat = V4L2_PIX_FMT_JPEG;
    // 3280x2464
    format.fmt.pix.width = 3280;
    format.fmt.pix.height = 2464;

//    if(ioctl(fd, VIDIOC_S_FMT, &format) < 0){
//        perror("VIDIOC_S_FMT");
//        exit(1);
//    }

    struct v4l2_requestbuffers bufrequest;
    bufrequest.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    bufrequest.memory = V4L2_MEMORY_MMAP;
    bufrequest.count = 1;

UPDATE: I just checked dmesg again I'm seeing the following error - however Google has not turned up anything so far...

Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_QUERYCAP
Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_S_FMT
Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_REQBUFS
Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_QUERYBUF
Jan 06 01:40:20 raspberrypi kernel: bcm2835-v4l2: Failed to enable capture port - error -28. Disabling camera port again
Jan 06 01:40:20 raspberrypi kernel: video0: VIDIOC_STREAMON: error -1

UPDATE: Per suggestion - I tried increasing the buffer count to 3 - but it did not help:

struct v4l2_requestbuffers bufrequest;
bufrequest.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
bufrequest.memory = V4L2_MEMORY_MMAP;
bufrequest.count = 3;

if(ioctl(fd, VIDIOC_REQBUFS, &bufrequest) < 0){
    perror("VIDIOC_REQBUFS");
    exit(1);
}

fprintf(stderr,"Buffer Request is for 3 buffers...\n");

Answer 1

We have a case of a program that runs without error on some cameras (such as mine), but gives an error specifically with Raspberry Pi Camera Module V2. We can track this down starting with the dmesg error:

Jan 06 01:40:20 raspberrypi kernel: bcm2835-v4l2: Failed to enable capture port - error -28. Disabling camera port again

Error 28 (ENOSPC) is "No space left on device". Grepping the kernel, the code flagging this error is in bcm2835/mmal-vchiq.cpp:

    /* ensure there are enough buffers queued to cover the buffer headers */
    if (port->buffer_cb != NULL) {
        hdr_count = 0;
        list_for_each(buf_head, &port->buffers) {
            hdr_count++;
        }
        if (hdr_count < port->current_buffer.num)
            return -ENOSPC;
    }

So this driver expects all buffers to have been allocated by the time VIDIOC_STREAMON is called. That wasn't the case in the original code, while you'll see that VIDIOC_QBUF comes before VIDIOC_STREAMON in the strace log for v4l2-ctl --stream-mmap... which does work.

However, there's an additional mismatch in v4l2-test.cpp's strace log:

ioctl(3, VIDIOC_REQBUFS, {count=1, type=V4L2_BUF_TYPE_VIDEO_CAPTURE, memory=V4L2_MEMORY_MMAP}) = 0 ({count=3})

That means although the program requests a single buffer, V4L2 is insisting on three buffers! Perhaps the code was originally designed for a different camera, one like my own that permits bufrequest.count = 1 as is.

This revised program creates a loop to prepare all three buffers. To allocate and queue each one, we do VIDIOC_QUERYBUF, mmap(), then VIDIOC_QBUF. Once all buffers are set up, we can start the loop that pulls camera images from the circular buffer.

Another way of looking at the revised code: it runs a sequence very similar to what's found in the strace log for v4l2-ctl --stream-mmap.... Comparing the strace logs did initially bring red herrings such as extra linked libraries or O_LARGEFILE. Fortunately once those were ruled out, the ordering of ioctl() and other system calls provided a useful reference implementation.

Answer 2

I was getting VIDIOC_STREAMON: Operation not permitted while using cv2.VideoCapture(0). Setting the GPU memory to a higher value, solved the problem.

Answer 3

Adding to Pedro Lobito's response:

While I was testing at the command line using v4l2-ctl to perform test captures a lot of the supported pixelformats failed with "Operation not permitted" when trying for the full resolution supported by the chip 3280x2464 or just hang indefinitely with no error message and restarting the terminal and running any v4l2-ctl commands would from then on hang requiring a reboot of the Pi to fix.

But increasing the GPU memory in the "Raspberry Pi Configuration" to 256 fixed the "Operation not permitted" and the hanging for me.

I was using the Raspberry Pi Camera Module V2.