How do you set hardware video encoder options with V4L2 M2M?

Question

TL;DR When using ffmpeg and the h264_v4l2m2m hardware-accelerated codec, how do I set the encoder options such as the bitrate and key frame interval?

Details

I am using these commands to capture raw video from the Pi camera, draw the time and the Pi's temperature on the video, encode it to H264 with the hardware encoder, then stream it over the network:

v4l2-ctl -v width=1280,height=972,pixelformat=YU12
while true; do echo "PICAM02 $[`cat /sys/class/thermal/thermal_zone0/temp` / 1000]?C" > /tmp/msg; sleep 1; done &
ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x972 -use_wallclock_as_timestamps 1
  -i /dev/video0 -vf 'pad=h=(in_h+40),
    drawtext=x=(w-tw-8):y=(h-32):fontcolor=white:fontsize=30:text=%{localtime},
    drawtext=x=8:y=(h-32):fontcolor=white:fontsize=30:textfile=/tmp/msg:reload=1'
  -vsync 1 -r 30 -c:v h264_v4l2m2m -an -f rtp_mpegts "udp://224.0.1.2:5004"

It works, however when I use this command to view it, it only works if I start the viewer before streaming:

ffplay -sync ext -an -fast -framedrop -probesize 32 -window_title picam02 udp://224.0.1.2:5004

If the video is already streaming when I launch ffplay, I get a bunch of errors saying non-existing PPS 0 referenced. Apparently this error is because no keyframe has yet been received, and at least one key frame must be received before ffplay is able to start showing the video (because these frames include important things like the resolution of the video).

I therefore need to tell the hardware encoder to insert keyframes every few seconds, instead of the default which apparently only inserts one keyframe at the very start of the video.

How should I modify the above series of commands in order to configure the keyframe interval, and while I'm there, also the H264 bitrate and/or quality settings?

Answer 1

So when the V4L2 interface started providing hardware encoding/decoding, the method for setting these options changed.

Previously the options were global - you'd set them once on /dev/video0 or whatever, and they would take effect when the device was later opened for capture. You could also change them while the device was already open.

However for the hardware encoding/decoding devices, they can be opened multiple times. Each time they are opened a different instance is created, and the options are returned to their defaults.

This means the program you are using (such as ffmpeg) must set the options you want after it has opened the device. You can no longer set them with an external program, or set them once the device has already been opened.

This means that if you want to adjust any of these options, you have to figure out how to tell the program you're using, such as ffmpeg, to adjust those options for you. For example, when setting the video bitrate with the -b:v parameter, ffmpeg translates this into setting the video_bitrate V4L2 control.

Further info on the GitHub issue.