3

I have had good success on the RPi 3 using ffmpeg to decode h.264 streams from various Ethernet video sources. However, on the RPi 4, when I include the -c:v h264_mmal directive, it appears to generate blank frames, as if the decoder never outputs any data. I have re-compiled ffmpeg from the latest source, using the --enable-mmal flag, but it still only outputs what appear to be blank frames. Example output below.

Also, if I use ffplay to play the same video, it plays fine using the software decoder (but uses high CPU, due to software decoding), but when I add the -c:v h264_mmal tag to enable hardware decoding, I get nothing (never loads a video window, etc). (no error message, however)

However, omxplayer does successfully play the video and seems to be using hardware-assisted decoding, as the CPU usage is much lower than ffmpeg on software decoder.

What am I doing wrong? How can I get ffmpeg to work with the hardware decoder?

Example output of ffmpeg (generating snapshots of a video stream)

ffmpeg -c:v h264_mmal -i "rtsp://192.168.1.87/stream0:554?username=admin&password=admin" -vf fps=1/60 img%03d.jpg
ffmpeg version git-2020-02-20-56df829 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Raspbian 8.3.0-6+rpi1)
  configuration: --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree --enable-mmal
  libavutil      56. 41.100 / 56. 41.100
  libavcodec     58. 71.100 / 58. 71.100
  libavformat    58. 38.101 / 58. 38.101
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 76.100 /  7. 76.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[udp @ 0x36e1fc0] attempted to set receive buffer to size 393216 but it only ended up set as 327680
[udp @ 0x36f2310] attempted to set receive buffer to size 393216 but it only ended up set as 327680
Input #0, rtsp, from 'rtsp://192.168.1.87/stream0:554?username=admin&password=admin':
  Metadata:
    title           : RTSP/RTP stream from anjvision ipcamera
  Duration: N/A, start: 0.083000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 1280x720, 12 tbr, 90k tbn, 180k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_mmal) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A 

speed=N/A  frame=    0 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A  frame=    0 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A 
...
snip
...
speed=N/A  frame=    0 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A  [h264_mmal @ 0x381f0c0] Did not get output frame from MMAL.
    Error while decoding stream #0:0: Unknown error occurred
    Finishing stream 0:0 without any data written to it.
    [mjpeg @ 0x3718b30] bitrate tolerance 4000000 too small for bitrate 200000, overriding
    Output #0, image2, to 'img%03d.jpg':
      Metadata:
        title           : RTSP/RTP stream from anjvision ipcamera
        encoder         : Lavf58.38.101
        Stream #0:0: Video: mjpeg, yuvj420p(pc), 1280x720, q=2-31, 200 kb/s, 0.02 fps, 0.02 tbn, 0.02 tbc
        Metadata:
          encoder         : Lavc58.71.100 mjpeg
        Side data:
          cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
    frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x
    video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
    ^C
  • 3
    The GPU has been switched from VC4 to VC6. The Raspberry Pi 4 was a bit rushed such that most of the hardware features from VC6 have not been implemented yet. To my best knowledge I have not seen a Raspberry Pi 4 being able to hardware decode or encode video. This is a software issue. I'd suggest to stick to any other model where the software has been adapted to use the hardware capabilities. – kwasmich Feb 21 at 15:38
  • 3
    Thanks @kwasmich for this info. It's discouraging considering the length of time the RPI 4 has been out, but at least it's good to know what's up. However, how does this account for omxplayer seeming to use hardware decoding? The CPU usage of omxplayer was around 1%, whereas ffmpeg on software decoding was 25%. That's a huge difference, so omxplayer must be using hardware, right? – Ryan Griggs Feb 21 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.