1

I don't know much about video, but to play an h264 encoded file into a screen, don't you have to decode it to something like RGB first? I would expect playing a video that is in RGB format to skip the decoding step, therefore being strictly faster than a video that has to be decoded (even if I decode it with hardware acceleration, as I did). That doesn't seem to be the case, but I can't tell whether that is due to a software issue, like the RGB file being played inefficiently. I must be very wrong about something.

I've been playing around all day with GStreamer trying to smoothly play video files on my Raspberry Pi 3. I finally managed it in an almost fresh install of Jessie (after apt-get update, apt-get upgrade and apt-get install gstreamer1.0-tools).

I have an uncompressed (8-bit RGB) .mov version of the video I want to play, and an h264-encoded .mov version. I played both of them:

gst-launch-1.0 filesrc location=h264_test.mov ! qtdemux name=demuxer ! h264parse ! omxh264dec ! ximagesink

gst-launch-1.0 filesrc location=uncompressed_test.mov ! qtdemux ! videoparse format=GST_VIDEO_FORMAT_RGB height=800 width=480 ! glimagesink

Honestly, I expected the uncompressed version would play faster, but it's slow. The h264-encoded version, which omxh264dec decodes with hardware acceleration, plays at the correct speed (which in this case is 25 FPS), but the other one lags and I get GStreamer warnings about buffers being dropped and the computer being slow. I used different sinks for each one (ximagesink and glimagesink) because neither file would play correctly on the other one, which I'm not sure makes a difference.

I think I would've preferred that the uncompressed version was faster, because that means I don't have to bother with fixing banding issues. Really, though, I'm asking this question because I feel like I have a misconception about how this works.

UPDATE: I tried adding a queue, which I understand should buffer video, thinking that SD card reading was the bottleneck:

gst-launch-1.0 filesrc location=uncompressed_test.mov ! qtdemux ! videoparse format=GST_VIDEO_FORMAT_RGB height=800 width=480 ! queue max-size-bytes=125000000 max-size-time=5000000000 max-size-buffers=1000 ! glimagesink

As far as I can tell, the file is playing just as slowly, and I'm still getting GStreamer warnings. Maybe the OpenGL renderer is slow? I'll script some read speed tests later to see how long exactly it takes to read the file.

The particular file I'm using is 116 MB, 4 seconds long and has a bitrate of 232 Mbps.

  • 1
    I don't think it'll make much difference to CPU usage (h264 is decoded by the GPU) - whereas, uncompressed raw video could also be hampered by disk read speed? – Jaromanda X Jul 26 '18 at 4:01
  • Yes, I think you might possibly be right about reading the SD card being the bottleneck. I would love it if someone had a more definite answer to confirm/deny that. – TAL33 Jul 26 '18 at 4:26
  • 10Mb per second i think – Jaromanda X Jul 26 '18 at 4:53
  • Yes! After looking up SD card read speeds. I realized you're right! If I recall correctly (I don't have my computer right now) my uncompressed file was at least 50 MB for 2 seconds, and that link puts my SD card at 17 MB/s on a Raspberry Pi. Thanks! – TAL33 Jul 26 '18 at 5:06
  • So, I tried using GStreamer with a queue which should buffer input before playing the video, but it still plays slowly (I updated my question). Maybe the OpenGL renderer is slow? – TAL33 Jul 26 '18 at 15:16
0

Since you didn't mention enabling the open source stack, I suspect you're using the bcm video drivers and gstreamer's GL will be using Mesa software rendering instead of hardware. To get a fair comparison you'd need to enable the open source display stack in raspi-config (or use debian or fedora instead)

With glimagesink, gstreamer uploads the decompressed frame into a GL texture, then draws it to the screen. Because it's GL, you get vblank synchronization like any other GL app. Your copies of the video data are (texture, framebuffer) if you're full screen pageflipping or (texture, framebuffer, screen) if you're windowed. Probably add another copy if you're windowed and compositing. (edit: sorry! there's one more copy unless you're using Mesa master because hadn't implemented an important fastpath)

With ximagesink (unless they've added usage of the Present extension, which I highly doubt), you won't get vblank synchronization. Your copies will be (shm, screen) whether or not you're full screen. Add another copy if you're compositing.

Normally glimagesink would be more of a win, not just for vblank synchronization feature, because it uploads YUV data to textures and uses a shader to colorspace convert during the draw, so the upload is smaller and you use less memory bandwidth than the X11 case and a whole lot less CPU for colorspace conversion.

  • This answer is a bit over my head right now (thank you though! - and sorry for the late response, I've had my mind elsewhere). I did try the experimental driver in raspi-config and it did play faster but it was crashing my raspi when I used ximagesink. Also it doesn't seem to work when I have set a "custom" resolution (and I think it caused my raspi to stop booting at one point). When you say video data such as (texture, framebuffer) you are talking about what the sink has in memory, yes? If glimagesink does colorspace conversion, shouldn't it be slower? Do screens usually take RGB input? – TAL33 Jul 29 '18 at 3:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.