I got the Pi B+ and the Pi camera and am now trying to find the most efficient (low CPU) and lowest-latency configuration to stream H.264 encoded video from the camera to my home server.
I've read the following:
(All links use gstreamer-1.0 from
deb http://vontaene.de/raspbian-updates/ . main.)
A lot has been done in this regard in the past years.
Originally, we had to pipe the output of
gst-launch-1.0 (see link 1).
Then (link 2) the official V4L2 driver was created which is now standard, and it allows to directly obtain the data without a pipe, using just gstreamer (see especially the post by towolf » Sat Dec 07, 2013 3:34 pm in link 2):
gst-launch-1.0 -e v4l2src do-timestamp=true ! video/x-h264,width=640,height=480,framerate=30/1 ! h264parse ! rtph264pay config-interval=1 ! gdppay ! udpsink host=192.168.178.20 port=5000
gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! fpsdisplaysink sync=false text-overlay=false
If I understand correctly, both ways use the GPU to do the H264 decoding, but the latter is a bit mor efficient since it doesn't need to go through the kernel another time since there's no pipe between processes involved.
Now I have some questions about this.
Is the latter still the most recent way to efficiently get H264 from the camera? I've read about
gst-omx, which allows gstreamer pipelines like
... video/x-raw ! omxh264enc ! .... Does this do anything different to just using
video/x-h264, or might it even be more efficient? What's the difference?
How do I find out what gstreamer encoding plugin is actually used when I use the
video/x-h264 ...pipeline? This seems to be just specifying the format I want, as compared to the other pipeline parts, where I explicitly name the (code) component (like
In this reply to link 1 Mikael Lepistö mentions "I removed one unnecessary filter pass from streaming side", meaning that he cut out the
gdpdepay. What do those do? Why are they needed? Can I really strip them off?
He also mentions that by specifying
caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"parameters for the
udpsrcat the receiving side, he's able to start/resume the streaming in the middle of the stream. What do these caps achieve, why these specific choices, where can I read more about them?
When I do what's suggested in question 3 and 4 (adding the
gdpdepay) then my video latency becomes much worse (and seems to be accumulating, the latency increases over time, and after a few minutes the video stops)! Why could that be? I would like to get the latency I obtained with the original command, but also have the feature of being able to join the stream at any time.
I've read that RTSP+RTP usually use a combination of TCP and UDP: TCP for control messages and other things that mustn't get lost, and UDP for the actual video data transmission. In the setups above, am I actually using that, or am I just using UDP only? It's a bit opaque to me whether gstreamer takes care of this or not.
I would appreciate any answer to even a single one of these questions!