11

There are gazillions of forums and threads about getting rid of a 5 second latency when using a Pi together with a PI-Cam as a surveillance camera. Many tutorials show how to use vlc to encode and stream the images using the RTP protocol which results in a ~5 second lag.

According to me, the reason is that raspivid is encoding the stream to H264, while VLC has to decode it again and re-encode it to whatever RTP is. The commandline looks like this:

raspivid -w 640 -h 480 -o - -t 0 |cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/}' :demux=h264

The first part tells raspivid to stream video to the standard output:

raspivid -w 640 -h 480 -o - -t 0

The part after the pipe, tells VLC to pick it up, and decode it using h264:

cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/}' :demux=h264

This mux-ing and demux-ing is quite a waist of resources!

I found the sources of raspicam at github, and I think something can be done in the encoder_buffer_callback method (currently at line 848) to skip the encoding. However I'm not good at c, and not familiar with video encoding at all, so I don't have a clue where to begin.

On Github I can see 330 forks, but they don't seem to be specifically for raspicam (rather for the whole userland project). I got lost trying to find a fork that removed encoding or implemented something simpler like mjpeg.

Could someone with c and video codec knowledge help me and the other gazillion users to get rid of the latency? Probably the solution is already out there in one of those forks, but I've spent hours searching for it without any luck.

p.s. I'm not looking for a browser solution, but I ultimately want to stream it to a Synology, preferably using mjpeg streaming (but not via a webpage, rather a standard mjpeg stream that comes built into most commercial ip-cams). First step is gettig rid of h264.

Improve this question
5
  • That is a very thorough investigation. Using MJPEG is out of the question because (at the time I was looking) the built in JPEG encoder had no library and software was rubbish. I managed to get about 1s lag using nginx-rtmp (FLV packaged), custom build, in HD! The pi used about 30% CPU but VLC struggled to decode it because of timing frames that were missing and becuase its FLV :( Also my CCTV software used VLC sink and 1 720p stream used 40% CPU when i finally got it working but was very unstable.
    – Piotr Kula
    Commented Jul 28, 2014 at 19:51
  • @ppumkin I don't believe MJPEG is out of the question. Once H264 is gone we can encode to whatever we wish without having the overhead of decoding and re-encoding. Actually having H264 is quite a luxury! Only we should be able to switch it off. It's like a luxurious pluche padded and diamond stubbed throne in a cramped toilet chamber so nothing else fits in the room (except for some headroom where we can jump up and down a little)... when you open the door, you have to climb over a high armrest, and when you sit there's no room for your legs...
    – Louis Somers
    Commented Jul 29, 2014 at 17:31
  • Yes I understand what you mean. I have spend days trying to get MJPEG in a decent quality to my CCTV IP server. Manybe things have changed but direct hardware JPEG encodding and piping to a stream is non existent, as the API is not avaible. The only way I know is software and the best solution I found was nginx-rtmp JPEG sink. HLS for iPhone works great actually but its got a 5s-10s lag :(
    – Piotr Kula
    Commented Jul 29, 2014 at 17:54
  • 1
    You might try raising this as a question on the Github project for the camera software. You'll probably get get an explanation of why it's hard to do, but if others find it a useful feature, someone might implement it.
    – TomG
    Commented Aug 28, 2014 at 11:42
  • demux is not the same as decode... please check this out
    – Flash Thunder
    Commented Jul 4, 2015 at 6:38

2 Answers 2

Reset to default
5

That's probably not what you are wanting from answers, but I do not recommend VLC streaming at all..

For a school project, I tried some streaming options (on RPi too!) :

  • VLC
  • MJPEG
  • GStreamer

Using VLC and MJPEG (and some other less known), I had latency between 3 and 5 seconds..
Using GStreamer, NO LATENCY and with a best resolution (and lots of more options) !
If you are interested, you can check it out here.

And if you'll use it, here is my pipeline :

raspivid -t 0 -w 640 -h 480 -fps 25 -b 1200000 -p 0,0,640,480 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=YOUR_IP port=YOUR_PORT
Improve this answer
5
  • 1
    Yes, I have tried Gstreamer already and it works great when streaming to a PC or so, but it does not work well with 3rd party software like the Synology DiskStation. It looks like GStreamer just reuses the h264 stream and wraps a streaming protocol around it without decoding it, which is a great solution but unfortunately for a small number of use-cases. If I'd want to see the stream on an Android device I'd have to develop my own app for it. MJPEG is much more widely supported, and I'm sure Raspivid can be altered to skip needless encoding leaving that to VLC or so. Thanks for the tip any way
    – Louis Somers
    Commented Jul 29, 2014 at 12:30
  • 1
    I also have a Synology but not tried GStreamer on it (mine is not powerful enough I think). Btw, about Android, you can use GStreamer in it too ! I'm using it for my project and it is working well :) !
    – Val
    Commented Jul 29, 2014 at 17:11
  • Problem with gstreamer it doesnt include timeframeinterval into the frame data and its unusable as a sink (even with the option added!!) :( There are mod scripts available to pipe via but I found recieving gstreamer to VLC bombs out often. This was about 6 months ago when I was looking to do saome CCTV.. but it never worked reliably :(
    – Piotr Kula
    Commented Jul 29, 2014 at 17:35
  • @Val true, there is an SDK for Android, but not many out-of-the-box players in the play-store that support pipelines like gst-launch-1.0 -v tcpclientsrc host=11.22.33.44 port=1234 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false. For iOS there's even less available. The Synology is a great hub that supports most mainstream devices, can do motion-detection, recording and notifications, all out of the box (and without waisting SD-cards).
    – Louis Somers
    Commented Jul 29, 2014 at 18:08
  • Are you still using this like this? There is the new U4VL driver, but still same lag problems with h264 streaming to VLC
    – Piotr Kula
    Commented Dec 5, 2014 at 18:38
0

Some folks have been working hard on this since I first asked this question, and by this time there are a few options (strange that no-one has responded to this question yet). I have tried RaspberrIPCam and had some success, however it seems like the rtsp packets had an extremely short TTL or something. Having the Pi hooked up directly to a router next to my PC it would work perfectly. But as soon as I installed the cam where I wanted it, and tried to access the stream with two routers between, no image would arrive. I checked the source-code and found the TTL set to maximum. I never figured it out completely.

Currently I would recommend RaspberryIPCamera wich has a nice user interface (see screenshots) and there even is a ready-made SD-card image for it. I have tried the SD-card, but reverted to doing a manual install as described here with great success (my current setup). Instructions for hooking it up to a Synology DiskStation are also available and are working perfectly on my system. The problem with the SD card image was that I was unable to expand the filesystem to the full extent of the SD card (I also want to run some other stuff on it to control some relays via the GPIO pins).

The above solution uses components of the UV4L project. The documentation of the UV4L project on this page also mentions:

Among the other things, it offers a Web interface from which it’s possible to see the video stream in various ways and a Control Page allowing to fully control the camera settings while streaming with any Video4Linux application.

I have not yet tried it lake that yet though (since I don't want to mess up my current set-up).

Improve this answer
2
  • 1
    Not sure if this was your problem, but if you're sending multicast RTSP traffic through a router, make sure it has IGMP snooping enabled and make sure your PC isn't blocking the IGMP queries from the router. Otherwise the router won't realise your PC is trying to receive the packets, so it will never forward them on.
    – Malvineous
    Commented Aug 20, 2017 at 0:13
  • I think the problem with RTSP is that it uses multiple connections: the client opens a port and tells the server to send the data to that port. But since that port is closed on your second router, it fails. I recommend using other protocols like HTTP or raw TCP. en.wikipedia.org/wiki/Real-Time_Streaming_Protocol#SETUP
    – Martin
    Commented Jun 29 at 18:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.