4

I want to stream the live video feed captured from a camera attached to my Raspberry Pi Zero to some browser with as low latency as possible (<1s). My end goal is attaching the Pi Zero & Camera to a drone and streaming the video via LTE.

However, I'm kind of lost in the jungle of different solutions that exist here (MJPEG, RTMP, RTSP, WebRTC).

So far, I tried a simple mjpeg streamer following this guide :

import cv2
import  pyshine as ps #  pip3 install pyshine==0.0.9

HTML="""
<html>
<head>
<title>PyShine Live Streaming</title>
</head>

<body>
<center><h1> PyShine Live Streaming using OpenCV </h1></center>
<center><img src="stream.mjpg" width='320' height='240' autoplay playsinline></center>
</body>
</html>
"""
def main():
    StreamProps = ps.StreamProps
    StreamProps.set_Page(StreamProps,HTML)
    address = ('127.0.0.1',9000) # Enter your IP address
    try:
        StreamProps.set_Mode(StreamProps,'cv2')
        capture = cv2.VideoCapture(0)
        if capture.isOpened():
            capture.set(cv2.CAP_PROP_BUFFERSIZE,2)
            capture.set(cv2.CAP_PROP_FRAME_WIDTH,240)
            capture.set(cv2.CAP_PROP_FRAME_HEIGHT,320)
            capture.set(cv2.CAP_PROP_FPS,8)
            StreamProps.set_Capture(StreamProps,capture)
            StreamProps.set_Quality(StreamProps,40)
            server = ps.Streamer(address,StreamProps)
            print('Server started at','http://'+address[0]+':'+str(address[1]))
            server.serve_forever()

    except KeyboardInterrupt:
        capture.release()
        server.socket.close()

if __name__=='__main__':
    main()

While the latency of this simple solution is sufficiently low, it's nowhere near robust enough for my use case. Whenever the connection is interrupted, the stream stops until the browser is refreshed. With slow internet, the stream tends to have a large delay (15s or more) after a while.

What kind of solution should I use for my use case?

Thanks for your help!

Improve this question
1
  • What did you end up doing here? Did you get low-latency streaming over LTE? How did you deal with firewalls?
    – vercellop
    Jul 4 at 7:04

1 Answer 1

Reset to default
1

For low latency, you really can't do better than an MJPEG stream.

The implementation put together by the RPi-Cam-Web-Interface project is very configurable and I get sub-second latency on my applications (although I'm not streaming through LTE so ymmv) Also this only works with Rpi camera port compatible cameras, so no USB cameras.

For the easiest set up you need to be on the same wifi network, but you can set it up with ngrok or have it vpn into your network with something like tinc

Not specific to your question but I put together a bit of python boilerplate for using the RPi-Cam-Web-Interface stream with CV tasks. Grabs the latest frame for some CV and then grabs the next one. Maybe it's useful for your application

import numpy as np
import urllib.request
import cv2

camera_name = "raspberrypi.local"
stream = urllib.request.urlopen('http://' + camera_name +'/html/cam_pic.php')


WINDOW_NAME = "Camera"
cv2.namedWindow(WINDOW_NAME)
cv2.startWindowThread()

async def main():
    while True:
        bytes += stream.read(1024)
        a = bytes.find(b'\xff\xd8') #frame starting 
        b = bytes.find(b'\xff\xd9') #frame ending
        # new frame is available
        if a != -1 and b != -1:
            jpg = bytes[a:b+2]
            bytes = bytes[b+2:]
            frame = cv2.imdecode(np.frombuffer(jpg, dtype=np.uint8), cv2.IMREAD_COLOR)

            #do your CV stuff here

            cv2.imshow(WINDOW_NAME, frame)

            #get the next frame by adding any new string to the end of the url, here we use a timestamp
            url = 'http://' + camera_name +'/html/cam_pic.php?' + str(timestamp)
            #print(url)
            stream = urllib.request.urlopen(url)
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.