2

I am trying to stream my Intel RealSense D435 RGB camera input to a multicast address on my network.

Please note, video streaming, encoding, is very new to me - I do not understand much, when explaining a solution, please give me a few extra details, thanks :)

My Hardware Setup:

  • Intel RealSense D435 (with USB Type-C connector)

  • Raspberry Pi 4 (4gb Ram) (the server)

  • 5G cabable Wifi router (mentioned for full details)

Firstly, what is the problem:

When attempting to stream my /dev/video4 (RBG input) to the network using (full output):

ffmpeg -f v4l2 -input_format yuyv422 -video_size 1280x720 -i /dev/video4 -c:v h264_omx -pix_fmt yuv420p -s 1280x720 -r 30 -g 60 -f mpegts udp://238.0.0.1:5000

then client side I try viewing the stream with VLC and ffplay.

  • VLC

using VLC on a separate network client with network stream udp://238.0.0.1:5000 loads infinitely with no picture (I tried streaming to locahost on the pi, and on localhost with udp://127.0.0.1: 5000 ffmpeg stream did not show either).

  • FFPlay

FFPlay stream was detected on a separate network client, but gives the this error output when using ffplay udp://238.0.0.1:5000.

Why am I using h264_omx?

Raspberry Pis have a hardware encoding ability, and hoping to take advantage of this, I configured ffmpeg with

  --enable-omx \
  --enable-mmal \
  --enable-omx-rpi \

this (all commands uncommented) is the setup script I used, adapted from this.

Was I able to get a picture through the network

Yes, but there were problems. The video was very very choppy and extremely bad quality. To get this, I used the same ffplay command as mentioned above on a separate network client and the server command used was with the SW encoder libx264 (server full output) and (client full output):

ffmpeg -f v4l2 -input_format yuyv422 -video_size 1280x720 -i /dev/video4 -c:v libx264 -pix_fmt yuv420p -s 1280x720 -r 30 -g 60 -f mpegts udp://238.0.0.1:5000

what really bothered me about this is I had waited 25s after firing up both client and server to get a picture with the above command.

Further, and this is a major issue I have experienced so far, I have a 5-7s delay on my stream - I understand that hd encoding may help and network propogation takes time, but on a small 5G network I should not have to wait 7s for a bad quality (almost 144p @ 5-10fps) video stream

Diagnosing this issue on IRC, I was asked to provide the following:

  • ffmpeg -f v4l2 -list_formats all -i /dev/video4 result

  • v4l2-ctl --list-devices

    pi@raspberrypi:~ $ v4l2-ctl --list-devices
    bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12
    
    Intel(R) RealSense(TM) Depth Ca (usb-0000:01:00.0-2):
        /dev/video0
        /dev/video1
        /dev/video2
        /dev/video3
        /dev/video4
        /dev/video5
    
  • v4l2-ctl --list-formats

    pi@raspberrypi:~ $ v4l2-ctl --list-formats
    ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture
    
        [0]: 'Z16 ' (16-bit Depth)
        [1]: '' (00000050-0000-0010-8000-00aa003)
    
  • ffmpeg -encoders | grep h264 full output

     V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
     V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
     V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
     V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
    

    V..... h264_vaapi H.264/AVC (VAAPI) (codec h264)

  • ffmpeg -encoders | grep h264 full output

     VFS..D h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
     V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
     V..... h264_mmal            h264 (mmal) (codec h264)
    

What is preventing me from getting a good quality 720p image over the network, and what causes this 5-7s delay (even after adding -g 60 as mentioned here)

1

Actually Intel provides a software to utilize a RPi4 as backend, to make your camera available in your network and directly use all API commands: https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras

FRAMOS has a similar middleware for their reference models: enter image description here D435_Manual_V1.1 - Page 24

https://www.framos.com/en/framos-depth-camera-d435e-camera-only-22806

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.