UV4L audio + video recording or stream?

Question

I have experimented with a number of different ways of streaming both audio and video from the raspberry pi. The UV4L web server has great webRTC support, and the bi-directional audio/video conferencing is amazing.

I can get a video stream in mjpeg and H264 from the same server.

However, I have not found the ability to stream audio + video from the raspberry pi in a non-webrtc format such as rtmp, and I also do not see how to record audio + video to a file programatically. Has anyone else figured this out? I suspect aconv or ffmpeg in conjunction with the video4linux2 /dev/video0 uv4l device could accomplish both streaming or writing to file, but I have not arrived at the correct command string.

Edit: This command seems to get the job done in terms of recording audio and video to a file. I did notice ALSA buffering issues that caused the audio to crap out, but moving the thread_queue_size progressively higher finally achieved acceptable audio. This command can also be used to stream rtmp by piping to some sort of rtmp server. Is there a better way to do this using pre-existing UV4L functionality?

avconv -thread_queue_size 2000 -f alsa -ac 1 -i hw:1,0 -f video4linux2 -s 320x240 -r 25 -thread_queue_size 2000 -i /dev/video0 -pix_fmt yuv420p -acodec libmp3lame -ar 44100 -r 25 -f mp4 example.mp4

Edit2: This is my second iteration of the command, which records a 10 second clip, syncs audio and video, and uses native h264 copy of video stream. Seems to perform well on the RPI.

avconv -t 10 -thread_queue_size 1024 -f alsa -ac 1 -i hw:1,0 -f video4linux2 -input_format h264 -video_size 1280x720 -r 30 -thread_queue_size 1024 -itsoffset 0.5 -i /dev/video0 -c:v copy -acodec aac -strict experimental -t 10 -y -pix_fmt yuv420p out.mp4

Answer 1

If you connect to the UV4L Streaming Server and open the /stream/webrtc demo web page, you will certainly notice a Record button to record audio and video in webm format while streaming. There is also a button to play the recorded video in the browser. So at least on client side this is possible. On server side, it's possible with the help of the Janus Gateway. By default you should get audio too while streaming, if you have a microphone attached to the Rpi. I am surprised you did not see this stuff as these buttons are as big as elephants and you should hear audio coming from the rpi by default if you configured your e.g. USB audio card properly (see the installation instructions about audio for more details)

Answer 2

Here's an example that will stream audio and video using FFMPEG out to YouTube. You could send to your own RTMP server and record, or just use YouTube's recordings. I suppose you could also just output directly to an FLV file, but I have zero use for that, so I have never tried/tested. The latest version of Stretch has FFMPEG already packaged, just install with sudo apt-get install ffmpeg To ensure FFMPEG is installed: which ffmpeg

I assume you have your USB mic worked out properly. This can be a pain in the arse; DM me if you have issues with that.

With Audio:

ffmpeg -ar 44100 -ac 1 -acodec pcm_s16le -f alsa -thread_queue_size 2048 -i hw:0,0 -thread_queue_size 2048 -r 10 -i http://localhost:8080/stream/video.mjpeg -vcodec libx264 -crf 18 -preset ultrafast -profile:v baseline -acodec libmp3lame -ac 2 -ab 128k -f flv 'rtmp://x.rtmp.youtube.com/your-you-tube-stream-name/your-you-tube-key'

With empty Audio:

/usr/bin/ffmpeg -thread_queue_size 2048 -r 10 -i http://localhost:8080/stream/video.mjpeg -thread_queue_size 2048 -ar 44100 -ac 1 -acodec pcm_s16le -f s16le -ac 2 -i /dev/zero -r:v 11 -vcodec libx264 -crf 18 -preset ultrafast -profile:v baseline -acodec libmp3lame -ac 2 -ab 128k -f flv 'rtmp://x.rtmp.youtube.com/your-you-tube-stream-name/your-you-tube-key'

Raspi Config Settings:

width = 426
height = 240
bitrate = 500000
framerate = 10

I use low resolution so the mjpeg stream is still crisp and real-time over a reverse SSH tunnel. Higher resolution the reverse SSH tunnel to go very slow. If you are not worried about this, use higher resolution.