I am doing video conferencing on my Raspberry Pi 3b+. I have tried both Google Hangouts, Google Meetings and Jitsu but the best (GM with resolution set to low at both ends) still have a 3-5s lag between the real world and the self-image shown on the pi. The pi's image from the remote end is also 3-5s behind the reality at the remote end as well. Looking at the remote endpoint shows that there is no lag there and I have used an iPhone, a laptop and an iPad at the remote end - pi performance is unchanged in all situations.
This is a problem: I need a video conference setup that is super super simple so I can talk to my children while I'm working remotely. A kiosk raspberry pi would be perfect. However this kind of lag makes it impractical. Other things could be traded off - image quality, framerate. How do I fix the lag?
System set up:
- Raspberry Pi 3b+ (fan cooled - no throttling, it does not get hotter than ~55C even when all four CPU's are running at 100% for a long time)
- Raspbian Stretch
- custom compilation of ffmpeg for Raspberry pi (instructions: https://www.raspberrypi.org/forums/viewtopic.php?t=199775)
- official Raspberry Pi camera, attached using the 15-wire thin cable
- video memory set to 256mb via raspi-config
- speaker in the stereo jack and a USB microphone plugged in (they don't lag)
- only the video conferencing tab is open in Chromium, no other apps open
- htop shows all CPU's absolutely maxed out
- about 80% of the pi'S 756mb of RAM is used up, mostly Chromium
- very little swap use (a few mb)
Based on this and some experiments about what makes the lag worse, it is the decoding that is killing performance. Based on this I need to either reduce the remote camera framerate or reduce the remote camera resolution.
I will ask a more targeted question and should I ever crack this problemI will post the full solution here - RPi4 + custom ffmpeg wouldn't be enough, but nearly, so if I can force the remote camera to use a lower res or framerate ...