1

Good day,

I am currently working on an obstacle avoiding UAV using stereo vision to obtain depth maps. I noticed that the quadcopter would sometimes not steer to the correct direction.

I am using the Raspberry Pi Compute Module IO board which comes with two CSI ports used with two v1 Pi Cameras.

Issue

I soon found out that due to the latency between the cameras, the left and the right images are not in sync thus the errors in the depth map result.

Steps taken:

I noticed the image blur when moving the cameras around so I adjusted the shutter speed by setting the UV4l/raspicam driver. With the shutter speed, I also tried to increase the framerate as I've read, it improves the latency issue. In my code which uses the opencv library, I used the grab() and retrieve() commands to replace the read() command so that the frames from both cameras is grabbed at the nearest time possible however it didn't help much.

Does anyone know any possible solutions?

  • 1
    if you're using the stereoscopic support in the firmware you shouldn't be grabbing the images separately - one capture should result in a single image which contains both frames (side by side, or top to bottom depending on how the camera was configured on initialization). If you're having to do two captures then perhaps the UV4L driver doesn't support stereoscopic mode (or it does but you're not configuring it properly) - unfortunately I don't know much about UV4L so I can't help with that. – Dave Jones Jul 14 '16 at 14:45
  • @DaveJones Thank you so much it worked. Can you post your comment as answer so that I could accept it? – user123456098 Aug 11 '16 at 13:56
  • 1
    UV4L supports all the stereoscopic modes: side by side, top/bottom, and as separate captures. As said, it should be enough to enable the side by side mode to avoid the out of sync – prinxis Aug 14 '16 at 12:20
4

The Pi Compute Module's stereoscopic support causes captures (and video recordings) to produce frames containing images from both attached cameras, either side-by-side or top-and-bottom, depending on how the camera was configured on startup.

There are various restrictions on resolution (and other things) when working with stereoscopic mode, which 6by9 details in this thread announcing stereosopic support. For posterity, I'll paraphrase the non-MMAL-specific stuff below:

  • You need to specify a resolution that will incorporate both frames in their expected orientation. For example, if you want two 320x240 frames side by side, you would use the resolution 640x240.
  • For video recording, the H.264 encoder has an absolute limit of 1920 pixels width, and 1344 pixels height (which implies 720 top-bottom won't work). The width must also be a precise multiple of 128.
  • For unencoded captures and recordings the resolution will be the size of the MMAL port format (typically this implies that the width will be a multiple of 32, and the height a multiple of 16).
  • One of the cameras (the one specified as the camera number on startup with raspivid, picamera, etc.) will be the "primary". AGC, AWB, etc. will be calculated for frames from this camera. The secondary camera merely mirrors these settings. Hence, covering the primary sensor will cause output from the secondary to over-saturate.

Evidently UV4L does support stereoscopic mode (as you got it working), but I'm afraid I know nothing about configuring it for this mode.

2

In May 2016 Lattice Semiconductor published a guide (2:1 MIPI CSI-2 Image Sensor Aggregator Bridge Demo) for a Raspberry Pi dev kit used with the CrossLink Master Link board featuring a Lattice LIF-MD6000 device.

There are instructions on page # 8 Chapter 4 (Hardware Setup), that describes the use of one Raspberry Pi's camera's oscillator to synchronize both cameras. See the link to the PDF below.

http://www.latticesemi.com/~/media/LatticeSemi/Documents/UserManuals/1D/CrossLinkDualCSI2toCSI2DemoKitUG117.pdf?document_id=51685

**Note - the RPI cameras referenced in this answer are the older version 1.3 OV5647. The new versions are Sony IMX219. The OV5647 only has two CSI-2 data lanes and thus the horribly slow frame transfer rates (FPS). The IMX has 2 or 4 data lanes but I have not seen a good spec sheet that indicates accurately how the new RPI Cam is tooled for Pi use.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.