Webcam issue apparently not related to usb system

Question

There are many posts on the web regarding the apparently poor usb system performance of the Raspberry Pi. In fact it seems that the usb system causes numerous problems and that there is ongoing work to fix this: http://www.raspberrypi.org/phpBB3/viewtopic.php?f=28&t=23544

There are also many posts in which people complain that they can use webcams connected to the Pi via USB only at low resolutions, because at high resolutions (starting at 640x480) image data gets corrupted, the image being in fact incomplete. A small strip in the upper part of the image is fine, everything else is random memory content. Following post shows a sample of such images: http://www.raspberrypi.org/phpBB3/viewtopic.php?f=38&t=23800#p227124

I have the same symptoms when using my webcam (eb1a:2571 eMPIA Technology, Inc. M035 Compact Web Cam) with a resolution of 640x480. Each image has approximately 50 lines with correct image data, the remainder is random memory content. There have never been more than 100 correct lines. All lower resolutions work fine, even for 352x288 all images are complete.

This suggests that the lower resolution relieves the strain on the USB connection, thus preventing package loss. I believe though, that in my case, this is not related to the usb system.

When I was using the webcam on my laptop, running Ubuntu 12.04, where I had no issues, I used wireshark to analyze the traffic on the USB connection. To my astonishment I found that the amount of data sent by the webcam was always the same, regardless of the choosen resolution. Apparently the webcam always transfers image data at the resolution of the cmos sensor (640x480) and the image is then scaled internally by the driver.

Unfortunately I have not yet managed to install wireshark on the Pi (the package files were not found) to verify that on the Pi the behaviour is the same.

If the amount of USB traffic is the same for all resolutions, the issue can not be related to the usb system.

I did another test to verify if package loss on the usb controller causes the problem.

Since the ethernet jack uses the same controller than the USB ports, packet loss on the USB connection should mean that packets are also lost on the ethernet connection. While my Raspberry Pi was performing live capture from the webcam at a resolution of 640x480 and thus with not a single complete image, I sent 5000 ICMP echo requests at a rate of 1000 requests per second to the Pi, each with a packet size of 4 KB. Not a single of these packets was lost.

$ sudo ping -c 5000 -i 0.001 -s 4096 192.168.42.201
...
--- 192.168.42.201 ping statistics ---
5000 packets transmitted, 5000 received, 0% packet loss, time 6698ms

Might it be that this type of webcam issue is not related to the USB system, but rather occurs because webcam drivers perform image conversions internally and the Pi's processor simply is not suited for this?

Answer 1

I have managed to partially restore the image data of captured frames from the packets I have captured with Wireshark. That gave me following insights:

• The image data transmitted on the USB connection is exactly the same as I get from Video4Linux, the driver does not manipulate it. If I request lower resolutions, then the image data is scaled by the webcam and transmitted in the requested resolution.

That means that there are no internal scaling operations that would explain the problems I experience at high resolutions.

• The packets captured by Wireshark are incomplete, even if Wireshark pretends that everything was captured. The data is transfered in URB packets which are never longer than 62016 bytes. Each URB packet has up to 32 iso descriptors. Each iso descriptor has an offset and a length value, which locate the data associated with it within the packet. Usually half of these descriptors point to data that is beyond the end of the packet, e.g. I have an iso descriptor with an offset value of 94860 bytes in a packet that has only 62016 bytes. The corresponding data apparently has not been captured.

I learned that wireshark can only capture packets with a maximum size of 65535 bytes, though it should be possible to bypass this by explicitely setting the snapshot length:

$ tshark -i usbmon2 -s 131072 -w cap

Unfortunately that did not help. The captured packets were still no longer than 62016 bytes.

• The amount of data captured by Wireshark is always the same regardless of the choosen resolution, because between the data associated with the iso descriptors there are always large areas filled with 0's.

For example, the iso descriptors of a captured packet would be as follows:

USB isodesc 0: Offset [bytes]: 0, Length [bytes]: 584
USB isodesc 1: Offset [bytes]: 3060, Length [bytes]: 584
USB isodesc 2: Offset [bytes]: 6120, Length [bytes]: 584
USB isodesc 3: Offset [bytes]: 9180, Length [bytes]: 584
USB isodesc 4: Offset [bytes]: 12240, Length [bytes]: 584
USB isodesc 5: Offset [bytes]: 15300, Length [bytes]: 584
USB isodesc 6: Offset [bytes]: 18360, Length [bytes]: 584
...

As one can see, there is a huge amount of unused space in this packet. The data associated with the first iso descriptors extends from 0 to 584 bytes. Then, from position 584 to 3060 there are 2476 unused bytes, that are all set to 0.

For high resolutions there are fewer unused bytes, for lower resolutions there are more. That way, the total amount of data captured is always approximately the same for all resolutions.

Are these packets effectively transfered as they are captured by Wireshark, with loads of 0's? Probably not, because then there should be problems for all resolutions, not only for the high ones.

So I guess that in the end, the issue is with the usb system.

Answer 2

There is a DSP on the Pi, presumably if you could use that (Like the camera add-on does), then the conversions would be less taxing.