How to stream high res pics from RPi camera?

Question

Currently working on a project that requires the highest resolution images from the RPi's camera (2592x1944) to be sent over LAN to PC image processing server. I've been digging and with some help from the raspberrypi.org forums I've managed to generate a MJPEG file with v4l2-ctl that can capture about 5 fps at that resolution. That fps is more than enough for my project.

Now I'm trying to figure out how to best stream that MJPEG over LAN to my PC that serves as my image processing server. We need to do some heavyweight opencv object detection and recognition on those big image frames, no way the RPi could handle it.

Ideally the streaming would happen as fast as possible and not even save the MJPEG video data to the RPi's SD Card, just shoot it across the network to PC client. Further, if the network couldn't keep up with the pace of the data generated, then something in the pipeline would be smart enough to buffer and throw things away without crashing the system.

Looking for some guidance on how to do this. I've read possible solutions with mjpeg_streamer, gstreamer, netcat, and a few others. Just not sure which one fits best with my problem. Getting ready to experiment, thought I'd ask around first hoping to save some time and effort.

----- edit -----

I feel like I made some good progress rolling my own solution. I needed a binary stream to the mjpeg, and then my OpenCV client handles grabbing the individual jpeg frames out of the stream. So I wrote a simple servlet running on tomcat running on the RPi. It reads stuff put into a FIFO named pipe and sends buffers of data to the client.

The named pipe seems to be great for keeping the amount of data the camera interfacing software generates in sync with the amount of data that can be sent over the network. And there is no cleanup necessary.

It can do about 10 fps and be close to real time at 1920x1080, but bogs down a little at the full 2592x1944. It can only do about 4fps with about a 4 second lag on my LAN at max res. Very clean simple solution so far, now I need to figure out how to handle multiple clients.

----- edit #2 -----

Streaming to multiple clients wasn't too hard once I realized that you can't write to ServletOutputSteam in a background thread. I just had to make sure every buffer from the named pipe was sent to each client, and further sent in each client's request thread.

One last hurdle is gaining a better understanding of how to handle re-opening a named pipe in my servlet. I'm pumping an MJPEG into the named pipe from v4l2-ctl. When v4l2-ctl finishes its last frame, I'm not sure what I need to do in the servlet so future requests will work if I start a new v4l2-ctl process to feed the named pipe. I've tried closing and reopening in the servlet, but that doesn't work--it reads a few buffers and thinks its done.

Answer 1

mjpeg_streamer and gstreamer are probably the easiest to deal with in that they won't involve any coding (although there might be a certain amount of compilation involved). On the other hand, I'd take a look at the image streaming recipes in the picamera library. Unlike mjpeg_streamer and gstreamer it'll involve doing a bit of Python coding, but you'll get complete control over the protocol involved.

For example, you could run it over UDP so in the case of congestion the underlying network would throw stuff away for you (although you'd need to handle dropped and duplicated packets in your own protocol on top of that - it's not terribly difficult though - just bung an incrementing sequence number in there so the client can tell when stuff's been dropped or duplicated).

I don't think any of the methods proposed (including the picamera one) would save stuff to the Pi's SD card so no worries about IO bandwidth on that front, but you might struggle to get 5fps out of individual JPEG captures with picamera (it's probably possible using an MJPEG video recording, but I've found the firmware's MJPEG support to be a bit flaky generally).