So I got a Raspberry Pi Zero Wireless with the Pi camera and I would like to use the camera with OpenCV in Python. The problem is, I know I can do this easily on the Pi itself but I highly doubt the Pi will be able to process what I want to do. I might be able to optimize my code sometimes in the future but I don't want to worry about that for now.
So what I wanna do instead is send the camera Data to my PC and run all the Python/Opencv code there. Ideally over Bluetooth. An USB connection to send the data would be sufficient as well.
I can find a lot about streaming video to a PC using VLC but not how to get the data into Python and latency seems to be a problem as well with this method.
If there is no easy solution for this I might just buy a tiny USB cam for now.
So I tried Dave Jones suggestion and went with this: On the Pi I simply use the provided code from rapid-capture-and-streaming and I can get close to 60fps with a decent enough resolution. The code looks like this:
import io import socket import struct import time import picamera class SplitFrames(object): def __init__(self, connection): self.connection = connection self.stream = io.BytesIO() self.count = 0 def write(self, buf): if buf.startswith(b'\xff\xd8'): # Start of new frame; send the old one's length # then the data size = self.stream.tell() if size > 0: self.connection.write(struct.pack('<L', size)) self.connection.flush() self.stream.seek(0) self.connection.write(self.stream.read(size)) self.count += 1 self.stream.seek(0) self.stream.write(buf) client_socket = socket.socket() client_socket.connect(('my_server', 8000)) connection = client_socket.makefile('wb') try: output = SplitFrames(connection) with picamera.PiCamera(resolution='853x480', framerate=60) as camera: time.sleep(2) start = time.time() camera.start_recording(output, format='mjpeg') camera.wait_recording(30) camera.stop_recording() # Write the terminating 0-length to the connection to let the # server know we're done connection.write(struct.pack('<L', 0)) finally: connection.close() client_socket.close() finish = time.time() print('Sent %d images in %d seconds at %.2ffps' % ( output.count, finish-start, output.count / (finish-start)))
On the client side I'm basically using the code from capturing-to-a-network-stream with an added cv2.imshow to get a preview. Everything displays fine but with a little bit of delay. Maybe a second or less.
import io import socket import struct from PIL import Image import cv2 import numpy as np # Start a socket listening for connections on 0.0.0.0:8000 (0.0.0.0 means # all interfaces) server_socket = socket.socket() server_socket.bind(('0.0.0.0', 8000)) server_socket.listen(0) # Accept a single connection and make a file-like object out of it connection = server_socket.accept().makefile('rb') try: while True: # Read the length of the image as a 32-bit unsigned int. If the # length is zero, quit the loop image_len = struct.unpack('<L', connection.read(struct.calcsize('<L'))) if not image_len: break # Construct a stream to hold the image data and read the image # data from the connection image_stream = io.BytesIO() image_stream.write(connection.read(image_len)) # Rewind the stream, open it as an image with PIL and do some # processing on it image_stream.seek(0) image = Image.open(image_stream) cv_image = np.array(image) cv2.imshow('Stream',cv_image) if cv2.waitKey(1) & 0xFF == ord('q'): break finally: connection.close() server_socket.close()
If I can get this working with an even lower delay I would like to get an even higher resolution at 60fps. I only need grayscale images on the client, so if I could only send grayscale images on the server side this should also give me some more headroom.