Can I use the PI Camera Stream in Python on a PC?

Question

So I got a Raspberry Pi Zero Wireless with the Pi camera and I would like to use the camera with OpenCV in Python. The problem is, I know I can do this easily on the Pi itself but I highly doubt the Pi will be able to process what I want to do. I might be able to optimize my code sometimes in the future but I don't want to worry about that for now.

So what I wanna do instead is send the camera Data to my PC and run all the Python/Opencv code there. Ideally over Bluetooth. An USB connection to send the data would be sufficient as well.

I can find a lot about streaming video to a PC using VLC but not how to get the data into Python and latency seems to be a problem as well with this method.

If there is no easy solution for this I might just buy a tiny USB cam for now.

EDIT:

So I tried Dave Jones suggestion and went with this: On the Pi I simply use the provided code from rapid-capture-and-streaming and I can get close to 60fps with a decent enough resolution. The code looks like this:

import io
import socket
import struct
import time
import picamera

class SplitFrames(object):
    def __init__(self, connection):
       self.connection = connection
       self.stream = io.BytesIO()
       self.count = 0

def write(self, buf):
    if buf.startswith(b'\xff\xd8'):
        # Start of new frame; send the old one's length
        # then the data
        size = self.stream.tell()
        if size > 0:
            self.connection.write(struct.pack('<L', size))
            self.connection.flush()
            self.stream.seek(0)
            self.connection.write(self.stream.read(size))
            self.count += 1
            self.stream.seek(0)
    self.stream.write(buf)

client_socket = socket.socket()
client_socket.connect(('my_server', 8000))
connection = client_socket.makefile('wb')
try:
   output = SplitFrames(connection)
    with picamera.PiCamera(resolution='853x480', framerate=60) as camera:
        time.sleep(2)
        start = time.time()
        camera.start_recording(output, format='mjpeg')
        camera.wait_recording(30)
        camera.stop_recording()
        # Write the terminating 0-length to the connection to let the
        # server know we're done
        connection.write(struct.pack('<L', 0))
finally:
    connection.close()
    client_socket.close()
   finish = time.time()
print('Sent %d images in %d seconds at %.2ffps' % (
    output.count, finish-start, output.count / (finish-start)))

On the client side I'm basically using the code from capturing-to-a-network-stream with an added cv2.imshow to get a preview. Everything displays fine but with a little bit of delay. Maybe a second or less.

import io
import socket
import struct
from PIL import Image
import cv2
import numpy as np

# Start a socket listening for connections on 0.0.0.0:8000 (0.0.0.0 means
# all interfaces)
server_socket = socket.socket()
server_socket.bind(('0.0.0.0', 8000))
server_socket.listen(0)

# Accept a single connection and make a file-like object out of it
connection = server_socket.accept()[0].makefile('rb')
try:
    while True:
        # Read the length of the image as a 32-bit unsigned int. If the
        # length is zero, quit the loop
        image_len = struct.unpack('<L', connection.read(struct.calcsize('<L')))[0]
    if not image_len:
        break
    # Construct a stream to hold the image data and read the image
    # data from the connection
    image_stream = io.BytesIO()
    image_stream.write(connection.read(image_len))
    # Rewind the stream, open it as an image with PIL and do some
    # processing on it
    image_stream.seek(0)
    image = Image.open(image_stream)
    cv_image = np.array(image)
    cv2.imshow('Stream',cv_image)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

finally:
    connection.close()
    server_socket.close()

If I can get this working with an even lower delay I would like to get an even higher resolution at 60fps. I only need grayscale images on the client, so if I could only send grayscale images on the server side this should also give me some more headroom.

Answer 1

I would use MJPEG-streamer to stream the video feed from the Pi as a motion-jpeg. https://github.com/jacksonliam/mjpg-streamer

Then, see the answers for this question for how to ingest the MJPEG stream from Python: https://stackoverflow.com/questions/21702477/how-to-parse-mjpeg-http-stream-from-ip-camera