0

I am getting the camera feed from a camera on a Raspberry PI to the computer using a TCP socket to use it for openCV. It works well with minimal delay. However, sometimes the frame gets stuck(freeze) and after a while the feed gets back or the openCV window crashes. I have looked everywhere and tried multiple things, but I just don't know what is causing it.

Server (PC):

import socket
import struct
import numpy as np
import cv2

host = "192.168.0.12"
portCar = 8010

# Camera socket
camS = socket.socket()
camS.bind((host, portCar))

camS.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

# Listen for camera
camS.listen(0)
print("Waiting for camera connection...")

camCon = camS.accept()[0]
camFile = camCon.makefile("rb")
print("Connection made with camera")

camS.settimeout(0.00001)

numOfBytes = struct.calcsize("<L")

try:
    while(True):
        camS.setblocking(False)

        imageLength = struct.unpack("<L", camFile.read(numOfBytes))[0]    

        if imageLength == 0:
            break

        nparr = np.frombuffer(camFile.read(imageLength), np.uint8)
        frame = cv2.imdecode(nparr, cv2.IMREAD_COLOR)

        cv2.imshow('RC Car Video stream', frame)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

finally:
    camFile.close()
    camS.close()
    cv2.destroyAllWindows()
    print("Server - Camera connection closed")

Client (PI):

import io
import socket
import struct
import time
import picamera

client_socket = socket.socket()
client_socket.connect(('192.168.0.12', 8010))

client_socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

connection = client_socket.makefile('wb')

class SplitFrames(object):
    def __init__(self, connection):
        self.connection = connection
        self.stream = io.BytesIO()

    def write(self, buf):
        if buf.startswith(b'\xff\xd8'):
            size = self.stream.tell()
            if size > 0:
                self.connection.write(struct.pack('<L', size))
                self.connection.flush()
                self.stream.seek(0)
                self.connection.write(self.stream.read(size))
                self.stream.seek(0)

        self.stream.write(buf)


try:
    output = SplitFrames(connection)
    with picamera.PiCamera(resolution='VGA', framerate=30) as camera:
        time.sleep(2)
        camera.rotation = 180
        camera.start_recording(output, format='mjpeg')
        camera.wait_recording(2000)
        camera.stop_recording()
        # Write the terminating 0-length to the connection to let the
        # server know we're done
        connection.write(struct.pack('<L', 0))
finally:
    connection.close()
    client_socket.close()
    print("Client - Connection closed")

Any help will be greatly appreciated.

  • 1
    "have looked everywhere and tried multiple things" - any details on that would be helpful. – Dmitry Grigoryev Mar 7 at 7:30
  • 1
    @DmitryGrigoryev As you can see from the code I am trying to use .settimeout(0.00001) and I tried catching that after .read(). That did not work. I also tried using select, I either did that wrong or it did not work. I also am setting to non blocking mode. That did nothing. – Rino Leonardino Mar 7 at 11:31
0

Do you have any TCP errors or retransmissions on the active network interface? It may be that your connection is so poor that TCP ends up losig data and then your program crashes since there's no code handling data loss. Check out the output of netstat -s | grep retransmitted and ifconfig | grep error some time after activating the network interface (ideally, after a crash) to confirm.

  • Thank you for your reply. It doesn't really crash. When it gets "stuck", the window says "Not responding" and then if I leave to for a while, about 5-10 seconds, the camera feed will come again after skipping(speed) through the frames it missed. – Rino Leonardino Mar 7 at 13:52
  • I did run the netstat -s | grep retransmitted and got 335 segments retransmitted. – Rino Leonardino Mar 7 at 13:54
  • RX errors 0 dropped 0 overruns 0 frame 0 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 RX errors 0 dropped 0 overruns 0 frame 0 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 RX errors 0 dropped 1 overruns 0 frame 0 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 That is the output of the ifconfig | grep error command – Rino Leonardino Mar 7 at 13:55
  • Hello. Just wondering if you have any more questions or you think what is wrong. – Rino Leonardino Mar 10 at 22:03
  • @RinoLeonardino No, sorry. Debugging this in person could already be a challenge, not to say about remotely guessing what could be wrong. – Dmitry Grigoryev Mar 11 at 8:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.