Send image from one raspberry pi to another via socket

Question

I am new to posting questions on this site so forgive me if this isn't in the best place or if I do something wrong.

I have two Pi's and I am trying to capture images in a loop (giving me a video), with the Raspberry Pi camera and send each through a socket to the other Pi and display them with pygame. I will also need low latency.

I have tried many different methods and think I have gotten fairly close. The code I have right now is posted at the bottom. There are however, a lot of other things I have tried. For example, I tried using the PiCamera module to capture an image and then make it a PIL image right away (instead of capturing an image with pygame). The code below seems to be fairly close but I can only seem to get the delay to about 5 seconds.

I also tried using UDP but could not get time image.tostring small enough to send through the socket.

I realize there are other ways to do this, like mjpg-streamer but I wanted something a bit less complicated that I could understand.

Any help would be appreciated!

server:

import socket
import pygame as game
import sys
from PIL import Image

port=9012
address="192.168.1.10" #server's ip
size=width, height= 640, 480
scale=width, height=40, 10
timer = 0
previousImage = ""
image = ""
message=[]

#establish socket connection
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((address, port))
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)



#initialize pygame
game.init()
screen=game.display.set_mode(size)
game.display.set_caption('Socket Raspi Camera Viewer')
#clock = game.time.Clock()

#listen on the port
s.listen(1)


#main loop
while 1:
  for event in game.event.get():
    if event.type == game.QUIT:
      sys.exit()

  #accept socket connection
  connection, addr= s.accept()

  #Recieve data from the server:
  data=connection.recv(1024000)
  print(sys.getsizeof(data))

  #store the previous recieved image incase the client fails to recive all of the data for the new image
  previousImage=image

  #We use a try clause to the program will not abort if there is an error:
  try:

    #We turn the data we revieved into a 120x90 PIL image:
    #image = Image.fromstring("RGB",(120,90),data)

    #We resize the image to 640x480:
    pil_image = Image.fromstring("RGBA", size, data)
    pil_image= pil_image.resize(size)
    new_data=pil_image.tostring()
    image=game.image.fromstring(new_data, size, "RGBA")

  except:
    #If we failed to recieve a new image we display the last image we revieved:
    print("falied to recieve image")
    image = previousImage

  #Set the var output to our image:    
  #output = image

  #We use PyGame to blit the output image to the screen:
  screen.blit(image,(0,0))

  #We update the screen:
  game.display.flip()`

client:

import pygame as game
import pygame.camera
import sys
import time
from PIL import Image


port=9012 
address="192.168.1.10" #server's ip
size=width, height=  640, 480
scale=width, height= 40, 10
timer=0

game.init()
game.camera.init()
screen=game.display.set_mode(size)

camera=game.camera.Camera("/dev/video0", size )
camera.start()
time.sleep(1)
while True:
  if timer < 1:
    #connect to the server
    s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((address, port))

    #get image
    image=camera.get_image()

    #convert to pill image to scale
    string_image=game.image.tostring(image, "RGBA",False)
    pil_image=Image.fromstring("RGBA",size,string_image)
    pil_image =pil_image.resize(scale)
    buffer=pil_image.tostring()

    #print size of buffer
    print(sys.getsizeof(buffer))
    s.sendall(buffer)
    #time.sleep(0.01)
    s.close()
    timer=30

#

Stream without resizing: client-

while True:
  image=camera.getimage()
  buffer=pygame.image.tostring(image)
  socket.send(buffer) 

Server:

while True:
   message=[]
      while True:
         string=socket.recieve(1024 * 1024)

         if string == '':
            break

         else:
            message.append(string)

      full_string="".join(message)
      image=game.fromstring(full_string)
      game.blit(image, (0,0))
      game.display.update()

Answer 1

I must confess that I know nothing about image processing from the Pi camera and I've only played around a little bit with Python. Yet some things look suspicious about your code.

You seem to be creating a socket connection, sending one image (still frame) down the pipe, closing the socket, then repeat. You don't want to be doing it this way. Too much overhead and I think you will quickly run out of socket numbers. On the server side, the part with the listen and accept: you only want to do this ONCE. i.e. Do your listen and wait for the client side to connect, then server accepts, THEN you want to stay in your loop sending and receiving image frames. Only when you are all done do you want to close the socket. The way you are doing it (I think) is creating tremendous overhead in the comms portion of your system. You may want to invent a little protocol for yourself so that after processing a received image frame, the server sends (for example) "OK" back to the client to say "last frame good, ready for next". The client waits for "OK" before sending another frame.

Likewise, with the image processing, again please forgive me if I'm talking out of turn because I don't know anything about this specifically, but I think you want to get the data from the camera and transmit it immediately, not be messing around resizing, converting to/from strings, etc, on the client side. Let the server do the needed transformations because he is the guy who is going to display it. In my opinion, as soon as you camera.get_image you should have the data for one image sitting in your buffer "image". SEND IT. As long as you know the length of your data, sockets should have no problem sending it as binary data. You don't need to convert to string (unless that is the only way you can find out the length). All those conversions and resizes are expensive and that has to be killing your performance. Just get the data from the camera and do as little as possible and then send it. Let the server side monkey around with it.

Here again, DON't CLOSE YOUR SOCKET. Keep using it!

That's my two cents. Good luck.