How to stream from Pi Camera to YOLO model?

Question

I am new to Raspberry Pi and I want to do real-time object detection using YOLOv8 tentatively (will change if the fps is too low). I realize that by using the code below, it can't detect Pi Cam as the source and run into an error. The Pi Cam works fine with libcamera-hello and managed to capture photo everything. I also enabled the legacy camera feature. The code works fine with USB webcam.

Code:

from ultralytics import YOLO

model = YOLO('best.pt')
results = model.predict(source=0, show=True)

print(results)

Error: ConnectionError: 1/1: 0... Failed to read images from 0

I thought of one solution which is to capture image every time and pass to the model to predict, but it only produces an image every 2.5s. I guess maybe the model reinitialize every time it predicts an image again.

Code:

import cv2
from picamera2 import Picamera2
from ultralytics import YOLO
import numpy as np

cv2.startWindowThread()

picam2 = Picamera2()
picam2.configure(picam2.create_preview_configuration(main={"format": 'XRGB8888', "size": (640, 480)}))
picam2.start()

model = YOLO('best.pt')

while True:
    im = picam2.capture_array()

    img = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
    results = model.predict(img)

    for box in results[0].boxes:
        x = int(np.floor(box[0].xywh[0][0].item()))
        y = int(np.floor(box[0].xywh[0][1].item()))
        w = int(np.floor(box[0].xywh[0][2].item()))
        h = int(np.floor(box[0].xywh[0][3].item()))
        
        cv2.rectangle(im, (x, y), (x + w, y + h), (0, 255, 0))

    cv2.imshow("Camera", im)

So, there are two questions in my mind:

1. How to set Pi Cam as the default source=0 to have real-time object detection using YOLOv8?
2. How to prevent reinitialization of the YOLO model for image prediction? (If my guess is correct)

Answer 1

I got an idea:

1. to see if the Pi Cam can be detected by your computer
   in cmd: 'pip install pygame'
   and run the following python code
   import pygame.camera
pygame.camera.init()
camera_id_list = pygame.camera.list_cameras()
print(camera_id_list)
   If the result comes out to be 0 which is the id of your computer own cam, you can't use the Pi cam directly.If there are other numbers in the result, you can try by changing to 'source=1' for example.

2. to get other components provided by raspberrypi
   I don't have an intimate knowledge of this, so I can't offer you some valuable suggestions.

3. to read the picamera document
   click the link: https://picamera.readthedocs.io/en/release-1.13/recipes2.html
   and you can see something helpful for you in the section 4.2 Capturing to an OpenCV object

Answer 2

Check out the Ultralytics docs on this: https://docs.ultralytics.com/guides/raspberry-pi/#quick-start-with-yolov8