I recorded an uncompressed / un-encoded video in yuv format using camera of my raspberry pi. I want to read each frame as an image and also count the total number of frames of the video.

I found this answer : https://stackoverflow.com/questions/2231518/how-to-read-a-frame-from-yuv-file-in-opencv But it looks old and doesn't seem to work.

Do I have to convert yuv video to some other format first to read each frame. How should I go about it?
Which tool will be better for this, OpenCV or Matlab or some other?

  • 1
    have you looked at ffmpeg? it supports YUV and can convert a video into a set of images.
    – rob
    Feb 24 '15 at 11:00

Let's tackle counting frames before anything else as that's relatively easy. The Pi's YUV format is YUV420p which uses 1.5 bytes per pixel, so as long as you know the resolution you recorded at and the size in bytes of the final recording, we can easily work out how many frames there are.

For example, I've just made a test recording in YUV format on my Pi at a resolution of 320x240, and the resulting file is ~167Mb, so the number of frames is:

174988800 / (320 * 240 * 1.5) = 1519 frames

It's worth noting though, that the Pi rounds certain resolutions up a bit. The relevant info is in this recipe in the picamera docs. You'll need to adjust the formula above if you use a resolution that involves such rounding.

Now, for reading the frames we just need to remember that OpenCV's image format is planar BGR so we need to read a YUV frame, convert it to BGR and pass it to OpenCV. Thankfully the code in that same recipe can help us do that pretty easily (actually it converts YUV to RGB, but we can easily swap the planes in the conversion matrix to make it produce BGR). The following code reads the fourth frame from the file, converts it to BGR and displays it with OpenCV:

import numpy as np
import cv2

width = 320
height = 240

stream = open('test.yuv', 'rb')
# Seek to the fourth frame in the file
stream.seek(4 * width * height * 1.5)
# Calculate the actual image size in the stream (accounting for rounding
# of the resolution)
fwidth = (width + 31) // 32 * 32
fheight = (height + 15) // 16 * 16
# Load the Y (luminance) data from the stream
Y = np.fromfile(stream, dtype=np.uint8, count=fwidth*fheight).\
        reshape((fheight, fwidth))
# Load the UV (chrominance) data from the stream, and double its size
U = np.fromfile(stream, dtype=np.uint8, count=(fwidth//2)*(fheight//2)).\
        reshape((fheight//2, fwidth//2)).\
        repeat(2, axis=0).repeat(2, axis=1)
V = np.fromfile(stream, dtype=np.uint8, count=(fwidth//2)*(fheight//2)).\
        reshape((fheight//2, fwidth//2)).\
        repeat(2, axis=0).repeat(2, axis=1)
# Stack the YUV channels together, crop the actual resolution, convert to
# floating point for later calculations, and apply the standard biases
YUV = np.dstack((Y, U, V))[:height, :width, :].astype(np.float)
YUV[:, :, 0]  = YUV[:, :, 0]  - 16   # Offset Y by 16
YUV[:, :, 1:] = YUV[:, :, 1:] - 128  # Offset UV by 128
# YUV conversion matrix from ITU-R BT.601 version (SDTV)
# Note the swapped R and B planes!
#              Y       U       V
M = np.array([[1.164,  2.017,  0.000],    # B
              [1.164, -0.392, -0.813],    # G
              [1.164,  0.000,  1.596]])   # R
# Take the dot product with the matrix to produce BGR output, clamp the
# results to byte range and convert to bytes
BGR = YUV.dot(M.T).clip(0, 255).astype(np.uint8)
# Display the image with OpenCV
cv2.imshow('image', BGR)

It shouldn't be too tricky to adapt this to loop over the frames of the file, reading and converting each in turn (obviously it'll be a bit slow on a Pi, but you could run this anywhere).

  • This works but what if I want to extract and save all the frames of the video. Do I have to run a loop ?
    – Coderaemon
    Feb 25 '15 at 12:59
  • Yup, given that one run through the script reads and produces a single frame you could stick everything after the open() call in a while loop and it would happily run through each frame in the file (until it finally hit EOF which would probably raise some sort of IOError during the np.fromfile call)
    – Dave Jones
    Feb 25 '15 at 22:58
  • I get ` Y = np.fromfile(stream, dtype=np.uint8, count=fwidth*fheight).\ ValueError: cannot reshape array of size 0 into shape (2464,3296)` Jan 24 '21 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.