always-on security camera, uploading to cloud, looping video recording/saving

Question

I'm trying to get started on a raspberry pi security camera project. It would function akin to a dash camera, with the following features:

• Always recording and saving 3-5 minute files
• Once the folder where these files are stored reaches a certain size, the oldest files are deleted and replaced with the newer files (goal being the camera will always have the past ~3 days)
• Uploading to OneDrive or Google Drive (on the same cycle above)

Are there any options out there currently? Everything I'm finding has motion as the most important factor, but that's really not important for my situation. The end goal is to have a few cameras setup inside and always recording entryways, etc.

Answer 1

This is fairly easy to accomplish with the picamera API, specifically the record_sequence method. I won't tackle the cloud upload bit as that'll bloat this example horribly, but it's pretty easy to add Google Drive integration (they've got plenty of docs on using Python to interface to Google Drive), and I'll point out where you'd want to add this.

Firstly, some assumptions given this is for a security camera:

• 720p is "good enough" resolution. I'd recommend not using 1080p as it has a limited field of view (esp. with V2 modules)
• 15fps is a "good enough" framerate. Most security cameras use a slower framerate to reduce storage requirements, but in the case of the Pi's camera module with its rolling shutter this also allows it to use a longer exposure when it needs to (e.g. in low light situations).
• We want to output "proper" MP4 files so they'll play back at our slower framerate (see FAQ 13)
• We want to limit the bitrate and quality fairly strictly. If cloud upload is going to be added at some point we'll need to be producing video at a rate slower than our available upstream bandwidth. This is why I've selected 1Mbps as the bitrate limit in the example (drop the quality parameter to 25 and you could probably decrease this further).

The following script defines a small custom output called VideoFile which uses VLC in a sub-process to mux the H264 into a "proper" MP4, and provides methods and properties for querying the size of the resulting file and removing it when necessary. It uses an infinite generator (outputs) to construct instances of VideoFile, and finally some simple logic in the main loop to track when the total size of video files exceeds a specified limit.

import io
import os
import logging
import datetime as dt
from subprocess import Popen, PIPE, DEVNULL
from picamera import PiCamera


DESTINATION = '.'
RESOLUTION = '1280x720'
FRAMERATE = 15 # fps
BITRATE = 1000000 # bps
QUALITY = 22
CHUNK_LENGTH = 300 # seconds
SIZE_LIMIT = 1024 * 1048576 # bytes


class VideoFile:
    def __init__(self, dest=DESTINATION):
        self._filename = os.path.join(
            dest, dt.datetime.utcnow().strftime('CAM-%Y%m%d-%H%M%S.mp4'))
        # Use a VLC sub-process to handle muxing to MP4
        self._process = Popen([
            'cvlc',
            'stream:///dev/stdin',
            '--demux', 'h264',
            '--h264-fps', str(FRAMERATE),
            '--play-and-exit',
            '--sout',
            '#standard{access=file,mux=mp4,dst=%s}' % self._filename,
            ], stdin=PIPE, stdout=DEVNULL, stderr=DEVNULL)
        logging.info('Recording to %s', self._filename)

    def write(self, buf):
        return self._process.stdin.write(buf)

    def close(self):
        self._process.stdin.close()
        self._process.wait()
        # If you want to add a cloud upload, I'd suggest starting it
        # in a background thread here; make sure it keeps an open handle
        # on the output file (self._filename) in case it gets deleted

    @property
    def name(self):
        return self._filename

    @property
    def size(self):
        return os.stat(self._filename).st_size

    def remove(self):
        logging.info('Removing %s', self._filename)
        os.unlink(self._filename)
        self._filename = None


def outputs():
    while True:
        yield VideoFile()


logging.getLogger().setLevel(logging.INFO)
with PiCamera(resolution=RESOLUTION, framerate=FRAMERATE) as camera:
    files = []
    last_output = None
    for output in camera.record_sequence(
            outputs(), format='h264',
            bitrate=BITRATE, quality=QUALITY, intra_period=5 * FRAMERATE):
        if last_output is not None:
            last_output.close()
            files.append(last_output)
            total_size = sum(f.size for f in files)
            while total_size > SIZE_LIMIT:
                f = files.pop(0)
                total_size -= f.size
                f.remove()
        last_output = output
        camera.wait_recording(CHUNK_LENGTH)

Some things to note:

• I've assumed Python 3
• The length of each clip won't be precisely the specified chunk length. Files will only split on an intra-frame and to improve compression ratios, I've allowed a long intra-period (5 times the framerate so there'll only be an intra-frame scheduled every 5 seconds, although record_sequence does request intra-frames when it iterates).
• The specified file-size limit will be exceeded by the currently recording chunk, but hopefully that's minimal compared to the overall limit. Keeping the video recordings strictly within the defined limit is actually quite hard (for the reasons mentioned above you can't split a recording precisely on demand; you have to request and wait for an intra-frame).
• UTC timestamps are used to generate the filenames so that if the Pi's clock rolls back (at the end of daylight savings) we don't accidentally clobber earlier recordings.