Why is my python script stopping after 1 month?

Question

I'm running three python scripts on a Pi 4 Model B. These scripts are ran from boot using the '.config/autostart function'. I have a batch script placed in 'autostart' which then in turn starts the three required python scripts in succession. This batch script can be seen below:

import os
import RPi.GPIO as GPIO
import time
from sys import exit

os.system("python3 LoggerChannel1.py & python3 LoggerChannel2.py & python3 shutdown.py")

The issue I'm having is with the 'LoggerChannel1.py' and 'LoggerChannel2.py' scripts. If I start up the pi, all of the scripts run from boot as they should and they function as they should for around a month. After they have been running for a month, these two scripts both stop. When in this stopped state, the Pi itself is still fully functional and the 'shutdown.py' script still works... as this script is also ran from 'batchscript.py' I do not think the issue lies with the batch script.

'LoggerChannel1.py' and 'LoggerChannel2.py' are duplicates of the same script apart from different variables such as different assigned GPIO pins etc. The scripts basically work by sampling the state of a GPIO pin each second and recording its state as a 0 or 1 into a list, after 300 seconds, all entries into the list are summed and this number, along with a timestamp, is recorded into a csv file. The list is then cleared and the loop starts again. The 'LoggerChannel1.py' script can be seen below:

import RPi.GPIO as GPIO
import time
from datetime import datetime
from sys import exit
import os

#This configures BCM GPIO pinout and disables associated GPIO warnings.
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

#This defines the pin used for the stop push button and configures it as a GPIO input.
StopLogger = 5
GPIO.setup(StopLogger, GPIO.IN)

#This pin defines the pin used for the 'Logger Running' indicator LED and configures it as a GPIO output.
LoggerRunning = 16
GPIO.setup(LoggerRunning, GPIO.OUT)

#This defines the pins used for digital inputs and outputs for channel 1 and channel 2. 
DI01 = 17
DI02 = 27
DO01 = 23
DO02 = 24

#This deduces which channel is to be used, if activechannelIN = DI01 then channel 1 will be used, if activechannelIN = DI02 then channel 2 will be used.
activechannelIN = DI01

#This ensures that if channel 1 input is selected then channel 1 output will be selected and similarly, that if channel 1 input is selected then channel 1 output will be selected.
if ((activechannelIN) == DI01):
    activechannelOUT = DO01
if ((activechannelIN) == DI02):
    activechannelOUT = DO02

iomap = {'17':'Channel 1', '27':'Channel 2', '23':'Output 1', '24':'Output 2'}
GPIO.setup(activechannelIN, GPIO.IN)
GPIO.setup(activechannelOUT, GPIO.OUT)
logdir = "/home/pi/shared/"
 
def logToFile(filename, content):
    with open(logdir + filename, "a") as logfile:
        logfile.write(content + "\n")
 
lst = "sample_list" 
#This is the list in which the rawstate samples are collected and summed.
sample_list = []
#This is the time interval between rawstate samples in seconds.
sample_frequency = 1
#This is the time interval between each log to the csv file in seconds.
logging_frequency = 300
#This is the number of loop iterations after which the sum of sample_list will be logged to the csv file. For as long as sample_frequency = 1, the logging_time can be seen as the logging_frequency minus 1.
logging_time = ((logging_frequency)-1)


while True:
    restart = False
    
    for i in range(0,logging_frequency):
        time.sleep(sample_frequency)
        rawstate = GPIO.input(activechannelIN)
        sample_list.extend([int(rawstate)])
        print(sample_list)
        print (i)
        GPIO.output(LoggerRunning, True)
        if (GPIO.input(StopLogger) == True):
            print("stop button pressed")
            GPIO.output(LoggerRunning, False)
            GPIO.cleanup()
            import loggerstopped.py
            break
        if i == (logging_time):
            print (i)
            csvlog = sum(sample_list)
            channelname = iomap[str(activechannelIN)]
            timestamp = datetime.now().strftime("%d/%m/%Y %H:%M:%S")
            record = timestamp + "," + channelname + "," + str(csvlog)
            print (record)
            logToFile("LoggerChannel1.csv", record)
            sample_list.clear()
            restart = True
            break
        elif (GPIO.input(activechannelIN) == True):
            GPIO.output(activechannelOUT, True)
        elif (GPIO.input(activechannelIN) == False):
            GPIO.output(activechannelOUT, False)
        elif (GPIO.input(StopLogger) == True):
            print("stop button pressed")
            GPIO.cleanup()
            import loggerstopped.py
            break
    if not restart:
        break

I feel like this issue has something to do with the list that is formed getting too large since it is happening every time after about a month of running, however,I was under the impression that my use of 'list.clear()' prevented this happening?

Apologies in advance if the code is awful or I'm not making much sense with my questions... this is my first project with python and coding in general and I would appreciate any advice you can give me.

Answer 1

Let's keep it short. Your scripts could be failing for so many reasons. If you had a stack trace or an error message of some sort, we could at least speculate on what is going wrong but we have nothing.

This code is not optimal nor robust. Two things are lacking right now:

1. error handling
2. logging

This is all the more important when your scripts are running unattended. You can't debug efficiently without a trace.

My suggestion is to wrap your code inside a try/catch/finally like this:

import sys
import logging

from pathlib import Path

LOG_FILE_PATH = "/tmp"
LOG_FILE_NAME = "myapp.log"


# set up logger
logger = logging.getLogger(__name__)


def main():

    # the logger shall emit all messages regardless of severity
    logger.setLevel(logging.DEBUG)

    # console handler: show DEBUG messages (and higher)
    stdout_handler = logging.StreamHandler(sys.stdout)
    stdout_handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
    stdout_handler.setFormatter(formatter)
    logger.addHandler(stdout_handler)

    # file handler, show all messages (level DEBUG and higher)
    log_file = Path(LOG_FILE_PATH) / LOG_FILE_NAME

    file_handler = logging.FileHandler(filename=log_file)
    file_handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter(
        '%(asctime)s\t%(filename)s\t%(lineno)s\t%(name)s\t%(funcName)s\t%(levelname)s\t%(message)s')
    file_handler.setFormatter(formatter)
    logger.addHandler(file_handler)

    logger.info("Starting program")
    logger.debug(f"Using log file: {log_file}")

    # do your stuff here


try:
    if __name__ == '__main__':
        main()

except KeyboardInterrupt:
    logger.warning("Shutdown requested (KeyboardInterrupt)...")
    sys.exit(0)

except Exception:
    logger.error("Exception", exc_info=True)
    sys.exit(1)

finally:
    logger.debug("Running finally")

What this code does:

• it logs messages to the console
• it logs messages to a log file (with additional fields)
• it implements a basic exception handling routine

If an error occurs, then at least you'll have a trace on file. The stack trace should make it pretty obvious where the error lies.

When you have implemented logging properly you can also replace your prints and logToFile with the appropriate logging functions eg logging.info

For the sake of demonstration I am logging to /tmp but I suggest you log to persistent storage instead (SD card) eg /home/pi, depending on which user is running the script (verify permissions). If you shut down the device, the data in /tmp will be lost (as it is tmpfs).

I don't really know what your scripts are doing, certainly you could simplify the whole setup. You are saying you have duplicates so they can be consolidated into one single script, which could cause fewer issues. You could also implement your script as a systemd service, especially if you want to manage dependencies or run commands in a specific order. There should be many threads touching on the subject.

In addition to what has already been said above, you could also run your scripts in detached screen or tmux sessions, so you have the opportunity to "attach" to the console at any time, and see what the script is doing (or witness a crash).

In conclusion, I recommend that you use a skeleton along these lines for any Python script. This will save you time and headaches.