Python script run from crontab does not recognize USB drive mounted later

Question

I am having a bit of a problem here, I really hope some of you can point me in the right direction:

I made a python3 program running a loop and checking if the file /DATALOGGER/DATALOGUSB/SETUP.py exists, and if it does the program continues. The DATALOGUSB directory is the mount point for a specific USB drive set up in etc/fstab with the UUID of the USB drive.

This all works flawless when I execute the script directly from the terminal - the OS mounts the USB drive in the correct place, the script recognizes that the SETUP.py file suddenly exists, and continues execution.

My issue is that I require the program to run headless and stand-alone, i.e. I need the python3 program to start up upon reboot and just wait for the USB drive. Once the drive is mounted, the program should just continue exactly like when I run it in the terminal. I have tried to start the program during startup both from the crontab and from rc.local - and the program starts up fine (I can see that from the status LEDs the program controls). Unfortunately, when I then afterwards insert the USB drive, the program does not recognize that the file SETUP.py suddenly exists - so it is just stuck waiting for that file. If I check using the desktop GUI, I can see that the USB drive does indeed mount to the correct mount point, but for some reason the python3 program does not recognize it.

I really have no idea what to do now, as to me it seems strange that the program executes correctly from the terminal - but not if it is scheduled via crontab or startet from rc.local upon reboot.

If anyone has any idea as to what might be wrong, please do not hesitate to reply.

Best regards, Rasmus

Minimum example of my python3 script. The functionality lies in the if os.path.isfile('DATALOGUSB/SETUP.py') branch, but the execution never gets here if the script is started from crontab or rc.local:

# Package import
import time
import os
from multiprocessing import Process

shutdown_flag = False

# The core functionalities of the program
def datalogger():
    while True:
        if os.path.isfile('DATALOGUSB/SETUP.py'):
            # Program functionality, but execution never gets
            # here when started from crontab or rc.local
        else:
            time.sleep(0.1)

# Main loop
datalogger_thread = Process(target=datalogger)
datalogger_thread.start()
while True:
    if shutdown_flag:
        shutdown_flag = False
        shutdown()
    time.sleep(0.1)

My etc/fstab file. The last line is my USB-drive set up by its UUID to mount to the described mount point (it is in one line with a blankspace between vfat and rw - I don't know if the format is correctly showed here). It does mount here, but if the DATALOGGER.py program above is started from crontab or rc.local, the program does not recognize the file SETUP.py as existing.

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p6  /boot           vfat    defaults          0       2
/dev/mmcblk0p7  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
UUID=CCE1-5BCB  /home/pi/Documents/DATALOGGER/DATALOGUSB vfat rw,uid=1000,gid=1000,umask=000,nofail 0 0

Finally my rc.local file:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
  printf "My IP address is %s\n" "$_IP"
fi

sudo python3 /home/pi/Documents/DATALOGGER/DATALOGGER.py &

exit 0

Answer 1

crontab (or anything else run at boot) does not know where python3 is - use full paths

Answer 2

Your problem is to start a program at boot when an usb drive is connected. Nowadays we use systemd for this and we only need a 9 liner. systemd will do all things for you: waiting until the drive is available, mount it then and start the script. I use Raspbian Stretch Lite 2018-04-18.

The short story

Assuming the program to start is ~/Documents/DATALOGGER/datalogger.py without multithreading/multiprocessing.

Create a service:

rpi ~$ sudo systemctl edit --force --full datalogger.service

Insert this text into the editor save it and quit the editor:

[Unit]
Description=My Datalogger Service
RequiresMountsFor=/home/pi/Documents/DATALOGGER/DATALOGUSB
[Service]
User=pi
WorkingDirectory=/home/pi/Documents/DATALOGGER
ExecStart=/home/pi/Documents/DATALOGGER/datalogger.py
[Install]
WantedBy=home-pi-Documents-DATALOGGER-DATALOGUSB.mount

Enable the service:

rpi ~$ sudo systemctl enable datalogger.service

Reboot and your program runs as soon as the usb drive is attached, no need to mount it.

If it doesn't work

Stop the service:

rpi ~$ sudo systemctl stop datalogger.service

I have the same entry in /etc/fstab than you and made a small test script that reads all 5 sec. from the usb drive and log it to the journal. Don't use threading/multiprocessing because systemd already works parallel. Threading/multiprocessing within multiprocessing may be confusing.

rpi ~$ grep ^UUID /etc/fstab
UUID=00C1-9714  /home/pi/Documents/DATALOGGER/DATALOGUSB vfat rw,uid=1000,gid=1000,umask=000,nofail 0 0

rpi ~$ mkdir -p Documents/DATALOGGER/DATALOGUSB/
rpi ~$ sudo mount Documents/DATALOGGER/DATALOGUSB/
rpi ~$ echo hello world > Documents/DATALOGGER/DATALOGUSB/hello.txt

Create a test script:

rpi ~$ cat > Documents/DATALOGGER/datalogger.py <<EOF
#! /usr/bin/python3
import time

print("USB-Drive is mounted, start with logging", flush=True)
# read hello.txt and print it
fh = open("/home/pi/Documents/DATALOGGER/DATALOGUSB/hello.txt","r")
while(True):
    print(fh.read(), end="", flush=True)
    fh.seek(0)
    time.sleep(5)
EOF

Test it on the command line.

rpi ~$ chmod u+x Documents/DATALOGGER/datalogger.py
rpi ~$ Documents/DATALOGGER/datalogger.py
USB-Drive is mounted, start with logging
hello world
hello world

Terminate with <ctrl>C

Check the service, enable and start it:

rpi ~$ sudo systemctl edit --full datalogger.service # if required
rpi ~$ systemctl cat datalogger.service
rpi ~$ sudo systemctl daemon-reload
rpi ~$ sudo systemctl enable datalogger.service
rpi ~$ sudo systemctl start datalogger.service
rpi ~$ systemctl status datalogger.service
rpi ~$ journalctl --unit=datalogger --pager-end
Apr 18 02:57:57 raspberrypi datalogger.py[651]: USB-Drive is mounted, start with logging
Apr 18 02:57:57 raspberrypi datalogger.py[651]: hello world
Apr 18 02:58:02 raspberrypi datalogger.py[651]: hello world

Test mounting:

rpi ~$ sudo systemctl stop datalogger.service

umount the usb drive, detach it and start the datalogger.service. The service should wait (don't return to the prompt). Attach the usb drive and the service will start (return to the prompt).

Reboot.

Answer 3

You could just add a sleep in your rc.local script to allow the drive mounting process to occur:

sleep 60; sudo python3 /home/pi/Documents/DATALOGGER/DATALOGGER.py &