Project Overview
I have a series of RPi 3B+ which collect T/RH data using connected sensors. These "periphery" units collect data every 5 minutes, append the data to a CSV file that holds a day's worth of data, and stores the data locally in a /data/ directory.
I have a "master" RPi 3B+ whose job is to periodically (approximately every 15 minutes) connect to these periphery units via SSH and copy the available data from the /data/ directory, process it, generate figures, and push these figures to GitHub.
The master RPi has been set up to connect without a password to the periphery units via ssh-copy-id
. I was able to ssh into the periphery units from the master and, through that, stored the periphery units in the /home/pi/.ssh/known_hosts directory.
The Problem
I created a script to automate this process and it works perfectly when I manually run it. The code used to connect to each device and download the data is shown below:
os.system(f'scp -r -o ConnectTimeout=3 pi@{ip_address}:{path_to_raw} {self.path_to_data}/interim/')
I have run the script successfully both within and outside a virtual environment. The problem happens when I try to schedule the script via /etc/crontab. When I do this, the master can no longer connect to the periphery units with the error Host key verification failed.
The /etc/crontab
file is below:
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
#*/5 * * * * root sh /home/pi/bleed-orange-measure-iaq/update_figures.sh
*/5 * * * * root /usr/bin/python3 /home/pi/bleed-orange-measure-iaq/src/data/make_dashboard.py
The first command I have commented out was my attempt to run the script by wrapping it in a shell script but that was also unsuccessful so I figured I would just execute the python command.
The Question
My question is why can I not connect to the periphery units when I place the script within crontab or in a service file? I understand now that the cron and login environments are different, but I have supplied both a PATH
environment variable and used full path names.
ssh
looks for the host key. The host key is stored in theknown_hosts
file. Theknown_hosts
file is stored in/home/pi (or your username)/.ssh/
folder when you usessh
as the normal user. In the cronttab you use the script asroot
. In that casessh
looks forknown_hosts
in the folder/root/.ssh/
. It does not find that file or the particular entry in the fileknown_hosts
forroot
, as the host is "unknown" to theroot
.crontab
withsudo crontab -e
. This created a crontab for theroot
. You can revert the change and usecrontab -e
as the normal userpi
orfritz
or whatever, and put the line there, then thescp
will run as the user, and not root.pi
worked. Thank you for pointing out that there are multiple locations for theknown_hosts
file.