I have a headless Raspberry Pi sitting in a corner, and I'd like to know if something's gone wrong with it, like it's lost a network connection or crashed. I don't need a complex monitoring or logging system, I just want a basic heart beat mechanism to get an email or other notification when something goes wrong.
https://healthchecks.io/ is a great tool for basic monitoring and alerting, and works perfectly for a heartbeat check. Simply create a check with a relatively frequent period (I went with a 30 minute period and 15 minute grace, meaning a single skipped beat won't alert, but two successive will), and then configure a cron job to ping the check every 20 minutes:
# m h dom mon dow command 10,30,50 * * * * curl -fsS --retry 3 https://hc-ping.com/YOUR_HEALTH_CHECK_ID > /dev/null
This will alert if either the network or machine go down for more than an hour.
healthchecks.io also has a neat feature that supports including a small (10kb) payload with the heartbeat, which seems like a nice way to include some basic status details about the machine as well. I created a small script that sends a payload of basic system diagnostics, and will also fail the check on a few other conditions such as not enough disk space.
I'm still experimenting with exactly what information and checks are worth incorporating, I'd welcome any feedback: https://gist.github.com/dimo414/10d6f162fb6d72f517a041d28d92314f
On Debian/Raspbian the default application to monitor a system is watchdog. It is available from the repository and can simply installed with:
rpi ~$ sudo apt install watchdog
By default it will reboot the system if some configurable tests fail. But on the man page of watchdog you will find that it is also possible to send an Email:
If the machine is halted an email is sent to notify a human that the machine is going down. Starting with version 4.4 watchdog will also notify the human in charge if the machine is rebooted.