I am deploying a raspberry pi in a remote location with a stable internet connection. The pi will run 24/7 and has a few services which output their logs to /var/logs/A.log, /var/logs/B.log etc.

I am planning to write a python script that will be run by crontab every 15 minutes. This script will read through the logs and only send what's not been sent before.

How should I do this? What would your recommendations be?

Due to the fact that the raspberry pi runs on a sd card, I don't know whether it would be that good an idea to use a database as I want to limit the number of writes to the SD card which has a higher rate of failure.

  • You shouldn't do this. First, you should check if someone else had not done it before. Start with Syslog (from year 1980) and its derivatives, or with Logstash / Elastic Stack if you are into more modern things. – techraf Dec 17 '16 at 9:54
  • To add to what @techraf said there are several good remote logservices including loggly, papertrail etc. You can get a more complete list here alternativeto.net/software/loggly – Steve Robillard Dec 17 '16 at 9:58

Well, this:

stat -c "%s" /var/log/syslog

Will give you just the current size of the file in bytes. And this:

tail -c $((A-B)) /var/log/syslog

With the shell will give you the last A - B bytes, presuming "A" and "B" are actual numbers (or variables representing integers).

That's not python, but that's the idea.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.