7

A common worry of Raspberry Pi users is the wearing out and destruction of their SD card (which have limited write cycles) by too many writes. The experience reported by users supports this assumption.

Logging is one source of frequent write access.

So the question arises how this can be reduced on Raspbian.

  • 2
    You can move the location of the log files to an external drive or flash drive, but IMO these concerns are overblown. – Steve Robillard Feb 26 '17 at 11:22
  • 3
    "The experience reported by users supports this assumption." -> Only if they are devoid of reason. The internet is full of hysterical reports by hysterical people. Bigfoot! UFOs! Logging destroyed my SD card! Etc. Please see unix.stackexchange.com/a/84905/25985 – goldilocks Feb 26 '17 at 11:42
  • 2
    Of more concern to me, as someone who often pulls the plug, is to know nothing will be writing to the SD card when I yank the cord. – joan Feb 26 '17 at 11:51
  • 3
    @SteveRobillard @goldilocks More than to much writes, is the unexpected power loss; this may not be the sd damage culpit but, just in case, I always do a shutdown now and no more SD corruptions. – fcm Feb 26 '17 at 13:53
  • 2
    I agree it's hyperbolic -- but I think that is explicit. That screed is an antitoxin. The point is the vast majority of users do not need to worry about this. People should not consider it fine and normal to operate with a completely filled root filesystem on a pi, their laptop, an inet server, etc; by default ext fs' set aside 5% for root use only to help mitigate against that. So really you are talking about corner cases, and the beef I have here is that you've presented it as if it should be considered an issue by every self-aware citizen. No, it should not. – goldilocks Feb 26 '17 at 17:30
10

If you are not interested in the logs you can switch a lot off using a log configuration setting.

Edit the file /etc/rsyslog.conf and just after the section starting

###############
#### RULES ####
###############

add the following line.

*.*     ~

If you want to be more fine-grained you will need to read the file comments.

Do not forget to restart rsyslog daemon:

sudo service rsyslog restart
  • Easy and clean solution. I commented everything which writes in /var/ out. It should work. – Fusseldieb May 3 '17 at 18:02
  • But I am interested in the logs. They are very important for finding problems of the system! – Frank Breitling Nov 18 '17 at 10:17
  • 2
    If you're interested in logs, there's lots of approaches to retain them, yet avoid SD writes. Send the logs to /dev/shm (RAM), then run logrotate every 300s, and send the logs to Amazon S3. Or Dropbox. Remember, logs have LIMITED value which is diminished over time. You often only really need timely, current logs... – Scott Prive Jan 9 '18 at 19:40
7

My solution for Raspbian 8.0 (Jessie) based on logging to RAM

There already exists the Ramlog Debian package and installation instructions for this. However, this didn't work for me (Starting ramlog-tmpfs 2.0.0: Error: /var/log is in use... [fail]).

Using iotop -bktoqqq I figured out most frequent write access. It turns out that also /var/cache/samba/ is frequently written to. So this also has to go to RAM in addition to /var/tmp/ where the new log files will be.

1. Creating the ramdisk

So first these two entry have to be added to /etc/fstab:

tmpfs           /var/tmp        tmpfs   size=10M,nodev,nosuid     0       0
tmpfs           /var/cache/samba tmpfs   size=5M,nodev,nosuid     0       0

2. The log2disk script

We need to save this script in /usr/local/bin/log2disk which will append and delete the contents from all log files in /var/tmp/log/ to the files in /var/log/.

#!/bin/sh

# Author: Frank Breitling <frank.breitling@gmx.de>
DESC="Moving contents from /var/tmp/log/ to /var/log/"

if [ $(id -u) -ne 0 ]
then echo "Please run as root"
     exit
fi

echo $DESC

exec >>/var/log/log2disk.log 2>&1

date
cd /var/tmp/

for i in log/*; do
    basename $i
    cat $i >>/var/$i
    >$i
done

and make it executable sudo chmod +x /usr/local/bin/log2disk.

3. Adding to crontab

We want to run this script every 3 hours and add this line to the system's /etc/crontab

10 */3  * * *   root    /usr/local/bin/log2disk

(Don't forget a final newline which is needed by crontab.)

4. Installing the log2disk.service

We need to create a systemd service in /lib/systemd/system/log2disk.service that executes this script before shutdown and reboot, so that the log file contents gets preserved:

[Unit]
Description=Write log files to disk
RequiresMountsFor=/
Before=rsyslog.service

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/bin/true
ExecStop=/usr/local/bin/log2disk

[Install]
WantedBy=multi-user.target

and install it with sudo systemctl enable log2disk.

5. Selecting the log files for RAM

Now we can tell /etc/rsyslog.conf which logfiles to keep in RAM. These files are auth.log, syslog, daemon.log, user.log and messages and we replace for each of their entries the log/ path by tmp/log/ for example like this:

auth,authpriv.*                 /var/tmp/log/auth.log

Done!

After a reboot, the system will now log the most frequent log entries to /var/tmp/log and sync them back every 3 hours and before shut down.

We can use iotop again to find a significantly reduced write activity. However we should not be worried about the green ACT LED flashing. Apparently this is not a good write access indicator.

  • 2
    "I have worked out the following solution for Raspbian 8.0 (Jessie) based on the idea of logging to ram." -> Seeing as how Jessie is systemd based, it is already a few tweaked config lines away from that via journald. Have a look at man journald.conf and note Storage=volatile, etc. Then just plain disable rsyslog (it's fed from journald now anyway) and presto, RAM based logging with lots of bells and whistles and customization options. Without any fuss. – goldilocks Feb 26 '17 at 17:35
  • @goldilocks I wonder how this could be useful for creating such a log file buffer that I have created. – Frank Breitling Feb 26 '17 at 18:19
  • I'm not sure what you mean. TBH I'm not a fan of journald (mostly because of the binary format), and the reason I know this much about it is this is what I do with it -- disable logging to disk (and leave that to rsyslog), and give it a modest amount of RAM to use as a buffer. If you mean periodically flush it to disk, I don't see why not. – goldilocks Feb 27 '17 at 8:09
  • @goldilocks Then how? – Frank Breitling Feb 27 '17 at 9:05
  • If you've never used it, you should have a look at man journalctl. Like I said I'm an old fashioned syslog guy, but I notice there's a -s --since switch; then you just have to get it to dump it in a way appropriate to redirection to a file (that may not happen by default...if you play around a bit you'll get the picture). – goldilocks Feb 27 '17 at 11:00
0

The way I circumvent this is by installing my root directory to a USB instead of the SD card. I use the SD card only for boot.

This kind of saves me having to worry about writes to my SD Card.

I use berryboot to achieve this.

0

To reduce logfile write operations on the RPi you could set up a centralized log server with rsyslog. If you want to keep the entries, just send the messages to the remote log server, else to /dev/null.

0

Rather than editing rsyslog.conf you can simply stop the service if you want to eliminate all logs.

sudo service rsyslog stop

Then, you can disable it at boot:

sudo systemctl disable rsyslog

to enable it again at boot:

sudo systemctl enable rsyslog
  • 2
    But that stops all logging. – RalfFriedl May 31 at 22:09
  • 1
    You forgot to prefix the commands with sudo. If you use systemctl you should also use it to stop a service with sudo systemctl stop rsyslog.service. – Ingo Jun 1 at 19:21
  • Very good feedback. I updated this answer to reflect both of these comments. – Nick Painter Jun 4 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.