Creating symbolic link in RASPBIAN (reduce I/Os on memory card)

Question

I'm doing this to reduce I/Os on memory card of raspberry pi. I want to redirect logs under /var/log to a mounted external drive. I am using following commands in the stated order to create the links. There are 2 solutions i'm exploring.

Method: 1

sudo /etc/init.d/rsyslog stop
ln -fs /var/log/messages 
/path/to/mount/messages
sudo /etc/init.d/rsyslog stop

I have removed files in case this doesn't work using rm command. My primary problem is reboot and remounting of drives. Whenever I reboot the RPi I need to mount the drives again. However, the logs start getting written to /var/log and I have to go through the trial and error process again. I have edited logging path in software's wherever possible. However for system processes and logs such as messages, mail, wtmp, debug i am unable to find a solution.

What is the most robust way to ensure I log on the external drive all the time. /mnt/path/ all the time? including for system applications?

Method 2:

This is where I mount my external device to /var/log folder using fstab. I've taken a backup of my fstab file using sudo cp -p /etc/fstab /etc/fstab.17.11.2018

Following is what the file contents are:

proc            /proc           proc    defaults          0       0
PARTUUID=dd5ad381-01  /boot           vfat    defaults          0       2
PARTUUID=dd5ad381-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

Following is the output of blkid

user@hostname:~# sudo blkid
/dev/mmcblk0p1: LABEL="boot" UUID="FBD8-71DF" TYPE="vfat" PARTUUID="dd5ad381-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="e9646bf0-ef1f-4e8b-983b-c9f97f60e931" TYPE="ext4" PARTUUID="dd5ad381-02"
/dev/mmcblk0: PTUUID="dd5ad381" PTTYPE="dos"
/dev/sdb1: LABEL="logs" UUID="50043501-276b-473d-a6a5-bda12a845d67" TYPE="ext4" PARTUUID="1068b060-01"
/dev/sda1: LABEL="label" UUID="f1da78dc-d69b-4902-9646-f1719b637634" TYPE="ext4" PARTLABEL="part-label" PARTUUID="f8a547cb-9870-49ee-8055-70f7ff025926"

The current directory I mount the sdb1 is /mnt/logs. I am ok changing this. How should my line for the logs file should look like? Is the following correct?

PARTUUID=1068b060-01 /var/logs ext4 sync,auto,nodev,noexec,suid,rw,nouser, 0 2

I wanted to run this by the community given the sensitivity of the operation. I was reading the man page at (https://help.ubuntu.com/community/Fstab#Device). I am not sure but can I lock down the mount for only administrator? I don't see a log file that is for anyone but the root anyway.

Following is the content of /etc/rsyslog.conf file. I've tried doing this but it doesn't work too. Is it because I've created symbolic links?

user@hostname:~ $ cat /etc/rsyslog.conf
#  /etc/rsyslog.conf    Configuration file for rsyslog.
#
#                       For more information see
#                       /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html


#################
#### MODULES ####
#################

module(load="imuxsock") # provides support for local system logging
module(load="imklog")   # provides kernel logging support
#module(load="immark")  # provides --MARK-- message capability

# provides UDP syslog reception
#module(load="imudp")
#input(type="imudp" port="514")

# provides TCP syslog reception
#module(load="imtcp")
#input(type="imtcp" port="514")


###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

#
# Set the default permissions for all log files.
#
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022

#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf


###############
#### RULES ####
###############

#
# First some standard log files.  Log by facility.
#
auth,authpriv.*                 /mnt/logs/auth.log
*.*;auth,authpriv.none          -/mnt/logs/syslog
#cron.*                         /mnt/logs/cron.log
daemon.*                        -/mnt/logs/daemon.log
kern.*                          -/mnt/logs/kern.log
lpr.*                           -/mnt/logs/lpr.log
mail.*                          -/mnt/logs/mail.log
user.*                          -/var/log/user.log

#
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info                       -/var/log/mail.info
mail.warn                       -/var/log/mail.warn
mail.err                        /var/log/mail.err

#
# Some "catch-all" log files.
#
*.=debug;\
        auth,authpriv.none;\
        news.none;mail.none     -/var/log/debug
*.=info;*.=notice;*.=warn;\
        auth,authpriv.none;\
        cron,daemon.none;\
        mail,news.none          -/var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg                         :omusrmsg:*

The reason of doing this is because I'm running a web server on my RPi and the logs are in excess of 2 GB a day.

Answer 1

Although I've pointed out elsewhere the motivation here is misguided, a simpler way to do this is to just change the rsyslog configuration. Here's a section of the default /etc/rsyslog.conf:

# First some standard log files.  Log by facility.
#
auth,authpriv.*                 /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
daemon.*                        -/var/log/daemon.log
kern.*                          -/var/log/kern.log
lpr.*                           -/var/log/lpr.log
mail.*                          -/var/log/mail.log
user.*                          -/var/log/user.log

You will similarly want to deal with the entries under "catch-all" log files below this in the configuration file.

You can change those paths then sudo systemctl restart syslog. Rsyslog has been around for a long time (a decade more than the Pi), and the syslog protocol (of which it implements a superset of features) decades more than that, so you should easily be able to fine lots of documentation online and on the system if you are interested. Rsyslog also has an HTML doc package, but man rsyslog.conf is best place to start.

There may or may not be boot issues for you this way; make sure the partition is automounted in /etc/fstab.

Is the following correct?

PARTUUID=1068b060-01 /var/logs ext4 sync,auto,nodev,noexec,suid,rw,nouser, 0 2

You may want noatime in there too, depending on how you feel about saving wear on the drive ;P and how important getting access times is to you. But that should work. Maybe the last field should be 3, I think those should be unique (probably better would be to make the boot partition 3 and this one 2).

A problem with symlinking the files is that these files are periodically rotated.

Answer 2

Following if the final configuration that fulfills all my requirement.

ls -l /dev/disk/by-uuid/
#note the UUID
#make backup of your  fstab file. I add date and time to name of any file to make it a backup file. You can do something different such as .backup
sudo cp -p /etc/fstab /etc/fstab.25.11.2018.1425
sudo nano /etc/fstab
#Add the line UUID=bca6c466-d4aa-44b8-9696-a11b610ec47c /mnt/1/ ext4    auto,noexec,rw,sync,nouser,nosuid,nofail    0   2
#you can read about the options I've added here - https://wiki.debian.org/fstab
#now to move the logging configuration to the new folder. Please keep in mind, you will have to do this for all the application(s) you have. My configuraiton is for the system and UFW (firewall).
#first stop the rsyslog sevice
sudo /etc/init.d/rsyslog stop
#backup the rsyslog coniguration
sudo cp -p /etc/rsyslog.conf /etc/rsyslog.conf.24.11.2018.2249
#edit the file
Sudo nano /etc/rsyslog.conf
#edit the directories to your mount point for me it is /mnt/1
#edit the UFW configuration
cd /etc/rsyslog.d
#backup the file
sudo cp -p 20-ufw.conf 20-ufw.conf.25.11.2018.1434
#edit the file
sudo nano 20-ufw.conf
#point the log files to your mount point (folder)
# start the service again
sudo /etc/init.d/rsyslog start


To check the configuraiton reboot your pi
sudo reboot now