rsync is working great, with the proper exclusions so it can be run from my Raspbian systems that are on 24x7. But it thinks it is the only program on the network and tries to take over.

Of course, I use Win32DiskImager every few months to make sure I have a known, good copy of the setup for each of my 7 Raspbian sensor systems scattered about. Some of these systems operate outside in the SoCal heat. SD cards work great, until they don't.

Every night at midnight the cron scripts fire up to collect the day's data and put it all on hard disk.

To protect against forgetting the system files (like in /etc) that I have changed since the last SD card backup, I run rsync about once a week with options that make it act a lot like Windows Robocopy with the /MIR option.

However, it is really hogging the bandwidth of the PiNet subnet. Enough to block other traffic.

It also seems to stop for many minutes at kern.log while pushing more than 3-4 Mbps, for a long time.

When I look at kern.log it is usually just informational messages having to do with the WiFi drivers - useless information.

How do I solve this?


Turns out that rsync has a built-in throttling mechanism.

Simply add --bwlimit=1000 and this will limit it to 1 Mbps. Use whatever speed works for you.

Then by using the command

du -sch /var/log

it was obvious that log files were huge. About 8 GB.

So then by using

du -sch /var/log/* | sort

it became clear that the kern log files (and their rotations) were taking up about 3.9 GB each.

Plus, messages.log and all the others were taking up another 3.9 GB.

The solution to this is to use a script for the rsync that includes these commands:

sudo rm /var/log/kern*
sudo rm /var/log/messages*

They will get recreated when they are needed again.

This makes for a much happier network and saves space.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.