RasPi resets fake-hwclock time on USB Stick on reboot (Overlay FS)

Question

Introduction:

To make sure my RasPi has the longest possible live duration, i've enabled read-only boot partition as well as the overlay filesystem.

I want to save the fake-hwclock.data (current system time) to a mounted USB, so that when the machine reboots, it starts on the "current" time (last time before reboot) and not the time from before the overlay filesystem got enabled.

To make sure that important logs are saved too, these are written to the USB too.

So before I enabled the overlay fs i did the following steps:

1. Mount USB Stick

   PARTUUID=a633bb72-9ab7-c643-96c2-5817177752ef /mnt/storage ext4 defaults,noatime 0 0

2. Create Symlink to file on mounted USB Stick

   sudo rm /etc/fake-hwclock.data

   sudo ln -s /mnt/storage/fake-hwclock.data /etc/fake-hwclock.data

3. Check if symlink got created

   ls -ln /etc/fake-hwclock.data

   which returns:

   lrwxrwxrwx 1 0 0 30 Feb 7 15:06 /etc/fake-hwclock.data -> /mnt/storage/fake-hwclock.data

---

Problem Statement:

The problem is, the logs are getting saved, as well as the time in fake-hwclock.data but when the machine reboots, the time is still set to the time before the overlay filesystem got enabled.

I think it's reading the data from /etc/fake-hwclock.data (which gets reset at reboot) instead of /mnt/storage/fake-hwclock.data.

I can't see what i am missing. I tried to modify /sbin/fake-hwclock file (which is responsible for reading/saving data to /etc/fake-hwclock.data) to save/read the data directly to/from /mnt/storage/fake-hwclock.data - this hasn't changed anything though.

• Why is the time on my USB reset as soon as i reboot? (Or rather, why does it still take the time from /etc/fake-hwclock.data
• Is /etc/fake-hwclock.data not the only file where the time is saved? Because i can't find any other file
• Does Raspbian read the time from somewhere else than /etc/fake-hwclock?

Answer 1

This is most likely due to the fact that fake-hwclock gets called before your USB stick is mounted. Make sure fake-hwclock.service is executed after local-fs.target.

Note that simply adding After=local-fs.target may not work as you'd expect, because fake-hwclock is required for sysinit.target, which normally runs on just a read-only root filesystem and which (AFAIK) must complete before local-fs.target.

You'll likely have to remove the WantedBy=sysinit.target from fake-hwclock.service to resolve the problem above, and accept that there will be a short period during boot when your system will have an inconsistent date/time.

Or, if you want to be really pedantic, you do the following:

1. mount your USB stick read-only
2. run fake-hwclock
3. remount the USB stick read-write
4. mount the rest of the partitions

Achieveing this will most likely be too much effort with little practical advantage.

Answer 2

I'm simply writing an answer, to share the "config" of /lib/systemd/system/fake-hwclock.service.

Important Note: Before implementing any of this, please make sure to read the Edit at the bottom of the post.

---

So, i've tried out a few variations after reading @DmitryGrigoryev's answer. Sadly, the changes on WantedBy and Before didn't work reliable. Sometimes the time was taken from /mnt/storage/fake-hwclock.data and sometimes from /etc/fake-hwclock.data and I couldn't figure out the reason - I'm guessing it just depends on how fast the USB is mounted (sometimes its faster and sometimes slower).

Taking a new approach in the same direction; Why not just wait with reading the time until the USB is mounted?

---

Find the name of your partition:

sudo systemctl list-units --type=mount

Look for the mount path in DESCRIPTION and copy the UNIT name.

UNIT                      LOAD   ACTIVE SUB     DESCRIPTION
mnt-storage.mount         loaded active mounted /mnt/storage

Modify your /lib/systemd/system/fake-hwclock.service file:

sudo nano /lib/systemd/system/fake-hwclock.service

Add the following line:

[Unit]
...
After=mnt-storage.mount

[Service]
...

[Install]
...

This solved it for me. The time is now read reliably from /mnt/storage/fake-hwclock.data.

---

Edit:

After implementing this, I've encountered the error, that when the RasPi is not shutdown properly (due to power failure, me being uncautious or whatever else), it started in emergency mode, by pressing <ENTER> once the boot process continues. This of course is pretty difficult if the RasPi is (far) away or not accessible through SSH.

Usually this happens if the OS wants to mount the USB and it's not plugged in, which can be fixed by simply adding nofail to /etc/fstab - this doesn't help in this case though, as the USB must be present at all times.

After looking into the boot logs, i've noticed that by adding this line:

After=mnt-storage.mount

... I've messed up the boot order, which looks like the following:

mnt-storage.mount: Found ordering cycle on local-fs-pre.target/start
mnt-storage.mount: Found dependency on systemd-remount-fs.service/start
mnt-storage.mount: Found dependency on systemd-fsck-root.service/start
mnt-storage.mount: Found dependency on fake-hwclock.service/start
mnt-storage.mount: Found dependency on mnt-storage.mount/start
mnt-storage.mount: Job local-fs-pre.target/start deleted to break ordering cycle starting with mnt-storage.mount/start
[ SKIP ] Ordering cycle found, skipping Local File Systems (Pre)

By adding the said line above the boot order is messed up, as the time needs to be read before mounting any devices - which is not possible as the time is read from a mounted device (or atleast sometimes).

This problem didn't occur at every startup, but it occured enough to be annoying. After that I simply bought an RTC for about 15$ and it resolved all of my problems. Now this of course is only necessary because I want consistently correct time with the overlay fs enabled.