How do I fix incorrect timestamps in my bash script that launches on startup?

Question

I'll try to explain this the best that I can. I am creating a simple bash script that runs on startup. This script is supposed to create an archive based on the current date, run a python program that generates an audio file (pyaudio), and then move the .wav files to the archive previously created.

Here is the code:

    #!/bin/bash

    timestamp=$(date +"%Y-%m-%d %H:%M")

    date=$(date +"%Y-%m-%d")

    log=/home/pi/bar/log.txt

    # archive=/media/pi/ARCHIVE/$date

    archive=/home/pi/bar/$date

    echo "[ $timestamp ] Script Started" >> $log

    if [ ! -d $archive ]; then
        mkdir -m 777 -p $archive;
        echo "[ $timestamp ] Archive Created Successfully..." >> $log
    else
        echo "[ $timestamp ] Archive Already Exists! No New Folder Created." >> $log
    fi

    sudo -H -u pi python3 /home/pi/bar/record_audio.py >> $log

    filecheck=$(ls /home/pi/*.wav 2> /dev/null | wc -l)

    if [ "$filecheck" != "0" ]; then
        mv *.wav $archive
        echo "[ $timestamp ] Files Moved." >> $log
    else
        echo "[ $timestamp ] No Files Found." >> $log
    fi

It works fine if I run the bash script manually through the command line, but if I allow it to run automatically at startup, the timestamps for files and archives are completely wrong. I'm not sure where to start with troubleshooting this issue.

Answer 1

When the Pi boots it has no time reference.

Raspbian (default installation) will restore the time from the last saved by fake-hwclock, which should be within 1 hour of the time os shutdown.

Depending on how you run the script it should use this saved time until synchronised by NTP.

You could install a RTC or write a systemd service which waits for synchronisation.

You could check the output of timedatectl before your script gets time,

This will show something like:-

      Local time: Tue 2019-06-04 10:23:02 AEST
  Universal time: Tue 2019-06-04 00:23:02 UTC
        RTC time: n/a
       Time zone: Australia/Sydney (AEST, +1000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Answer 2

You are aware the Pi has no built-in real time clock?

You either have to buy an RTC module for the Pi (DS3232 based preferred) or have to start your script after the Pi has got a valid time through NTP, from your Internet router for example.

Answer 3

There are several ways to wait until the system clock is synchronized with time servers after boot up. You can prepend a check in a loop to your script that will test with timedatectl and sleep if the the clock is synchronized. If you do that it is important to run the script in the background as service until it terminates successful.

You can also try to start your script as a oneshot service and if it fails because the time isn't synchronized you can restart it again after a pause set in the systemd unit file for the service.

I have made a service that will look into the journal for a logged message that the clock is synchronized and then execute the script. How to do it you can look at How can I delay the startup of systemd services until the datetime is set (no RTC on the Raspberry Pi). The unit file there does not fit exactly your needs. Instead you can use this one:

[Unit]
Description=Archive wav files
After=time-sync.target

[Service]
Restart=on-failure
RestartSec=30
ExecStartPre=/bin/bash -c '/bin/journalctl -b -u systemd-timesyncd | /bin/grep -q "systemd-timesyncd.* Synchronized to time server"'
ExecStart=/home/pi/myscript.py
WorkingDirectory=/home/pi
User=pi

[Install]
WantedBy=multi-user.target

You have to change myscript.py and maybe the WorkingDirectory.

Answer 4

I had the same issue while trying to run some startup script on my CoreELEC box, I was trying get a time stamp and external IP straight after reboot. Time was incorrect and failed to get an external IP. I guessed that's due to the services not having started yet. I putted a "sleep 5" before running everything else and problems fixed!

Time stamp right after reboot

Box rebooted on 01-01-2015, 11:00:26 AM

Get time stamp again after 5 seconds

Box rebooted on 03-07-2020, 02:24:01 PM

Answer 5

Question

Run [bash/python] automatically at startup, timestamp is wrong, ...

Answer

Or you can try the python datetime thing.

Update 2019jun05hkt2124

The OP says the following:

It works fine if I run the bash script manually through the command line, but if I allow it to run automatically at startup, the timestamps for files and archives are completely wrong. I'm not sure where to start with troubleshooting this issue.

I apologize that I suggested a solution but did not "troubleshoot" or explain why the OP gets the wrong time stamp.

The root cause is that linux does not update the user clock immediately after booting, and its reason is that it has too many higher priority (and less time consuming than getting network time) things to do (in the background) after booting. In other words, it is an engineering trade off and user experience/satisfaction vs system performance.

This is a quick and dirty update. Perhaps I can try a better explanation later.

Update 2019jun06hkt0857

Ideally, linux should keep a record (with the user's authorization) of what the unique user has been doing in the past. Say, if Rpi discovers the angry user, ie, me, hits the little real time clock icon angry (many times in 0.1 second), it knows I want network time update ASAP (of course also checking my Rpi is connected online every time, otherwise it is a waste of time).

Actually MS has been doing research in the past 20+ years for their UI/UE, to make their booting as fast as possible. They from time to time invite users to join their real time online survey observing and collecting statistics on what the gereral/average users do, immediately after booting, and after which job, etc, ... I always agree on their invitation as a volunteer, for the benefit of my user pleasure, and general user's pleasure. Below is an article I read yesterday. Of course you can easily google for many more such articles on UI/UE things. In the past 20+ years or so, I have been on alert on many enggr computing thing, including "Code Project Insider Developer News" where this boot user survey article is coming from.

Microsoft wants to know how you feel about Windows 10’s Start menu - Code Project Insider Developer News Darren Allan 2019jun06

Update 2019jun07hkt2226

pi-wake-on-rtc - bablokb 2018mar09

Introduction

This project implements a wake-on-rtc solution for the Raspberry Pi and other small computers based on the DS3231 real-time-clock.

Overview

The solution consists of two parts: a circuit doing the actual power switching and secondly some programs to control the rtc.

The circuit reacts to three events:

a push of the power-on button: this will switch power on and boot the pi

an alarm of the rtc: this will also turn power on

a gpio-interrupt from the pi after shutdown: this will switch off power

The software-part takes care to program the alarm of the rtc before shutdown, so the system will boot at the requested time.