What is the best way to log gps data from gpsd. I'm using gpspipe and gpxlogger to write to a file but my issue is that I won't have clean shutdowns with the Rpi in the car. What is the best way to get a closed file to write using these programs (or custom written in python/c?) that when there is a sudden power loss I don't lose the entire open data file, if there is such a way?

I see a lot of people doing this but none of them so far have addressed this issue , especially since the RPi's running in people's cars won't cleanly shutdown without a battery backup.


it's unlikely you'll find a software-only solution.

if i were you, i'd create additional device, that connects not only to IGNITION, but also to BATTERY, thus solving the issue with sudden power loss, and put IGNITION to the GPIO, so the software can analyze the current condition and shutdown nicely.

the whole setup will look like this:

[battery]=[power control board]=[RasPi]
                 |                 |

power control board is always powered on from the battery but since it's a tiny microcontroller the consumption is below 1uA.

when the engine is started, the power control board powers on RasPi and other electronics you use. when the engine is stopped, "ignition" becomes zero, RasPi can read this from GPIO and shutdown, power control board can also read the ignition status and start the timer, maybe 30 sec, after that the power to RasPi is turned off.

| improve this answer | |
  • I was just thinking if lets say it opens, writes and closes a new file every 5 minutes or so, that way even if it shuts down only the last five minutes of data is lost, which is perfectly fine. I could try writing something like that out, but was wondering if there is something simpler, maybe a bash script of some kind that runs gpxlogger for 5 minutes, kills and restarts? Clumsy, but perfectly functional. Otherwise I would just use a battery. I want it to be as portable as much as I can without having to rip out things in my car – user12885 Feb 19 '14 at 2:55
  • your biggest problem would be not the 5 minutes of the lost data, but inconsistent file system, that eventually leads to broken system. please, look at my answer once again, power control board could be as small as a nickel coin, definitely much smaller than any battery you may try to use. simple MCU costs $1-2, voltage regulator, a couple of power transistors, prototype board -- all in all probably less than $10 + a bit of soldering and programming. – lenik Feb 20 '14 at 5:30

In my opinion the best way is NOT to keep the data file open the WHOLE time.

I am unsure of the intrinsic workings of all these pipes/software but the concept is used in a lot of battery operated applications but even proper servers use this.

What you can do is something in the lines of:

  1. keep the data in a buffer (memory stream, string)
  2. flush the buffer (copy the data from the main buffer into a new buffer, then clear the main buffer to allow new data to continue coming in)
  3. Open your data in an append mode,
  4. Write the temp buffer into the file and SAVE the file.
  5. Dispose or clear the temp buffer.

step 3,4 need to be as fast as possible, don't do any other memory of file operations during opening, saving and closing the file, so that appending is as fast possible to reduce the risk of power down while writing. If you are using a text data file, the data will be recoverable in this situation.

One way to overcome corruption while writing during a power off is the create a new file with the buffer, make a copy of the main file, append to the main file, and if everything is committed, deleted the backup and new buffer. If you are going to save every 5 minutes, then this will be great. But if your frequency will be 5,30 seconds.. this might not be so great, for the SD card.

Also, consider make the system readonly to prevent System Corruption and make a new partition that is read/write. This will remove the risk of system becoming unstable during a hard power down and you wont have to do checkdisk on next boot.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.