Raspberry Pi Stack Exchange is a question and answer site for users and developers of hardware and software for Raspberry Pi. It only takes a minute to sign up.
Sign up to join this community
I am currently building offline logger - every 10 seconds the application will save GPS location + temperature and humidity.
And now comes the question: what is the best way to store these data on RPi?
I'm thinking about storing these data in .json file or sqlite database, but I suppose there can be better way to do this (something quick and what won't break my SD card by many write operations).
Thanks!
Just use CSV. That way you can just append your data to the existing file. That way only the last block will change, resulting in fewer writes to the SD. Once the block is full, it will never be rewritten.
Maybe do some logrotate to create a separate file for every day. That way any data-corruption will only mess up one day of data.
whatever.log to whatever.log.1 and create a new whatever.log, the process's filehandle will not be for either of those, and subsequent logging will be lost probably without causing any error. So if there's a persistent process with an open file handle involved...
timestamp temp humidity) --in fact I log temperature and barometric pressure with the pi exactly that way.
any plain old text file would do. still there are two things you should watch out for:
don't forget to use your file in open/write/close sequence. people often tend to forget about the "closing" part, they open a file and keep writing until something happens (crash, reboot?) leaving incomplete file. if you use python, this should take care of it:
with open( 'log.txt', 'a' ) as log :
print >>log, date_time, temperature, humidity, latitude, longitude, etc...
"with" will automatically close the file, so you don't have to worry about all this "you must restart after rotating the log" nonsense.
also, you don't really need to store temperature and humidity every 10 seconds, unless in a very harsh environment. it might be a good idea to write three separate log files and store the values only when changed -- this might considerably reduce your file size and wear of your SD card.
I might have suggested a flat file in the past, but SQLite has compelling advantages:
fopen().sudo apt-get install python-pyspatialite libspatialite3 proj-bin). It turns SQLite into a reasonably fully-featured geodatabase, so your data points have geometry. Want to know how far your logger has travelled? Want to see its track as a KML file for Google Earth? Both are single queries in SpatiaLite.Yes, SQLite files can get a lot bigger than flat files, especially when they are writing large journals. But this won't likely happen for a simple logging application.
I'm doing something similar with Node-RED. Basically, my phone sends GPS coordinates, battery temperature and charge level to Node-RED running on my Raspberry Pi, which then stores it in a database or CSV file or json file. You can get started Node-RED by following this tutorial.
