In my opinion the best way is NOT to keep the data file open the WHOLE time.
I am unsure of the intrinsic workings of all these pipes/software but the concept is used in a lot of battery operated applications but even proper servers use this.
What you can do is something in the lines of:
- keep the data in a buffer (memory stream, string)
- flush the buffer (copy the data from the main buffer into a new buffer, then clear the main buffer to allow new data to continue coming in)
- Open your data in an append mode,
- Write the temp buffer into the file and SAVE the file.
- Dispose or clear the temp buffer.
step 3,4 need to be as fast as possible, don't do any other memory of file operations during opening, saving and closing the file, so that appending is as fast possible to reduce the risk of power down while writing. If you are using a text data file, the data will be recoverable in this situation.
One way to overcome corruption while writing during a power off is the create a new file with the buffer, make a copy of the main file, append to the main file, and if everything is committed, deleted the backup and new buffer. If you are going to save every 5 minutes, then this will be great. But if your frequency will be 5,30 seconds.. this might not be so great, for the SD card.
Also, consider make the system readonly to prevent System Corruption and make a new partition that is read/write. This will remove the risk of system becoming unstable during a hard power down and you wont have to do checkdisk on next boot.