I was to use a raspberry Pi to collect syslog's from a number of devices. This has proved straight forward and works well. Which leads to a second concern.

There is the possibility of losing power and the Pi not being shutdown properly which can lead to corruption of the SD card in some cases. I have found a solution to that as well, using a union file system and mounting the root partition read only with the variable parts written to memory. This works well and prevents the SD card from getting corrupted.

The problem is when I want to do both. I can't just throw the syslog data away but writing it means making the SD card readable. One simple solution is to use an external USB device for capturing all the log data and we know that would work but it is not ideal.

One suggested solution has been to partition the SD card into a root and a data partition with the data partition read/write and root made read only.

The question I have is, Is there any danger of the read/write partition on the SD card corrupting the whole card in the event of power loss? Or SD cards don't work like that and like a hard disk the partitions are effectively isolated and worst case you can only corrupt the read/write partition?

up vote 0 down vote accepted

Unfortunately SD cards does not work the same way as spinning hard disks. The biggest problem here is wear leveling (ensuring that all blocks were written approximately the same number of times). Since it is done at the SD card level, system has no idea on how it's implemented.

The basic idea is the same, however - you try to write to block N but the card decides that it should instead write this file to block M, because N got too many writes already, which may require N to get swapped and so on. Details of this operation may vary from vendor to vendor. In any case, the card has to keep some kind of index of which blocks are mapped where and how many times where they written, etc. Unfortunately SD card has no idea about partitions on the card and so it's not impossible to swap blocks between different partitions. It's even quite likely since one partition is read-only so after some time it's blocks will be much better for writing than the ones from writeable partition. In case of power failure in the middle of such swap, it may corrupt filesystem even on read only partition.

That being said, because each SD card has limited number of writes, it may break quite quickly when written extensively. Because of that, SD card (and any other flash device) is not well suited for such workload.

  • interesting, so at a hardware level it tries to balance the writes. I also faced card-corruption issues with pi so figured ro was a good solution. For the persistant logs I moved to rsyslog. – feverDream Dec 16 '13 at 17:35
  • @feverDream: Yes, not only that. Basically every mass storage flash device (like SD card, USB sticks, etc.) uses NAND flash because it's cheap. Problem is, it's very error prone (many NAND chips have some corrupted sectors when they leave factory) and because of that, they are designed to replace broken sectors. This and wear leveling described in my answer happens magically behind the scenes and many people don't even realize that. – Krzysztof Adamski Dec 16 '13 at 18:16
  • Thank you very much for the information. So perhaps an external secondary USB stick would be a better option to log too separately. – Jason Rennie Dec 16 '13 at 22:06
  • @user2939408: That's probably a good idea. You will probably get bigger overhead when writing big amounts of data, though. Read some information about optimizing usb sticks for writing if you want to get better results (bigger blocks, maybe some other filesystem with some specific options, etc). – Krzysztof Adamski Dec 17 '13 at 8:13

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.