7

ext4 with journaling is a robust file system that avoids corruption after power-loss. However, this doesn't seem to work for Raspberry Pis. I assume this is because memory cards (and/or our memory card controller) are lying to the file-system about when writes are actually committed to permanent storage.

Since journeling is not effective for Raspbian, why not turn it off to avoid the extra disk writes that it involves?

Is there any brand/model/whatever of memory card out there which does NOT lie to the file-system and makes journaling robust?

Improve this question
5
  • PS: I've had experience with a number of cheap MMC controllers which do out of order writes, and which will lie to the OS about when those writes are actually committed to permanent storage. The combination is enough to corrupt nearly any file system. – DonGar Jun 17 '14 at 3:14
  • Could you share more details about this "MMC controller lying to the OS" experience? How does this happen and under which conditions? Is "out of order writes" the same as "voluntary command queue mode"? Was cache bit set in performance extension register? – Dmitry Grigoryev Oct 22 '20 at 9:00
  • I wrote this quite a few years ago, so I don't even remember which hardware I was thinking of at the time. But no, I've had to work around controllers that were operating out of spec. At the time, I was an app developer working with a kernel developer who was confirming the controller configuration. We had the expected behavior from most hardware, but repeated issues with some of the cheaper controllers we were using. – DonGar Oct 22 '20 at 16:23
  • If you can substantiate your claim (memory card controller [is] lying to the file-system), why not consider submitting a bug report to the RPiOS maintainers? – Seamus Oct 22 '20 at 17:31
  • I don't know if it is (or has ever) happened on the Raspberry Pi, only that I encountered such issues in 2013, and was told by knowledgeable people that it was fairly common with budget MMC hardware. – DonGar Oct 23 '20 at 18:09
6

I doubt that swapping the SD card for another solves the filesystem corruption issue. The following citation from Do journaling filesystems guarantee against corruption after a power failure? pretty much highlights the issue:

There are no guarantees. A Journaling File System is more resilient and is less prone to corruption, but not immune.

The corruption that you are experiencing is likely because the data has not fully been written yet, although the journal was correctly updated. If you did not have a journal, your file would probably be filled with garbage.

As for why Raspbian uses journaling with ext4, that is likely because it has been the default in Debian.

Improve this answer
2

Even if EXT4 with journaling is not infallible, it goes a long way towards protecting your data. Without a journal, you'd have to repair the partition with fsck essentially after every crash / hard reset. With a journal, it's only necessary one time out of 100.

BTW, journaling not guaranteeing 100% data integrity has nothing to do with memory cards lying to the OS about write operations, but with the kind of data that goes to the journal. If you need such guarantees, you'd have to put the entire files in journal before writing them to the disk, not just metadata like file names, sizes and block checksums. However, such journaling would bear a big penality to the performance and memory cards lifetime, and it's not used on the Pi.

Improve this answer
3
  • Agree. How did the OP get a notion that he card is lying to the OS? Is he thinking of the noatime option? – Seamus Oct 21 '20 at 20:43
  • @Seamus To be honest, I don't rule out the possibility that some SD cards will report the IO task status in a way that will mislead Linux into thinking that write operations have completed when in fact they are still in progress. That doesn't change the fact that data integrity cannot be guaranteed without a full journal which provides a second copy of any data that was recently modified. – Dmitry Grigoryev Oct 22 '20 at 8:48
  • That's interesting. In any case, the SD card is a bit of an Achilles heel IMO - too bad RPi doesn't have an m-SATA or M.2 interface... maybe someday. – Seamus Oct 22 '20 at 17:26
1

for me ext4 is working quite well on my RPI. In most cases I simply pull the power plug instead of a regular shutdown. Journalling then avoids annoying fsck when the FS is mounted next time.

Improve this answer
-1

Journaling file systems should NEVER repeat NEVER be used on a flash drive. The journal is a small section of the fs that gets re written EVERY time that you write data to the disk. This gives you a maximum of about 11,000 writes anywhere on the disk before you start to burn out the section that was used for the journal. I've managed to burn a card out in 5 days due to this. I'm going to find out how to force ext 2 on raspian for all my future projects.

Improve this answer
3
  • 6
    don't be silly, most contemporary SD cards use wear leveling, that spreads the write operations all over the empty space on the card. hence the recommendation to get the biggest size you could afford. – lenik May 26 '14 at 7:20
  • 1
    Agreed with @lenik, anonymous user15086 is confusing MTD devices (where this is an issue, and where you should use an MTD-aware filesystem like jffs2) with SD cards, where the Flash memory is hidden behind a block device interface. If it's a block device, you can use any of the regular block device filesystems, so ext4 is quite adequate. – Amedee Van Gasse Jul 24 '14 at 13:30
  • 2
    I'm tempted to downvote this answer because I don't believe it's accurate, and there are no references to back it up. – Seamus Oct 21 '20 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.