What problems can occur when unplugging a (sync'd) USB drive?

Question

I want to know if there are any specific technical reasons for not just unplugging a USB drive that is still mounted, provided:

1) All open files on the stick have been written to and closed

2) The file system has been sucessfully 'sync'd

3) No other processes touch the USB drive

The background is that I've used a raspberry pi to build a custom handheld camera for novel types of imaging (such as hyperspectral), and I'm aiming to make the user interface as simple as possible. Like, zero-cognitive-thought-requried simple.

Presently, if the user plugs in the USB stick the software:

• Detects the new mount,
• Launches rsync (to differentially copy any new or changed picures to the USB stick)
• After that is complete the software will run the unix 'sync' command (to ensure the files are acutally in the stick's flash memory, rather than a buffer in raspbian somewhere)

From a user's point of view, they just plug in a stick, and their files end up on it. When finished, a green 'ready' LED comes on, and the user knows can then unplug their stick or go on to take more images.

I'm aware that doing a 'sync' will increase wear on the flash memory. However given that in this application it would only be done after a whole batch of files are copied, the wear increase is trivial. (i.e. we are talking maybe 10 or 20 syncs a day, not a million)

My understanding is that the normal unmounting process is just a 'sync', followed by the actual unmounting. So it seems like unplugging a sync'd but still mounted system is safe?

Is there some clever technical reason why this is a bad idea? Is there some property of FAT, NTFS, or other common file systems that will react poorly if unplugged and end up corrupted? If so, can I just avoid it by using a different FAT type (e.g. ensure all sticks are FAT32, or something)?

Answer 1

The reason is that as long as the drive is mounted, it can be written to. sync only waits until current write operations complete, but doesn't prevent new ones from starting. This is somewhat less of a problem with simple file systems like FAT, but even with FAT you have possible write operations which are not initiated by you (the driver may decide to write file system information, or replicate the allocation table). Typically, those don't result in actual data loss, as there are two copies of the allocation table, and they are always at the same fixed locations.

NTFS is much worse in this regard as the allocation table (MFT) doesn't have a fixed place on the disk and can be moved / expanded / shrunk at the discretion of the FS driver. Unplugging the drive during an ongoing MFT operation will result in data loss.