3

In a recent article on the official Pi site the authors mention that a class 4 SD card which is optimised for random read/write can be faster than a class 6 or even a class 10 card which is optimised for large continuous read / write (such that digital cameras perform). How might one check for which type of read / write scenarios an SD card is designed before purchase?

Of course, I am aware that we can test the read / write of a card with hdparm however I do believe that this is also continuous.

$ sudo hdparm -t -T /dev/hda
  • 1
    What is the difference between random and continous R/W and why does it matter? – Matthew Aug 6 '13 at 11:08
  • Some block-level devices order bits which are part of a single file together on the physical medium. This was an issue in the spinning-rust days, but it seems that silicon has gotten fast enough where it matters on flash storage devices as well. Remember defragging FAT partitions? A running OS requires reading many small, disparates file from all over the physical medium, whereas a camera needs to just write a continuous block of bits and move on (ostensibly longer than the flash matrix). The SD cards are 'tuned' for the two different use cases. Think of fast-forwarding through a tape drive. – dotancohen Aug 6 '13 at 11:38
4

PC-compatible hard drives are subdivided into 512-byte sectors which may be independently written any number of times in arbitrary sequence. The flash memory used by most SD cards, thumb drives, and similar devices is not. Although one can get non-volatile memory chips which are organized in such a fashion, the on-chip circuitry required to handle page writes in arbitrary sequence is more complicated (and thus larger and more expensive) than would be required if write sequences were more restricted.

Typically, a NAND flash chip will be divided into pages of 528 bytes (512+16) each, and those pages will be grouped into blocks which range from 16 blocks each (on e.g. an ancient 2MB chip) to thousands of blocks each (on a multi-gig chip). Each block will generally have two associated restrictions:

  • Once a page has been written, the entire block containing must be erased before the page can be written again.

  • Each block may only be written about 10,000 times during the lifetime of the chip (different chips' specified endurance may be an order of magnitude higher or lower)

While it would theoretically possible to have each sector on disk mapped to a fixed page of memory, and have a sector-write request read into a buffer the entire block containing that page, update that sector in the buffer, and then erase the entire block and rewrite it from the buffer, the result would be a horrendously slow "disk" that would be worn out within a few minutes of heavy use. Much better performance and reliability may be obtained if each write request causes the controller to select a blank page and write out the appropriate data along with the sector number and some sort of sequence count (each page of flash has room for 16 bytes of housekeeping information in addition to 512 bytes of data). Each time a sector is written, the page which holds its previous contents will become "garbage"; periodically the system will identify blocks which contain largely "garbage" pages, move all of the data contained in their non-garbage pages to blank pages (making the pages from which the data was copied effectively redundant and thus "garbage"), and then erase the blocks once they contain nothing but garbage.

If all of this sounds complicated, that's because it is. On a small drive, it may be practical to, on power up, examine the "tags" associated with every sector and build a table in RAM that maps all the logical sectors to physical sectors, but on a 32GB drive (64 million sectors), even if one had 256 megs of RAM with which to hold such a table and could populate a million entries per second, populating the table after the drive powered up would take six seconds. Larger drives must thus use other approaches to keep track of their bookkeeping data. In many cases, a controller may keep an up-to-date table in memory and an almost-up-to-date table in flash along with a list of blocks that may be written without updating the latter table (if the flash gets unexpectedly powered down, it will load not-quite-up-to-date table from flash and update it with information from the blocks that were written later). If a controller doesn't have enough RAM to hold the entire map at once, it may sometimes have to write parts of it out to flash and load other parts. If this doesn't happen often, it's no big deal. On the other hand, it's possible for a sufficiently-"random" pattern of disk accesses to result in a lot of extra page-table-flush operations. Even if a controller is capable of reading and writing data quickly, page-table thrashing may slow it down by an order of magnitude.

  • Thank you, that is a very informative answer. It doesn't answer the question, but I gave it a +1 due to the information that it provides in this context. – dotancohen Aug 23 '13 at 11:03
  • @dotancohen: I've not really examined the SD card marketplace, but I thought that an understanding of what flash controllers have to do to mimic disk drives would help clarify why different usage patterns affect SD cards differently and also why higher-capacity SD cards may last longer in some applications (the larger an SD card, the less often each block will have to get recycled). – supercat Aug 23 '13 at 14:56
1

I doubt there's any reliable method to determine random R/W performance besides actual testing. That article mentions the manufacturer - Samsung - which (according to the article) has a focus on random-access performance, that's about as much as you can realistically assess before the purchase.

If you're ready to do hand-on testing, there are several tools which measure the speed of random or misaligned reads and writes. For Linux, two such well-known tools are FIO and flashbench. On Windows, you may want to look at CrystalDiskMark, for example.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.