Will reformatting external hard drive to Ext4 from NTFS speed up data transfers?

Question

I have two (externally-powered) external USB hard drives plugged into my Raspberry Pi. I use this to download torrents (using Transmission) and to share these with the other computers on the network (using Samba). I get about 2.5MB/s to 2.8MB/s max on torrents. If I use my laptop to download torrents, I can normally get about 4.5MB/s to 6MB/s on torrents, depending on the torrents. I've noticed using the top command that ntfs-3g uses up quite a lot of the processor during file transfers (normally between 20 and 40%).

I've overclocked the Raspberry Pi to 'High' but this hasn't really made much difference. If I were to reformat the drives to Ext4, would this speed up transfer rates?

Answer 1

An easy way to test this would be to try putting a torrent on the SD card, or just time some local transfers from it. All the filesystems involved below are ext4.

Creating a dummy file on the SD card:

> dd if=/dev/zero of=/tmp.test bs=4096000 count=25
25+0 records in
25+0 records out
102400000 bytes (102 MB) copied, 16.0711 s, 6.4 MB/s

Writing to the SD card is much slower than to a USB drive, and in fact in copying from SD to USB the bottleneck will be in reading from the SD card:

> time cp /tmp.test /mnt/hd/tmp.test

real    0m3.301s
user    0m0.010s
sys     0m2.590s

In this case it was not; 102 MB / 2.6s ~= 39 MB/s which is in excessive of what I have already tested as the read limit on the SD card (20 MB/s). This is because my RAM usage is low, and so /tmp.test would be in the file cache; it actually got copied directly from RAM. Hence 39 MB/s -- if I do the same dd as above directly to the hard drive I get around that.

So that gives some indication of the max transfer speed to the drive. The USB bus has a theoretical max of 480 Mb/s (60 MB/s). Note that the ethernet jack shares this bus, but even when it is running at peak 100 Mb/s (12.5 MB/s), there should be enough bandwidth left for this amount to then go back through to the drive.

I've noticed using the top command that ntfs-3g uses up quite a lot of the processor during file transfers (normally between 20 and 40%).

Using ext will eliminate that, although that doesn't mean it is the problem WRT the overall speed. But again, if you know the write speed to your SD card and it's better than 5 MB/s, you should be able to test this by writing a torrent to there.

Answer 2

If the drives are always connected to the Pi, you would be better using ext4, but I don't expect this would increase speed, although it may probably reduce processor usage.

The chip which controls the USB is the bottleneck on the Pi. All USB and Ethernet goes through this. It isn't incredibly speedy anyway, and you are running the data through it twice.