For a long time, I'm noticing that my Raspberry Pi 4 is slow in some scenarios, especially when some disk-consuming operations are running.
docker pull something might take around 5-10 minutes for not so big image. At first, it works pretty fast but after the first 10-20 seconds download/extract speed drops dramatically and then goes very slowly.
The same behavior for other processes which intensively involve random I/O.
Also, I have a monitoring set, from which I can see that during such operations, SD card read/write requests take up to 8-10 seconds(!!!).
The first spike on the below charts happened while performing
docker pull/docker build actions (around 18:15).
And the second spike is related to the Agnostics (Raspberry Pi Diagnostics) test (around 20:00).
Raspberry Pi Diagnostics test:
Raspberry Pi Diagnostics - version 0.9 Tue Feb 8 19:56:05 2022 Test : SD Card Speed Test Run 1 prepare-file;0;0;36942;72 seq-write;0;0;35158;68 rand-4k-write;0;0;938;234 rand-4k-read;470;117;0;0 Sequential write speed 35158 KB/sec (target 10000) - PASS Random write speed 234 IOPS (target 500) - FAIL Random read speed 117 IOPS (target 1500) - FAIL Run 2 prepare-file;0;0;2011;3 seq-write;0;0;1777;3 rand-4k-write;0;0;3;0 rand-4k-read;50;12;0;0 Sequential write speed 1777 KB/sec (target 10000) - FAIL Note that sequential write speed declines over time as a card is used - your card may require reformatting Random write speed 0 IOPS (target 500) - FAIL Random read speed 12 IOPS (target 1500) - FAIL Run 3 prepare-file;0;0;1857;3 seq-write;0;0;11574;22 rand-4k-write;0;0;10;2 rand-4k-read;1;0;0;0 Sequential write speed 11574 KB/sec (target 10000) - PASS Random write speed 2 IOPS (target 500) - FAIL Random read speed 0 IOPS (target 1500) - FAIL Test FAIL
Yes, those tests and observations were done on the running system, not a fresh installed/formatted, but nothing heavy in disk-consuming terms is running in my system. As you might see, the average number of SD card read/write requests is around 6-9 per second, which is a miserable amount and should not affect the system that much.
Earlier I was using Samsung EVO Plus 64 GB microSDXC Class 10 UHS-I U3 (yes, U3. Not U1 cards which are being produced now instead of U3 by Samsung). Then I started blaming it (faulty, fake card, etc.) and replaced it with SanDisk Extreme Pro 64GB microSDXC Class 10 UHS-I U3 A2. But now I see pretty the same things. So, it can't be both cards are faulty and I believe both are good and it should be something else.
Any ideas on how to troubleshoot, what to try etc.? That can't be true that the SD card performance is so bad on both different cards. I even started thinking about moving my system to an external HDD/SSD...
it can't be both cards are faulty- well, it could be, unlikely though - do you have a more modest SD card, 32GB or smaller - just to test - by the way, you'll find USB3->SSD absolutely flies compared to anything you stick in the SD card slot - all my pi4's run SSD's and I wouldn't go back for anything