1

A project I'm working on uses Raspberry Pi 3 boards to power a display which simply runs chromium in kiosk mode to display a webpage (a web app I've built for this project). Alongside this is a small NodeJS monitoring script which periodically reports back to our central server to help me keep track of the Pi's and know if any of them go offline.

The issue I'm having is best described by the below error log.

Oct 13 01:37:31 raspberrypi kernel: [122141.382467] INFO: task kworker/2:1:4296 blocked for more than 120 seconds.
Oct 13 01:37:31 raspberrypi kernel: [122141.382484]       Tainted: G         C        4.19.58-v7+ #1245
Oct 13 01:37:31 raspberrypi kernel: [122141.382489] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 13 01:37:31 raspberrypi kernel: [122141.382497] kworker/2:1     D    0  4296      2 0x00000000
Oct 13 01:37:31 raspberrypi kernel: [122141.382532] Workqueue: events_freezable mmc_rescan
Oct 13 01:37:31 raspberrypi kernel: [122141.382571] [<8082c6b4>] (__schedule) from [<8082cd24>] (schedule+0x50/0xa8)
Oct 13 01:37:31 raspberrypi kernel: [122141.382590] [<8082cd24>] (schedule) from [<806823d4>] (__mmc_claim_host+0x120/0x228)
Oct 13 01:37:31 raspberrypi kernel: [122141.382607] [<806823d4>] (__mmc_claim_host) from [<80682514>] (mmc_get_card+0x38/0x3c)
Oct 13 01:37:31 raspberrypi kernel: [122141.382626] [<80682514>] (mmc_get_card) from [<8068b588>] (mmc_sd_detect+0x24/0x7c)
Oct 13 01:37:31 raspberrypi kernel: [122141.382644] [<8068b588>] (mmc_sd_detect) from [<80684b3c>] (mmc_rescan+0x1cc/0x39c)
Oct 13 01:37:31 raspberrypi kernel: [122141.382665] [<80684b3c>] (mmc_rescan) from [<8013bf00>] (process_one_work+0x170/0x458)
Oct 13 01:37:31 raspberrypi kernel: [122141.382683] [<8013bf00>] (process_one_work) from [<8013c244>] (worker_thread+0x5c/0x5a4)
Oct 13 01:37:31 raspberrypi kernel: [122141.382701] [<8013c244>] (worker_thread) from [<80142530>] (kthread+0x138/0x168)
Oct 13 01:37:31 raspberrypi kernel: [122141.382719] [<80142530>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Oct 13 01:37:31 raspberrypi kernel: [122141.382728] Exception stack(0x90355fb0 to 0x90355ff8)
Oct 13 01:37:31 raspberrypi kernel: [122141.382739] 5fa0:                                     00000000 00000000 00000000 00000000
Oct 13 01:37:31 raspberrypi kernel: [122141.382753] 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Oct 13 01:37:31 raspberrypi kernel: [122141.382764] 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000

This error usually precedes reboot and matches the times users are reporting freezing / crashing on their end.

I've attempted adjusting the swappiness and set it to 10. This has certainly had an impact however it appears it only delays the same outcome. Several other attempts to resolve the issue have failed such as scaling down the monitoring script and removing software that isn't needed.

1
  • You can clearly see it’s SD-card-related. It may be failing/not properly seated/whatever. You better change the system to not write to the card whenever possible. – Daniel B Oct 14 '19 at 11:43
0

You are running on Raspbian with a older kernel version. I would suggest upgrading Raspbian completely, then see if you still get error.

If you don't want to upgrade, try the following solution found on raspbian github (it's for even newer kernel version then you're running)

sudo sysctl -w vm.dirty_ratio=10
sudo sysctl -w vm.dirty_background_ratio=5
sudo sysctl -p

Reboot to make changes permanent.

0

For some reason in my PasPi something was restoring the sysctl settings after reboot. It is suspected 'power managers' overwrite user's settings, however in my RasPi was not any power manager. Long search in the RasPi for a process or program or script which resets the sysctl settings to default gave nothing.

So I modified the /etc/rc.local to keep the changes in the kamil234 answer permanent.

sudo su
cat >> /etc/rc.local << EOF
sleep 20
logger 'Initial swappiness='`sysctl vm.swappiness`
sysctl -w vm.dirty_ratio=10
sysctl -w vm.dirty_background_ratio=5
sysctl -w vm.swappiness=10  
sysctl -p
logger 'Updated swappiness='`sysctl vm.swappiness`
EOF

The initial value after boot and updated value are printed to the system log for further analysis.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.