I am using raspberry pi 2 (B+) in a biology lab to do real time tracking of small insects.
Basically, we have an
opencv program that detect positions of animals and saves it in a
We typically run experiments for 2 weeks on sometimes as many as 30 pis simultaneously.
Often, some of our pis will freeze: the green ACT LED does not blink any more and they do not respond to ping/ssh. The power LED remains on, and the only way I find to reboot the devices is to power them off by hand.
This is very hard to reproduce in so far as a device can run fine for several days and then crash (or not).
journalctl does not provide any clues after it happens.
We keep a log of crashing devices, and it appears that some have never crashed while others keep crashing after a few days.
For several reasons, we think it is related to faulty SD cards:
- We had a much higher propensity to crash with alternative cards (Verbatim microSDHC, 32GB class 10).
- Swapping cards between devices indicates that the issue is related to the card -- as opposed to the not the power supply or other hardware issue.
- It does not seem that we run out of RAM either
I have tried to:
- Reburn card from img file
- Update firmware
- Use Pi3
Because of the random nature of the bug, testing every possible solution is a matter of time and statistics, so I am not quite sure where to start.
$ uname -a
Linux e043 4.4.13-1-ARCH #1 SMP Wed Jun 8 19:31:47 MDT 2016 armv7l GNU/Linux
The SD cards we use are '32G Samsung EVO SD card':
$ grep . /sys/class/mmc_host/mmc0/mmc0:*/* 2>/dev/null
Boot config file (Pi3)
$ /opt/vc/bin/vcgencmd get_config int