3B+ "Internal error: Oops: 17 [#1] SMP ARM" after system wide grep pattern matching

Question

friendly forum!

I had just setup my 3B+ with ssh certificates and things(booting from a Sandisk Cruze 16GB, no other device connected, wifi internet) and was tinkering with the motd when I ran a

grep -rnw '/' -e 'The black cat ran across the floor'

causing the Pi to think for ten or so seconds, show a few matches or none, then:

Message from syslogd@gra-owfs at Mar 31 18:14:11 ...

kernel:[ 240.377471] 3fe0: 00000000 7e84609c 0002f728 76e9453c 60000010 00000003 00000000 00000000

Message from syslogd@gra-owfs at Mar 31 18:14:11 ...

kernel:[ 240.398610] Code: ebfac537 e5141004 e5142008 e0881001 (e5913000)

Segmentation fault

Message from syslogd@gra-owfs at Mar 31 18:14:11 ...

kernel:[ 240.320014] Internal error: Oops: 17 [#1] SMP ARM

whereafter it stalls when attempting a reboot.

I have seen https://github.com/raspberrypi/linux/issues/2481 and run rpi-update (4.14 -> 4.19 ) , set sdram_freq=450 in config.txt and rebooted and tried the same grep: same result.

Flashed a twin Sandisk Cruze 16GB with the same raspbian lite november 2018 official image, booted, did the same grep immediately: just the same result.

Something appears amiss - I expect to be able to find a string in the file system and then proceed to reboot the pi.

Edit. I just tried starting the old 3B from a "new" rasbian on one of the Sandisk USB memory sticks, which worked, then I did the same grep and...precisely the same Oops and it stalls after "Stopped dhcpcd on all interfaces" just like the 3b+. Same with the other (identical official power source). (Can't test the 3A because can't use a keyboard while booting from a USB drive)

Any help mucho appreciated. (Don't currently have an extra SD card to try and repro it when not booted from a USB device but that seems the next logical step). Perhaps someone else with a Pi that boots via USB can try the same grep?

Answer 1

I used exact the same command than you on my RasPi 3B+ with an SD Card (to be less cryptic I use long options):

rpi ~$ grep --recursive --line-number --word-regexp '/' --regexp='The black cat ran across the floor'

I get a huge amount of error messages about "Input/Output Error", "Permission denied" and "Operation not permitted" but the search finished after 1m36.948s without a kernel Oops.

To avoid the huge amount of error messages I prepend the command with sudo. Now I get the same behavior then yours. The grep aborted with a kernel Oops when trying to search /sys/kernel/slab/. This are pseudo files to the kernel interface.

If I reboot, the kernel cannot terminate some services like network, systemd, plymouthd, etc. It waits for a timeout on every service. On my RasPi after about 10 min on waiting for all timeouts the kernel has managed to reboot it. You should wait this time to have a controlled shutdown.

To have a structured command as documented you should revise it. The normal usage is:

rpi ~$ grep --help | head -n1
Usage: grep [OPTION]... PATTERN [FILE]...

So first I would write it:

rpi ~$ grep --recursive --line-number --word-regexp --regexp='The black cat ran across the floor' /

The --regexp= does not use a regular expression so it's unnecessary and --word-regexp does not have much benefit in this case, so you can write:

rpi ~$ grep --recursive --line-number 'The black cat ran across the floor' /

Now you have to avoid that grep visit with root rights the critical pseudo filesystems from the kernel. A common way is to only search on the current filesystem and don't descend directories on other filesystems. Unfortunately grep doesn't have such an option but find has it with -xdev. So you can combine grep with find to only search on your storage:

rpi ~$ sudo find / -xdev -type f -exec grep --with-filename --line-number 'The black cat ran across the floor' {} \;

This command will find all filenames (-type f) and execute (-exec) grep on every single filename ({}). The disadvantage is that it takes about double the time than with a single grep command. A command with much better performance is to exclude critical directories from grep but it will not automatically exclude mounted filesystems:

rpi ~$ sudo grep --recursive --line-number --exclude-dir={dev,proc,run,sys} 'The black cat ran across the floor' /

Answer 2

Your grep is searching every file in every directory, including binaries, and special files such as those in /dev.

You could try using the -I option, which skips binaries, but I'm not 100% sure this ignores all special files, and I'm unwilling to risk my Pi to test it. :)

You could also confine your search to "non-magic" directories.

edit:

"Magic" in this context is my poorly-chosen word for directories that may contain device files and pseudo-devices.

/dev, /proc, /sys, /run, /mnt are well-known "magic directories" on most if not all Unix-derivative systems.