2

I have a Pi that has recently developed a habit of maxing the SD card with syslog messages (somewhere in the hundreds per second) of the message: "Date/Time user kernel: [ 2257.432445] rc rc0: IR event FIFO is full!"

I found this bug report and it was closed in Feb 2019 as "resolved". I looked at the change log for kernel version "linux/4.9.228-1" and found these 3 lines that seem possibly-related but nothing definitive:

    - USB: serial: ir-usb: add missing endpoint sanity check
    - USB: serial: ir-usb: fix link-speed handling
    - USB: serial: ir-usb: fix IrLAP framing

I also found this bug report and tried adding the following items to /etc/modprobe.d/blacklist-ir_event_fifo_is_full.conf and rebooted:

blacklist ir_lirc_codec
blacklist lirc_dev
blacklist sunxi-cir

I then discovered which mods are active with lsmod | grep "ir", revealing:

Module                  Size  Used by
ir_rc6_decoder         16384  0

So I tried adding that to the blacklist file and rebooted:

blacklist ir_rc6_decoder

And still, this doesn't seem to keep it from loading or stop the logs from filling up. Both /var/log/kern.log and /var/log/syslog fill with these same messages.

I haven't found anything recent where this was happening and have exhausted my searching abilities. I mean, searching for "rc rc0: IR event FIFO is full!" in Google with quotes yields 5 results that all seem to be log files.

I'm looking for any guidance on how to stop these messages from accumulating and filling up the SD card.

Peace, Kevin

6
  • Any guidance on what? What is the question?
    – joan
    May 2, 2021 at 9:29
  • 1
    @joan, I updated it to be more clear. I need to stop the logging for that particular message so it stops filling up the SD card. It's currently at a rate of many megabits per minute. So after a few hours, the SD card is completely full.
    – Beez
    May 3, 2021 at 22:39
  • I realize you were likely being hyperbolic, but just in case you seriously believe this is "maxing the SD card with syslog messages": A 100 byte message repeated 250 times per second is 0.25 MB/s, likely ~1% of the write speed.
    – goldilocks
    May 8, 2021 at 14:44
  • @goldilocks, I was just throwing a number out there that sounded high. But, I assure you, the drive was surely being filled by that message. Perhaps it was thousands of messages per second. I only scrolled for a few seconds and saw the same timestamp.
    – Beez
    May 11, 2021 at 6:54
  • 1
    @DmitryGrigoryev, Surely. None of the proposed solutions worked so I am filtering the log message out for the time being. In the logs, I now see a couple indicators that could lead toward a solution but will have to investigate further at a later date: rc rc0: gpio_ir_recv as /devices/platform/ir-receiver@d/rc/rc0 and rc rc0: lirc_dev: driver gpio_ir_recv registered at minor = 0, raw IR receiver, no transmitter.
    – Beez
    May 15, 2021 at 20:13

3 Answers 3

2

First off, you don't need to blacklist any of the sunxi stuff on a Pi, which uses exclusively Broadcom chips.

More to the point, you should check if your blacklist files actually have an effect, by running lsmod | grep ir after a reboot. If the blackisted modules are still there, they are loaded early at boot (specifying rdblacklist=module in the kernel options could help), via a different config file (e.g /etc/modprobe.preload) or by other means. "blackist" means no automatic loading, not that the module is actually forbidden in the kernel.

If editing config files doesn't help, two solutions exist:

  1. Physically remove the offending module file from /lib/modules

  2. Unload the module with rmmod, manually or via an autorun script

I have also heard that blacklisting evdev helps getting rid of the input device modules which keep getting loaded, but AFAIK unless your Pi is used only as a network server, you will need evdev to handle input devices comfortably. I'm not even sure if non-evdev input handling (/dev/kbd, /dev/psaux and the like) is still supported on Linux, especially with USB devices which were predated by evdev.

If there are no more ir modules loaded, and the messages keep coming, then the source of the messages is elsewhere. I would start by investigating what is rc0, perhaps lsmod | grep rc would give you a clue.

2

The way I would approach this problem, if it is just one specific/consistent log message causing trouble, would be to use the filter conditions built into rsyslog. Here is an example of an expression-based filter structure I've used from the relevant rsyslog documentation:

if expr then action-part-of-selector-line

Use the expression arg to capture the message giving your poor SD card grief, and set the selector action to '/dev/null' for example. Pop that filter into a configuration file that defines the log where the message is being stored and that should solve your issue.

1
+50

Raspbian/RpiOS, unlike most linux distros, is able to boot and run to a significant extent without any kernel modules at all available. This is because it was compiled to suit one very specific platform, and everything necessary to basic operation is built-in.1

To test if blacklisting a module could solve this problem, temporarily move /lib/modules:

sudo mv /lib/modules /lib/modules-tmp-moved

And reboot.

If that solves the problem, you could then do a process of elimination starting with the list of what's normally there.

However, in this case I am not sure it will. It's implicit from your question that you actually are not using an IR device, which begs the questions of why that ir module is loaded at all (it is not normally) and why an error is being produced. A grep of the kernel source reveals where that message is from:

> grep -RP "IR event FIFO is full!" drivers/
media/rc/rc-ir-raw.c:           dev_err(&dev->dev, "IR event FIFO is full!\n");

I'm guessing unfortunately that this is part of the "rc core" though, and that is compiled into the stock Pi kernel. This is something you may or may not be able to blacklist (see comments below starting with Dmitry's), if removing/blacklisting the ir module does not work.

If you are okay using the log filtering approach until you find a better solution (and/or find out why this happens at all), here's a example of what cnrcbr suggest in another answer:

if $programname == 'kernel' then {
   if $msg contains 'IR event FIFO is full!'
   then {
       stop
   }
}

Throw that in /etc/rsyslog.conf before the stuff at the bottom where things are dispatched to various files.


  1. I seem to believe from prior experience that this includes wifi, but that seems unlikely, so beware if the pi is headless and only accessible that way.
2
  • 1
    It's actually possible to blacklist even buit-in modules using kernel command line. Perhaps adding initcall_blacklist=ir_rc6_decode_init to cmdline.txt could help. May 10, 2021 at 8:28
  • 1
    I wasn't sure about that and did not check, corrected. I also misread the OP such that I thought the blacklisting had worked but the error still appeared. However, not every compile option of the kernel represents a potential "module". If the issue were that this error were occurring without the module loaded -- which would be strange, but it is strange it happens at all unless there really is an IR device involved -- then I dunno if you can blacklist core functions are always compiled in (or left out).
    – goldilocks
    May 10, 2021 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.