Network setup:
- A Fritz!Box 7530 AX router with IP 192.168.1.1
- A Netgear AP WAC104 access point with IP 192.168.1.2, which is connected with ethernet to the router.
- A Raspberry Pi 4 connected with ethernet to the access point. (192.168.1.39)
- A laptop connected with ethernet to the access point. (192.168.1.3)
All three devices have their static IP defined in the router.
On the Raspberry Pi, I have the following services:
- Jellyfin media server
- Deluge torrent server
- Pihole running in Docker (DHCP disabled)
- Snowflake proxy running in Docker
- DuckDNS cronjob updating my DuckDNS domain every 5 minutes
Output of docker ps
:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9163a3e70584 pihole/pihole:latest "/s6-init" 2 weeks ago Up 2 days (healthy) 0.0.0.0:53->53/udp, :::53->53/udp, 0.0.0.0:53->53/tcp, :::53->53/tcp, 0.0.0.0:8001->8001/tcp, 0.0.0.0:67->67/udp, :::8001->8001/tcp, :::67->67/udp, 80/tcp pihole
f1ad77eb8b50 thetorproject/snowflake-proxy:latest "/bin/proxy" 9 months ago Up 2 days snowflake-proxy
Disabling all docker processes did not fix the issue.
The problem
About once a day, the Raspberry Pi seems to break the access point it is connected to. This results in all devices connected to the access point to stop getting an IP. I connect my laptop to the router's WiFi, and notice that pinging the access point doesn't work: "Destination Host Unreachable".
The problem is fixed when I disconnect the power to my Pi. Pinging to the AP immediately works again when I do that. Repowering the Pi again will work fine until the next time it happens.
I used to have this exact problem every day when I was using home-assistant on my Pi (through Docker). Stopping the home assistant container fixed that problem. But two days ago I updated my Pi from bullseye to the bookworm repositories (because I was annoyed the bullseye packages were so outdated and wanted to test out bookworm) and since then the same problem has returned.
If I look in the logs, I notice the Pi seems to have stopped logging at 06:25: https://pastebin.com/yaF6h8tD. I unplugged the power at 11:20, plugged it back in at 11:21.
These are the last couple of lines before it stopped logging anything:
2023-07-19T06:25:22.138841+02:00 raspberrypi (udev-worker)[2418]: regulatory.0: Process '/lib/crda/crda' failed with exit code 254.
2023-07-19T06:25:24.308720+02:00 raspberrypi systemd[1]: systemd-fsckd.service: Deactivated successfully.
2023-07-19T06:25:25.319725+02:00 raspberrypi dhcpcd[454]: veth7e8dbc6: using IPv4LL address 169.254.174.162
2023-07-19T06:25:25.320258+02:00 raspberrypi dhcpcd[454]: veth7e8dbc6: adding route to 169.254.0.0/16
2023-07-19T06:25:25.321719+02:00 raspberrypi avahi-daemon[401]: Joining mDNS multicast group on interface veth7e8dbc6.IPv4 with address 169.254.174.162.
2023-07-19T06:25:25.322238+02:00 raspberrypi avahi-daemon[401]: New relevant interface veth7e8dbc6.IPv4 for mDNS.
2023-07-19T06:25:25.323011+02:00 raspberrypi avahi-daemon[401]: Registering new address record for 169.254.174.162 on veth7e8dbc6.IPv4.
2023-07-19T06:25:25.455712+02:00 raspberrypi (udev-worker)[2441]: regulatory.0: Process '/lib/crda/crda' failed with exit code 254.
- Rebooting the Pi (by unplugging & replugging the power) fixes the problem.
- Removing the ethernet cable from the Pi fixes the access point immediately. All devices connected to the router then work fine, except for the Pi itself.
- Plugging the ethernet cable back in without rebooting the Pi makes the problem reappear after a few seconds.
- The ethernet port's lights are still flashing when the problem occurs.
- Attaching a serial cable to the Pi doesn't open a terminal if I try to connect while the the problem is present.
- The Pi's green ACT LED is either continously on, or not on at all.
- Trying to open a link-local connection to the Pi from my laptop doesn't work when the problem is present, but I can ping the link-local IP.
- I have removed the
avahi-daemon
package as requested in the comments, but the problem still occurs. - I have removed the
crda
package as I read this isn't necessary anymore (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1003903)
I bought a TTL to USB serial cable to try to fix this, by connecting the cable to my laptop to see if I can get a serial console to the Pi when the problem occurs.
I tried that yesterday and today (2023-08-13) when the problem occured and both times it doesn't even open a terminal connection. Rebooting the Pi fixes it again, and I can confirm the serial connection does work properly if the Pi is running normally. If a serial connection doesn't even work, I guess this means the problem causes the entire RPi to stop working somehow?
What could be the cause of this issue? Is there anything else I can provide (config files, logs)?
Q&A:
The issue sounds predictable. Would it be possible for you to plug in a monitor and watch it while it happens
Watching a monitor while it happens is a bit difficult because it only happens a maximum of once a day (sometimes with multiple days of no issues in between), and I don't really have a spare monitor I can hook up for multiple days. I haven't been able to hook up a monitor while the Pi is already running, as it doesn't seem to accept the input video mode (see above).
Maybe your running a DHCP server of some kind on your PI and its handing out IP and causing confusion. service isc-dhcp-server status However, there are other light weight dhcp servers that exist. Maybe use "service --status-all". Your existing devices would maintain there existing IP until their IP lease time expired as soon as this happens it could grab an IP from your PI
$ service isc-dhcp-server status
Unit isc-dhcp-server.service could not be found.
I am running Pi-hole but the DHCP feature is disabled. Here's the full output of sudo service --status-all
: https://pastebin.com/raw/yZFigg6h
Updates
- Happened on 2023-07-20: https://pastebin.com/Z0ZRQfuQ
- Happened on 2023-07-24, at 15:36, while I was working on my laptop. My connection suddenly dropped and I . Here is the output of
journalctl --since 15:00
: https://pastebin.com/ZrdKzWj6. I rebooted at around 16:20. - Happened again on 2023-07-25 at 19:10. I didn't see anything new in the logs other than what is already in the logs posted above.
- Happened again on 2023-07-28 at around 21:50. This time, the green ACT LED of the Pi was not on, not even flashing.
- 2023-07-29: I stopped and disabled the avahi-daemon.service and avahi-daemon.socket using
systemctl stop
andsystemctl disable
. The problem happened again today (2023-08-01). - The problem occured again (2023-08-02).
- I now moved to a new house (2023-09-18) with a different modem (my provider's proprietary modem). I still use the same router. The problem still occurs.
- Yesterday (2023-10-14) the Pi's green ACT LED was suddenly on continuously, the Pi responded to pings but I could not connect to it (not through ssh nor through serial to usb connection). The router still functioned fine.
- Today (2023-10-15), the Pi's green ACT led was off, and the router and Pi both stopped working. Pinging to the router or Pi was not possible.
What's interesting: the issue did not occur between when I moved and yesterday. Earlier this week, I set up my DynDNS settings on my modem (DuckDNS on my Pi was already set up, but I hadn't set up the correct settings on my modem yet). That's the only thing I can think of that has changed recently.