2

The Issue:

SSH connections from a Macbook (Catalina - the SSH Client) to an RPi 4B (Buster, Linux v 4.19.97-v7l+ - the SSH Server) are reliably dropped at random times, but SSH connections from the same Macbook to an Ubuntu Server (18.04 LTS - another SSH Server) are never (hardly ever) dropped. For this reason, I think the cause is the RPi, and not the Macbook, but I am keeping an open mind. I am trying to understand the cause for this, but the primary question here is how to prevent the SSH disconnects.

  • The error message is always: client_loop: send disconnect: Broken pipe

  • The disconnects happen when the Macbook's lid is open, and when it's closed.

  • The disconnects happen with 3B and 1BP RPis also, but my question is only for the 4B.

Here's the setup:

1. /etc/ssh/sshd_config for Ubuntu & RPi are virtually identical; diffs are:

For the Ubuntu Server:
# $OpenBSD: sshd_config,v 1.101 2017/03/14 07:19:07 djm Exp $
Line 96: #UseLogin no
Line 123: PasswordAuthentication yes

For RPi 4B, sshd_config is default (as I found it) :
# $OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $

2. Login/Authentication:

  • Ubuntu logins are done w/ userid & password;
  • RPi logins are done w/ stored SSH credentials.

3. Network Connection & Provisioning:

  • Both RPI and Ubuntu are connected to the LAN via Ethernet.
  • Ubuntu Server has a fixed IP, RPi 4B uses DHCP.
  • Macbook connects to the same LAN as the servers, uses same DHCP server, but is connected via WiFi

4. SSH versions:

  • Macbook:

    % ssh -V
    OpenSSH_8.1p1, LibreSSL 2.7.3
    
  • Ubuntu:

    $ ssh -V
    OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017
    
  • Raspberry Pi 4B:

    $ ssh -V  
    OpenSSH_7.9p1 Raspbian-10+deb10u2, OpenSSL 1.1.1d  10 Sep 2019  
    

5. Keep-Alive Arrangements:

I've tried numerous settings on both the client-side (ssh), and the server-side (sshd). The results have been the same: RPi connections reliably drop at random times, Ubuntu Server connections never (hardly ever) drop. I finally decided to use default /etc/ssh/sshd_config files on all SSH servers, and deal with Keep-Alive in the (Macbook) SSH client. Here's what I've got now:

% cat .ssh/config
Host *
    UseKeychain yes
    AddKeysToAgent yes
    ServerAliveInterval 120

ServerAliveInterval
Sets a timeout interval in seconds after which if no data has been received from the server, ssh(1) will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server.

6. Is it a Power Issue?

I don't think it's a power issue as discussed here for three (3) reasons:

  • I'm using a 5V, 3A USB-C Power Supply mfd by Apple w/ a non-eMarked cable.

  • The Power Supply is plugged into a massive UPS that's lightly loaded. The UPS log shows no issues for weeks on end.

  • I have 2 other Raspberry Pis: a 1BP (buster), and a 3B (stretch) that reliably but randomly drop SSH connections just as the 4B does.

7. Is it a Macbook Issue?

Obviously I don't think so, or I'd post the question somewhere else. If the Macbook sits for a while, the connection will drop

But those of you familiar with Apple's myriad schemes for power saving may suspect that is the most likely cause. I thought so too initially, but as I've experimented w/ this, I've come to feel it's more likely a Raspberry Pi issue. ICYI, a s/s of the Macbook (SSH Client) Power Management settings is shown below. There's also a power management mode called standby that has been disabled (i.e. never go into standby) and confirmed as follows:

% sudo pmset -a standby 0 
% pmset -g | grep standby
 standbydelaylow      10800
 standby              0          # no standby confirmed
 standbydelayhigh     86400
 highstandbythreshold 50

Energy Saver config - macOS Catalina

  • 1
    I'm just looking at a similar issue with my RasPi using Buster. It does not disconnect completely but after a time it does not prompt immediately when hitting enter. It takes some time. Maybe it's the same reason? Try to decrease ServerAliveInterval maybe down to 15. Test ssh from the Ubuntu server to the RasPi. – Ingo Apr 12 at 11:02
  • 1
    From my Mac (Mojave) I get the same error after the Mac goes to sleep + I see the WiFi connection acting oddly - it can reach internal addresses but not any external ones! It's possible the System Settings / Energy Saver / Power Nap needs ticking. I'm just building a 64Bit x86 linux box and will try this in the next day or so... – Andyroo Apr 12 at 11:07
  • With my Mac awake and active, the ssh connection has stayed up nearly two hours so far - this is with the terminal shell to the Pi minimised. The Mac was being used for all sorts during this time AND I had Amphetamine running. The Pi is a Pi 3B running Buster. I'll let the Mac go to sleep and see what happens - I've one session to the Pi and one to an Ubuntu server running. – Andyroo Apr 12 at 15:11
  • 1
    @Ingo: I'll try changing the ServerAliveInterval. Wrt ssh from Ubuntu to RPi: I did that. It works, but when RPi disconnects, so does Ubuntu connection. I think what I need to properly test is two connections to Ubuntu: one to RPi, and one just to Macbook. Any thoughts on a method to log these "Keep Alive" messages? I don't even know the source of the disconnect error message... Server or Client? – Seamus Apr 12 at 21:36
  • 1
    @Andyroo: Yeah - I'm pretty sure that Sleep = No WiFi. I've played around with the power settings in the GUI, but nothing seems to help. There are LOTS more options in pmset, but documentation is poor. In fact, it's difficult to tell what happens in any of these modes - seems Apple changes them at their whim & can't be bothered to document anything. (Yes, I am currently unhappy with several of Apple's decisions.) – Seamus Apr 12 at 21:41
2

Sorry to say I think this is a Mac issue and not a Pi issue.

I set up this for a test yesterday:

Pi <==> Ethernet to Router <==> Mains network <==> Airport 5Ghz <==> Mac terminal

x86 Ubuntu server Ethernet <==> Mac terminal (i.e. directly connected)

Running ifconfig on the Pi box gives me:

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.10  netmask 255.255.255.0  broadcast 192.168.1.255
        ether b8:27:eb:15:ce:cf  txqueuelen 1000  (Ethernet)

Running ifconfig on the Ubuntu box gives me:

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.2.3  netmask 255.255.255.0  broadcast 192.168.2.255
    inet6 fe80::230:1bff:febf:eca4  prefixlen 64  scopeid 0x20<link>

Note the PI is IPv4 only, where as the Ubuntu box is IPv4 and IPv6

Connection to Ubuntu via IPv6 - Connection to PI via IPv4 - Internet sharing set on for Ubuntu link to stop the Mac going to sleep though screen saver set for 10 mins and sleep set for 45 mins. Sharing Internet overrides sleep mode.

Pi not doing anything but sitting at the command line Ubuntu copying nearly 3TB of data (1000s of files) via rsync -v - so lots and lots of text coming out on to the terminal session

Came back just now (just got up) to the Ubuntu linked shell showing:

packet_write_wait: Connection to fe80::230:1bff:febf:eca4%bridge100 port 22: Broken pipe

So the Mac is reporting the drop to the x86 box while the Pi connection was still solid and active.

So it is a possible overrun issue somewhere in the Terminal program / shell or network stack on the Apple - I often see the fail after lots and lots of text has been output (I'm moving thousands of archived pictures, files and TimeMachine backups around at the mo sorting out old 2014-2016 archives).

I have now seen this on both IPv6 and IPv4 connections BUT not seen it in remote TimeMachine backups.

A very quick trawl though the Mac system logs only shows Terminal starting so that's not a lot of help (TBH I'm not great at Mac debugging - not a lot of issues on it so could have missed something.)

| improve this answer | |
  • Ran the same task overnight to just now again but with output going to a disconnected tmux session - both the Pi and the Ubuntu box stayed up fine. Convinced its a Mac issue – Andyroo Apr 14 at 14:53
  • Thanks for your answer. I now agree with your opening assertion, "I think this is a Mac issue and not a Pi issue" (+1), BUT: I don't feel the rest of your answer re causation is totally relevant. I've discovered some information and seen enough now to convince me that your opening assertion is correct, but "the answer" seems to lie in some arcane settings in macOS. I'll post something like an answer within the next day or two - even though this now seems "off-topic" for Raspberry Pi SE. – Seamus Apr 14 at 20:27
1

I never made much progress trying to chase this down to determine the "root cause". However, for reasons unknown to me, the number of dropped SSH connections has decreased to near zero. The SSH software hasn't changed since I posted the question about 4 months ago... as of today:

RPi4B:

$ ssh -V
OpenSSH_7.9p1 Raspbian-10+deb10u2, OpenSSL 1.1.1d  10 Sep 2019

macOS:

% ssh -V                                                        
OpenSSH_8.1p1, LibreSSL 2.7.3

Of course there have been updates to both systems since the question was posted. Perhaps something in the networking stacks of one or both OS? At this point it seems unlikely a definitive answer will emerge, but I'll leave this open for a while longer before "closing" it with acceptance of an answer.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.