Why can't I connect to my Pi with SSH from outside my LAN?

Question

I'm trying to set up a SFTP server on my Raspberry Pi. I got it working fine from within my LAN, but I want it working from outside my LAN as well. I have forwarded port 22 to my Pi (as well as trying some other ports) but the connection times out after showing what key my server has.

I've tried to troubleshoot my problem for a while. Some common suggestions I've found that have not fixed my problem are:

1. You have not set up your port forwarding correctly

I already have apache up and running with port 80 and 443 forwarded correctly, so I don't think this is the problem.

2. Your ISP is blocking port 22

I've checked canyouseeme.org which shows that port 22 is open (when I forward it with my router). The other ports I tried to forward locally to port 22 (and another port that I enabled in sshd_config) were also shown as open by canyouseeme.org.

3. Make sure you are connecting from outside your LAN

I've tried connecting with my external domain name from both inside my LAN and outside of it, both to no success. I connected from outside my LAN via my cell phone as well as asking my brother who lives across town to connect.

More information

To further troubleshoot this problem I installed a SFTP server on my Windows machine on the same network, forwarded port 22 to my Windows machine and then I was able to connect to the SFTP server from outside my LAN. This makes me think that my problem has something to do with Raspbian blocking my connection attemps or maybe something in my SSHD config is wrong.

I also reinstalled openssh on my Pi (purging my previous install when removing it), but the problem still remains.

Can someone help me fix this problem? What should I do to find out whats causing this?

Edit 1

After a suggestion from @ott-- I installed and used tcpdump on my Pi to see if the packages are arriving. Tcpdump confirmed that packages are arriving to my Pi on port 22 when I try to connect with SFTP from outside my LAN. When I check my SSH log file I find this as the result of a connection attempt:

Aug 20 22:35:05 raspberrypi sshd[4314]: Accepted password for markus from xxx.xxx.xxx.xxx port 56185 ssh2
Aug 20 22:35:05 raspberrypi sshd[4314]: pam_unix(sshd:session): session opened for user markus by (uid=0)
Aug 20 22:35:05 raspberrypi systemd: pam_unix(systemd-user:session): session opened for user markus by (uid=0)
Aug 20 22:35:05 raspberrypi systemd-logind[387]: New session c27 of user markus.

Why is there something about "port 56185" in the log? Doesn't ssh/sftp only use port 22?

Note: I'm using Filezilla as my SFTP client when trying to connect to the server, when connecting I'm stuck at connecting to xxxxxxxxx.xxx until the connection times out.

Answer 1

I finally solved my problem! It all had to do with my Pi being connected to my network via Wifi. When connecting to my network with an ethernet cable instead of Wifi I can finally SSH to my Pi from outside my LAN.