Force use WAN for internet even if LAN is connected?

Question

I have two CPEs. One fast 4G CPE (High Speed, 150 GB/m and one slow 3G CPE (2 MBPS,U/L)

Primary 4G CPE which is basically my primary high-speed internet access is shared through a router and switch for LAN connections. Pi's ethernet is connected to this network. I access Plex, Sonarr, Deluge installed on Pi through the wired network backed by high-speed 4G CPE.

I have another secondary 3G CPE (wireless only, no ethernet port) which is slow but has unlimited download. Pi's WiFi is connected to this other slow 3G CPE.

My use case is I have to download using Secondary Wireless only 3G CPE and use Pi's LAN(backed by 4G CPE) only to access home network and Plex etc.

Scenario 1 - When I connect both WAN and LAN, Pi connects to internet through LAN only.

Scenario 2 - When I connect to LAN and WAN but disable internet using LAN from Router, the internet doesn't work. Pi doesn't use WAN internet when LAN is connected even if it doesn't have internet access

Scenario 3 - I download stuff by running sudo ifconfig eth0 down so Pi has only WiFI and it downloads from slow 3G Network

Scenario I want - Connected to LAN with internet access disabled from Router and only have home network access. Connected to WAN and used to access internet through 3G CPE.

How Do I achieve this?

Answer 1

This is a simple routing problem. You can route local traffic through eth0 and route all other traffic to the internet through wlan0 by setting the default route to this interface. For example I use these ip addresses on the RasPi:

rpi ~$ ip -4 -brief addr
lo               UNKNOWN        127.0.0.1/8
eth0             UP             192.168.50.246/24
wlan0            UP             192.168.30.117/24

The unmodified routing table for this looks:

rpi ~$ ip route
default via 192.168.50.1 dev eth0 proto dhcp src 192.168.50.246 metric 202
default via 192.168.30.1 dev wlan0 proto dhcp src 192.168.30.117 metric 303
192.168.30.0/24 dev wlan0 proto dhcp scope link src 192.168.30.117 metric 303
192.168.50.0/24 dev eth0 proto dhcp scope link src 192.168.50.246 metric 202

You have two default routes. This is what you makes your problems. The kernel can only use one, of course. It uses the one with the lowest metric and that is the route through eth0 with metric 202. All traffic to the internet (destination ip addresses not belonging to local networks -> default route) goes through eth0 but there is no internet. If you shutdown interface eth0 then also its default route is deleted and the route with lower metric is used. That's the connection through wlan0 and there you find the internet.

To fix this just delete the wrong default route:

rpi ~$ sudo ip route del default via 192.168.50.1 dev eth0

If you use Raspbian default dhcpcd networking you can make it persistent by adding this to /etc/dhcpcd.conf:

interface eth0
nogateway

If using systemd-networkd you have to modify the .network file in /etc/systemd/network/ that matches the interface eth0 (section [Match]). There you have to add this if you use DHCP on this interface:

[DHCP]
UseRoutes=no

If you defined a static ip address there then just omit the Gateway= option in section [Network].

Answer 2

I followed @Ingo's suggestion and it worked.

I'm running Openmediavault 5 on Raspbian buster for Pi 4 so I had to modify the below file(s):

ls /etc/systemd/network/

> 10-openmediavault-eth0.link     10-openmediavault-vethca8e950.link    
> 10-openmediavault-wlan0.link     99-default.link
> 10-openmediavault-eth0.network  10-openmediavault-vethca8e950.network 
> 10-openmediavault-wlan0.network  openmediavault-eth0.network

I added to [Network] DHCP=yes

[DHCP]
UseRoutes=no

To both the files through sudo nano

10-openmediavault-eth0.network
openmediavault-eth0.network

And it works now as internet is accessed through wlan0 only. Though sudo ip route shows wan0 as default route

sudo ip route

default via 192.168.8.1 dev wlan0 proto dhcp src 192.168.8.100 metric 1024 
192.168.31.0/24 dev eth0 proto kernel scope link src 192.168.31.68