6

I am trying to share my LTE modem connection (wwan0) to my ethernet lan port on my rPi.

  • wlan0 -> WiFi my home network
  • wwan0 -> LTE modem
  • eth0 -> LAN interface

@raspberrypi:~ $ ifconfig -a

eth0      Link encap:Ethernet  HWaddr b8:27:eb:2a:2a:0f  
      inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
      UP BROADCAST MULTICAST  MTU:1500  Metric:1
      RX packets:6056 errors:0 dropped:0 overruns:0 frame:0
      TX packets:7515 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:413209 (403.5 KiB)  TX bytes:431931 (421.8 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:2813 (2.7 KiB)  TX bytes:2813 (2.7 KiB)

wlan0     Link encap:Ethernet  HWaddr b8:27:eb:7f:7f:5a  
          inet addr:192.168.1.50  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:323 errors:0 dropped:2 overruns:0 frame:0
          TX packets:485 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:30591 (29.8 KiB)  TX bytes:55800 (54.4 KiB)

wwan0     Link encap:Ethernet  HWaddr 00:1e:10:1f:00:00  
          inet addr:100.64.181.186  Bcast:100.64.181.187  Mask:255.255.255.252
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1922 (1.8 KiB)  TX bytes:8367 (8.1 KiB)

sudo nano /etc/network/interfaces

#iface eth0 inet manual
allow-hotplug eth0  
iface eth0 inet static  
address 192.168.2.1
netmask 255.255.255.0

sudo nano /etc/dnsmasq.conf

interface=eth0      # Use interface eth0  
listen-address=192.168.2.1 # Explicitly specify the address to listen on  
bind-interfaces      # Bind to the interface to make sure we aren't sending things elsewhere  
server=8.8.8.8       # Forward DNS requests to Google DNS  
domain-needed        # Don't forward short names  
bogus-priv           # Never forward addresses in the non-routed address spaces.  
dhcp-range=192.168.2.1,192.168.2.50,12h # Assign IP addresses between 172.24.1.50 and 172.24.1.150 with a 12 hour lease time

sudo nano /etc/sysctl.conf

net.ipv4.ip_forward=1

sudo iptables -t nat -A POSTROUTING -o wwan0 -j MASQUERADE 

    pi@raspberrypi:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         setup.ubnt.com  0.0.0.0         UG    10     0        0 wlan0
default         100.64.35.177   0.0.0.0         UG    1000   0        0 wwan0
100.64.35.176   *               255.255.255.240 U     1000   0        0 wwan0
192.168.1.0     *               255.255.255.0   U     10     0        0 wlan0
192.168.2.0     *               255.255.255.0   U     0      0        0 eth0

...If I am connecting my client PC with rPi I get IP address via DHCP, but there is no internet connection. Can not ping (IP, hostname).

Route table after connecting the client:

pi@raspberrypi:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         setup.ubnt.com  0.0.0.0         UG    10     0        0 wlan0
default         100.64.35.177   0.0.0.0         UG    1000   0        0 wwan0
100.64.35.176   *               255.255.255.240 U     1000   0        0 wwan0
link-local      *               255.255.0.0     U     202    0        0 eth0
192.168.1.0     *               255.255.255.0   U     10     0        0 wlan0
192.168.2.0     *               255.255.255.0   U     0      0        0 eth0

syslog:

Jul 26 00:35:32 raspberrypi dhcpcd[760]: eth0: carrier acquired
Jul 26 00:35:32 raspberrypi kernel: [  662.708525] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xC5E1
Jul 26 00:35:32 raspberrypi dhcpcd[760]: eth0: IAID eb:2a:2a:0f
Jul 26 00:35:33 raspberrypi dnsmasq-dhcp[794]: DHCPDISCOVER(eth0) 00:0e:c6:d9:b4:b0
Jul 26 00:35:33 raspberrypi dnsmasq-dhcp[794]: DHCPOFFER(eth0) 192.168.2.40 00:0e:c6:d9:b4:b0
Jul 26 00:35:33 raspberrypi dhcpcd[760]: eth0: soliciting an IPv6 router
Jul 26 00:35:33 raspberrypi dhcpcd[760]: eth0: soliciting a DHCP lease
Jul 26 00:35:34 raspberrypi dnsmasq-dhcp[794]: DHCPREQUEST(eth0) 192.168.2.40 00:0e:c6:d9:b4:b0
Jul 26 00:35:34 raspberrypi dnsmasq-dhcp[794]: DHCPACK(eth0) 192.168.2.40 00:0e:c6:d9:b4:b0 Adrians-MBP
Jul 26 00:35:42 raspberrypi dhcpcd[760]: eth0: using IPv4LL address 169.254.153.193
Jul 26 00:35:42 raspberrypi avahi-daemon[430]: Registering new address record for 169.254.153.193 on eth0.IPv4.
Jul 26 00:35:42 raspberrypi dhcpcd[760]: wwan0: adding default route via 100.64.35.177
Jul 26 00:35:42 raspberrypi dhcpcd[760]: eth0: adding route to 169.254.0.0/16
Jul 26 00:35:43 raspberrypi dhcpcd[760]: wwan0: removing default route via 100.64.35.177
Jul 26 00:35:44 raspberrypi ntpd[784]: Listen normally on 8 eth0 169.254.153.193 UDP 123
Jul 26 00:35:44 raspberrypi ntpd[784]: peers refreshed
Jul 26 00:36:01 raspberrypi CRON[1732]: (root) CMD (/home/pi/script/watchdog.sh >/dev/null 2>&1)
Jul 26 00:37:01 raspberrypi CRON[1751]: (root) CMD (/home/pi/script/watchdog.sh >/dev/null 2>&1)
Jul 26 00:37:53 raspberrypi dhcpcd[760]: eth0: carrier lost
Jul 26 00:37:53 raspberrypi kernel: [  803.801956] smsc95xx 1-1.1:1.0 eth0: link down
Jul 26 00:37:53 raspberrypi avahi-daemon[430]: Withdrawing address record for 169.254.153.193 on eth0.
Jul 26 00:37:53 raspberrypi dhcpcd[760]: wwan0: adding default route via 100.64.35.177
Jul 26 00:37:53 raspberrypi dhcpcd[760]: eth0: deleting route to 169.254.0.0/16
Jul 26 00:37:54 raspberrypi dhcpcd[760]: wwan0: removing default route via 100.64.35.177
Jul 26 00:37:55 raspberrypi ntpd[784]: Deleting interface #8 eth0, 169.254.153.193#123, interface stats: received=0, sent=0, dropped=0, active_time=131 secs
Jul 26 00:37:55 raspberrypi ntpd[784]: peers refreshed

    pi@raspberrypi:~ $ sudo iptables -t nat -v -L -n --line-number
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1 packets, 240 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        1   240 MASQUERADE  all  --  *      wwan0   0.0.0.0/0            0.0.0.0/0

Only wwan0 sharing is not working, If I replace with Wifi wlan0 it's immediately starts working. What could be the problem?

IT'S WORKING!!!

pi@raspberrypi:~ $ sudo ip rule add iif lo priority 48000 table 2^C
pi@raspberrypi:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         setup.ubnt.com  0.0.0.0         UG    10     0        0 wlan0
default         100.64.12.62    0.0.0.0         UG    1000   0        0 wwan0
100.64.12.60    *               255.255.255.252 U     1000   0        0 wwan0
192.168.1.0     *               255.255.255.0   U     10     0        0 wlan0
192.168.2.0     *               255.255.255.0   U     0      0        0 eth0
pi@raspberrypi:~ $  sudo ip rule add iif lo priority 48000 table 2
pi@raspberrypi:~ $ sudo ip route add default via 192.168.1.1 dev wlan0 table 2
pi@raspberrypi:~ $ sudo ip rule add iif eth0 priority 48010 table 3
pi@raspberrypi:~ $ sudo ip route add default via 100.64.12.62 dev wwan0 table 3
pi@raspberrypi:~ $ ip rule
0:  from all lookup local 
32766:  from all lookup main 
32767:  from all lookup default 
48000:  from all iif lo lookup 2 
48010:  from all iif eth0 lookup 3 
pi@raspberrypi:~ $ ip route show table 2
default via 192.168.1.1 dev wlan0 
pi@raspberrypi:~ $ ip route show table 3
default via 100.64.12.62 dev wwan0 
pi@raspberrypi:~ $ ip route show table main
default via 192.168.1.1 dev wlan0  metric 10 
default via 100.64.12.62 dev wwan0  metric 1000 
100.64.12.60/30 dev wwan0  proto kernel  scope link  src 100.64.12.61  metric 1000 
192.168.1.0/24 dev wlan0  proto kernel  scope link  src 192.168.1.50  metric 10 
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1 
pi@raspberrypi:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         setup.ubnt.com  0.0.0.0         UG    10     0        0 wlan0
default         100.64.12.62    0.0.0.0         UG    1000   0        0 wwan0
100.64.12.60    *               255.255.255.252 U     1000   0        0 wwan0
192.168.1.0     *               255.255.255.0   U     10     0        0 wlan0
192.168.2.0     *               255.255.255.0   U     0      0        0 eth0
pi@raspberrypi:~ $ sudo ip route delete default table main
pi@raspberrypi:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         100.64.12.62    0.0.0.0         UG    1000   0        0 wwan0
100.64.12.60    *               255.255.255.252 U     1000   0        0 wwan0
192.168.1.0     *               255.255.255.0   U     10     0        0 wlan0
192.168.2.0     *               255.255.255.0   U     0      0        0 eth0
pi@raspberrypi:~ $ sudo ip route delete default table main
pi@raspberrypi:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
100.64.12.60    *               255.255.255.252 U     1000   0        0 wwan0
192.168.1.0     *               255.255.255.0   U     10     0        0 wlan0
192.168.2.0     *               255.255.255.0   U     0      0        0 eth0
pi@raspberrypi:~ $ ip route show table 2
default via 192.168.1.1 dev wlan0 
pi@raspberrypi:~ $ ip route show table 3
default via 100.64.12.62 dev wwan0 
pi@raspberrypi:~ $ ping dsl.sk
PING dsl.sk (217.67.19.197) 56(84) bytes of data.
64 bytes from hornik1.housing.dialtelecom.sk (217.67.19.197): icmp_seq=1 ttl=56 time=24.0 ms
64 bytes from hornik1.housing.dialtelecom.sk (217.67.19.197): icmp_seq=2 ttl=56 time=27.2 ms
64 bytes from hornik1.housing.dialtelecom.sk (217.67.19.197): icmp_seq=3 ttl=56 time=27.4 ms
^C
--- dsl.sk ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 24.039/26.248/27.428/1.563 ms
pi@raspberrypi:~ $ curl ipinfo.io/ip
90.64.28.65
pi@raspberrypi:~ $

One more thing left: ping is not working, when pinging from localhost trough wwan0 interface (tcp connections are working, like curl --interface wwan0 dsl.sk, or nping --tcp dsl.sk):

wwan0     Link encap:Ethernet  HWaddr 00:1e:10:1f:00:00  
          inet addr:100.64.248.238  Bcast:100.64.248.239  Mask:255.255.255.252
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5359 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5393 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:409828 (400.2 KiB)  TX bytes:471651 (460.5 KiB)

pi@raspberrypi3:~ $ sudo nano /etc/wwan2lan.sh 
pi@raspberrypi3:~ $ sudo ip rule
0:  from all lookup local 
32766:  from all lookup main 
32767:  from all lookup default 
48000:  from all iif lo lookup 2 
48010:  from all iif eth0 lookup 3 
48010:  from 100.64.248.238 iif lo lookup 3 
pi@raspberrypi3:~ $ sudo ip rule add from 100.64.248.238/32 iif lo priority 47010 table 3
pi@raspberrypi3:~ $ ping -I wwan0 dsl.sk
PING dsl.sk (217.67.19.197) from 100.64.248.238 wwan0: 56(84) bytes of data.
64 bytes from hornik1.housing.dialtelecom.sk (217.67.19.197): icmp_seq=1 ttl=57 time=292 ms
64 bytes from hornik1.housing.dialtelecom.sk (217.67.19.197): icmp_seq=2 ttl=57 time=19.3 ms

after reboot:

pi@raspberrypi3:~ $ sudo ip rule add from 100.64.5.41/32 iif lo priority 47010 table 3
pi@raspberrypi3:~ $ ip rule
0:  from all lookup local 
32766:  from all lookup main 
32767:  from all lookup default 
47010:  from 100.64.5.41 iif lo lookup 3 
48000:  from all iif lo lookup 2 
48010:  from all iif eth0 lookup 3 
48010:  from 100.64.5.41 iif lo lookup 3 
pi@raspberrypi3:~ $ ping -I wwan0 dsl.sk
PING dsl.sk (217.67.19.197) from 100.64.5.41 wwan0: 56(84) bytes of data.
From 100.64.5.41 icmp_seq=1 Destination Host Unreachable
From 100.64.5.41 icmp_seq=2 Destination Host Unreachable

in second terminal I did tcpdump:

pi@raspberrypi3:~ $ sudo tcpdump -n -i wwan0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wwan0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:40:00.878187 ARP, Request who-has 217.67.19.197 tell 100.64.5.41, length 28
19:40:01.923296 ARP, Request who-has 217.67.19.197 tell 100.64.5.41, length 28
19:40:02.963306 ARP, Request who-has 217.67.19.197 tell 100.64.5.41, length 28
19:40:04.004512 ARP, Request who-has 217.67.19.197 tell 100.64.5.41, length 28

I also tried to issue only ip rule wwan0ip...47010 (without 48010), it is not helped.

Here is the automated script to make things work automatically (add it to post-up...):

pi@raspberrypi3:~ $ cat /etc/wwan2lan.sh 
#!/bin/bash


sleep 10

# define interfaces

LOCALIF="wlan0"
WWANIF="wwan0"
LANIF="eth0"

#get gateway IP address

LOCALGATEWAY=$(ip route show 0.0.0.0/0 dev $LOCALIF | cut -d\  -f3)
WWANGATEWAY=$(ip route show 0.0.0.0/0 dev $WWANIF | cut -d\  -f3)


#local traffic gets one private default route:

ip rule add iif lo priority 48000 table 2
ip route add default via $LOCALGATEWAY dev $LOCALIF table 2

#eth0 traffic gets another private default route:

ip rule add iif $LANIF priority 48010 table 3
ip route add default via $WWANGATEWAY dev $WWANIF table 3

#delete shared gateways

ip route delete default dev $LOCALIF
ip route delete default dev $WWANIF

#enable wwan0 traffic from localhost if we specify the adapter
INETADDR=$(/sbin/ifconfig $WWANIF | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
ip rule add from $INETADDR/32 iif lo priority 48010 table 3
Improve this question
12

1 Answer 1

Reset to default
5
+50

Standard Limits

It is normal in IP related standards to assume that a HOST has a routing table that applies equally irregardless of the source of a packet. It is also normal for a filter to drop or mangle packets, but not to directly adjust/interact with routing.

Remaining Strategies

  1. Modify a routing relevant aspect of the packet.
  2. Use a virtual labeling standard that the OS supports. (Linux uses fwmark.)
  3. linux appears to support an alternate policy system ip-rule(8)

Example Solution - manipulate TOS as it is routing relevant

  # sudo iptables -t mangle -A POSTROUTE -j TOS -h
    (find TOS value/names besides Normal-Service: 0x0)
  # ip route add default tos 0x08 via 192.168.1.1 metric 10
    (set a TOS on a new route)
  # ip route add default tos 0x10 via 192.168.2.1 metric 10
    (set a different TOS on the other route)

  # iptables -t mangle -A FORWARD -i eth0 -j TOS --set-tos Minimize-Delay
    (set one on local origin packets so they use that route)
  # iptables -t mangle -A OUTPUT -j TOS --set-tos Maximize-Throughput
    (set the other on local origin packets so they use the other route)
  # iptables -t mangle -A POSTROUTING -j TOS --set-tos Normal-Service
    (prevent it from going out on the wire)
  # # then test TOS is applying to routes:
  # ip route get [remote_ip] tos 0x08      
  # ip route get [remote_ip] tos 0x10

Example Solution - via manipulating ip rules

local traffic gets one private default route:

  # ip rule add iif lo priority 48000 table 2
  # ip route add default via 192.168.1.1 dev wlan0 table 2

eth0 traffic gets another private default route:

  # ip rule add iif eth0 priority 48010 table 3
  # ip route add default via 100.64.35.177 dev wwan0 table 3

fwmark can be used to allow iptables or ping to explicitly select either default route:

  # ip rule add priority 47010 fwmark 3 table 3
  # ip rule add priority 47000 fwmark 2 table 2

check resulting table:

  # ip rule
  (verify table)
  # ip rule del priority [199|..]
  (clean up any old rules)
  # ip route delete default .. table main
  (remove any shared default routes)
  # ip route show table [2|3|main]

verify pings using fwmark:

  # ping -m 3 ...
  # ping -m 2 ...
  # ping ...
Improve this answer
26
  • In the meantime I started looking for policy based routing, maybe I was totally wrong. However your answer looks very promising. The first sudo iptables command is just showing the iptables help (because of the -h switch), it is ok? I updated my question with some test. On client I have internet (fully working, with almost full speed), but on server the internet is acting weird. Ping is, any other application with internet access takes minutes to execute (curl, wget) - 90% packet loss. CPU I think can't be the problem it's 99% idle.
    – user66638
    Jul 28, 2017 at 19:22
  • @user66638 the -h is just to find the TOS names/numbers, but I think the rule that 0x0 TOS routes can match is allowing it to round-robin between them (and therefore get the ip in SAS to.) I've added setting the TOS on both routes, but figuring out ip-rule would be a lot cleaner.
    – lossleader
    Jul 28, 2017 at 20:40
  • From -h I get: Accepted symbolic names for value are:(0x10) 16 Minimize-Delay (0x08) 8 Maximize-Throughput (0x04) 4 Maximize-Reliability (0x02) 2 Minimize-Cost (0x00) 0 Normal-Service. Tried this new solution, still fix 91% packets are lost.
    – user66638
    Jul 28, 2017 at 21:00
  • If I define interface for ping: ping -I wlan0 somedomain.com - it's much faster, less operation not permitted errors, packet loss around 10-30%. That's interesting.
    – user66638
    Jul 28, 2017 at 21:10
  • thanks for the ip rule solution. Should I delete main gateway from wlan0 first? If I delete network is unreachable from server, on client works, if I leave no internet on client, but yes on the server. Also should I leave rule "iptables -A POSTROUTING -o wwan0 -j MASQUERADE"? (it was enabled all the time). I add output from the ip rule method.
    – user66638
    Jul 29, 2017 at 10:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.