Multiple network interfaces causing unable to reach eachother

I’m trying to connect between macOS, Nano and Xavier where the Nano enables both WiFi interface for M1 and Ethernet interface for Xavier.

             wlan0               eth0
     M1    <------->    Nano   <------->   Xavier
(10.42.1.100)       (10.42.0.1)        (10.42.0.100)       
                    (10.42.1.1)

The Nano have 2 NICs wlan0 set to sharing WiFi Hotspot for M1 with static IP 10.42.1.100 gateway 10.42.1.1 and eth0 set to sharing Ethernet for the Xavier with static IP 10.42.0.100 gateway 10.42.0.1.

I’ve followed the usb-dev-mode README and configure IPv4 forwarding + NAT for eth0 with its IP as above 10.42.0.1 but I can only ping others from Nano and vice versa but not from/to M1 to/from Xavier. Is there any additional setting needed for their communication?

Both boards using L4T images, versions 4.9.140-tegra/4.9.253-tegra and the PC is on Big Sur 11.4.

Little curious about why “usb dev mode” is mentioned here?

I am also curious about this. Can you provide the full output from the “ifconfig” and “iwconfig” and especially the “route” command on the Nano?

It actually just about mentioning the IPv4 forward which one of the question in the forum asked using a USB WiFi Dongle and might be irrelevant info. Also, in my case it would be using the Intel M.2 WiFi Chipset.

Here are the outputs from Nano.
NOTE: I’m currently just trying to create a bridge network br0 for the eth0 interface. Otherwise, nothing is different.

❯ ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.42.0.1  netmask 255.255.255.0  broadcast 10.42.0.255
        inet6 fe80::c2ea:aa95:804d:8ed0  prefixlen 64  scopeid 0x20<link>
        ether 00:04:4b:e5:41:be  txqueuelen 1000  (Ethernet)
        RX packets 41  bytes 1886 (1.8 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48  bytes 6014 (6.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:85:52:e4:16  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:04:4b:e5:41:be  txqueuelen 1000  (Ethernet)
        RX packets 85  bytes 7266 (7.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 97  bytes 9234 (9.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 150  base 0xa000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 188  bytes 14347 (14.3 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 188  bytes 14347 (14.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.42.1.1  netmask 255.255.255.0  broadcast 10.42.1.255
        inet6 fe80::7670:fdff:fea7:9950  prefixlen 64  scopeid 0x20<link>
        ether 74:70:fd:a7:99:50  txqueuelen 1000  (Ethernet)
        RX packets 140  bytes 34568 (34.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 176  bytes 47985 (47.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
❯ iwconfig
br0       no wireless extensions.

dummy0    no wireless extensions.

eth0      no wireless extensions.

wlan0     IEEE 802.11  Mode:Master  Tx-Power=22 dBm
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:on

lo        no wireless extensions.

docker0   no wireless extensions.
❯ sudo route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.42.0.0       0.0.0.0         255.255.255.0   U     425    0        0 br0
10.42.1.0       0.0.0.0         255.255.255.0   U     600    0        0 wlan0
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 br0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0

Updated

With the current setup IPv4 forwarding and manually adding Xavier’s subnet to M1 route table with route add -net 10.42.0.0/24 10.42.1.1, I can now ping from M1 to Xavier but not the other way around.

I’ve also noticed that if I unplug/plug the Ethernet cable again between Nano and Xavier, then the opposite happen, Xavier can ping M1 but not the other way around.

When I compared the outputs of route, iptables on both Nano and Xavier before and after unplug/plug the Ethernet cable, nothing has changed and the same situation persist after a reboot of every device.

Here is the iptables and route (above) on Nano.

❯ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             10.42.1.0/24         state RELATED,ESTABLISHED
ACCEPT     all  --  10.42.1.0/24         anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
ACCEPT     all  --  anywhere             10.42.0.0/24         state RELATED,ESTABLISHED
ACCEPT     all  --  10.42.0.0/24         anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
Chain DOCKER (1 references)
target     prot opt source               destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere
Chain DOCKER-USER (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere

Here is the same info on Xavier.

❯ sudo route
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.42.0.1       0.0.0.0         UG    20100  0        0 eth0
0.0.0.0         192.168.55.100  0.0.0.0         UG    32766  0        0 l4tbr0
10.42.0.0       0.0.0.0         255.255.255.0   U     100    0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 l4tbr0
192.168.55.0    0.0.0.0         255.255.255.0   U     0      0        0 l4tbr0
❯ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

I see no error in the individual interfaces and route. I couldn’t say for sure where the error is, but it is no doubt in the bridging setup. I tend to make such changes via nm-connection-editor (you can “sudo apt-get install network-manager-gnome” if you don’t have this). Barring that, then I’d look at the bridging tutorials on the internet (this isn’t specific to a Jetson, most any Ubuntu tutorial would be valid).

I’ve alternated between nm-connection-editor and nmtui for the network settings. Thanks for the suggestions.

I too think the bridging is the problem, one suspicion is that it might be the default bridging network came with Xavier Image. I tried to follow the official instruction to safely remove it but it still created on the next boot up.

I’ve tried to put both IPv4 Method to “Shared to other computers” and additionally also with optional IP addresses 10.42.0.xxx/10.42.1.xxx on the Nano (so that other computer see Nano with different than the default addresses), but that also didn’t work.

I lack anything useful to add to this, but there will be bridging setup which is not from NVIDIA when WiFi is installed, plus the USB virtual wired ethernet may have some sort of bridging used. No idea what the effect is for removing any of this, but I would be far less surprised to see issues related to the “standard” WiFi bridging getting in the way of what goes on, while I would far more surprised to see the virtual ethernet device bridging as a problem.