Xavier NX: how to share internet across 4x Jetson ethernet network?

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX1
1

I have a network of four Xavier NX modules installed in a box with a built-in ethernet switch. Specifically, it’s this box from SeedStudio:

All four Jetsons run JetPack 4.6; all boot normally. I’ve configured all four to have static IP addresses 192.168.4.1, 192.168.4.2, 192.168.4.3, and 192.168.4.4 on their ethernet interfaces (eth0). They can all ping and ssh to each other through the SeedStudio box’s built-in ethernet switch. Cool!

Only one of the four NX modules (#1) has internet access. On module #1, internet is provided by a USB3-to-Wi-Fi dongle. I would like to share that internet connection with the other three NX modules in the box. This should be easy but I’ve had a devil of a time figuring out how.

Here’s what I’ve done.

On NX module #1 (with the USB3-to-WiFi adapter):
enabled IPv4 forwarding by un-commenting this line in /etc/sysctl.conf:
ipv4.ip_forward=1

On NX module #2, #3, #4: (without internet access)
Set the gateway IP address (in /etc/network/interfaces) to 192.168.4.1

I gather that my next step should be to “bridge” the eth0 and wlan0 interfaces on NX module #1. Every attempt I’ve made to do this has failed. I’ve based my attempts on instructions for a Raspberry Pi, here:

(Although please note, I am NOT trying to create a wireless access point on NX module #1’s wlan0 interface! I am just trying to share its wireless internet access, provided by my home router, with the other 3 NX modules ethernetted together inside the box.)

Despite much googling and prowling the NVidia developer forums, I’ve found no examples of a working configuration that bridges eth0 and wlan0 - so I still can’t share internet across my four-Xavier-NX network.

If it helps, on NX module #1, the wlan0 interface (corresponding to my USB3-to-WiFi adapter) gets IP address 192.168.0.4 from my home WiFi router via DHCP. This works consistently. The home router’s network (192.168.0.x) and the internal static IP ethernet network (192.168.4.x) are different, so I do not believe I have any problem with IP address conflicts.

Send in the troops!

2

This would likely be the same for a Jetson as any other Linux (setting up a bridge). I see a simple example here, but keep in mind there is already a “br0” on the Jetsons, so you would likely need to add a “br1” instead of “br0”:
https://www.cyberciti.biz/faq/ubuntu-20-04-add-network-bridge-br0-with-nmcli-command/

If some sort of firewall software is running, then it might also need adjustment.

Note: Your other Jetsons (the ones not running the outside world connection) might need to have a gateway added as a default route to the first Jetson.

3

Respectfully, I do not need another “here’s an internet link to instructions that might work,” I have literally spent hours already prowling the internet and trying many such links. None of them worked. What I need are instructions that have actually been tested on a real NVIDIA Jetson running L4T, and are known to actually work. Not “here’s something similar that might work”.

For the record, I tried the following, based on the link you provided:

sudo nmcli con add ifname br1 type bridge con-name br1
sudo nmcli con add type bridge-slave ifname eth0 master br1
sudo nmcli con up br1
sudo nmcli connection modify br1 ipv4.addresses '[192.168.4.25/24]
sudo nmcli connection modify br1 ipv4.gateway '192.168.0.4'
sudo nmcli connection modify br1 ipv4.method manual
sudo reboot now

That resulted in my Jetson #1, with the USB3-to-WiFi dongle inserted, being no longer able to see the internet at all.

Bridging eth0 and wlan0 to share an internet connection across an ethernet LAN of Jetsons can’t be this hard.

Please send in the troops.

4

Sorry, I can’t test, I lack the hardware. On the other hand, after the above, what do you see for the “route” command? Bridging is not so much about having an IP address as it is about gluing together the route with some external network (you’d also need to include ifconfig output since this is what route works with).

5

I’d think it should be possible to test this with any Jetson and and any ethernet switch. I would not think there’s anything specific to the Jetson Xavier NX or the SeedStudio cluster box with its own built-in ethernet switch. For the record, here is what the “route” command gives me, on the Xavier NX configured as I described above on Friday:

zeus@jupiter:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    20425  0        0 br1
default         _gateway        0.0.0.0         UG    20600  0        0 wlan0
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 br1
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.0.0     0.0.0.0         255.255.255.0   U     600    0        0 wlan0
_gateway        0.0.0.0         255.255.255.255 UH    20425  0        0 br1
192.168.4.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.4.0     0.0.0.0         255.255.255.0   U     425    0        0 br1

Here is the output of the “ifconfig -a” command on the same NX module:

br1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.4.25  netmask 255.255.255.0  broadcast 192.168.4.255
        ether 36:aa:82:a1:d4:fe  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 68 (68.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:4e:2b:b1:04  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

dummy0: flags=130<BROADCAST,NOARP>  mtu 1500
        ether 22:f8:a5:dd:55:37  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.4.1  netmask 255.255.255.0  broadcast 192.168.4.255
        inet6 fe80::4ab0:2dff:fe38:3dec  prefixlen 64  scopeid 0x20<link>
        ether 48:b0:2d:38:3d:ec  txqueuelen 1000  (Ethernet)
        RX packets 189  bytes 17188 (17.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 270  bytes 29687 (29.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 37  

l4tbr0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether f6:e5:e3:64:04:6d  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 1169  bytes 87640 (87.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1169  bytes 87640 (87.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

rndis0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether f6:e5:e3:64:04:6d  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether f6:e5:e3:64:04:6f  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.30  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 ::80ea:5e1a:3248:fafe  prefixlen 64  scopeid 0x0<global>
        inet6 ::a064:2711:a8e5:db46  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::f68a:3245:f545:5691  prefixlen 64  scopeid 0x20<link>
        ether 98:48:27:b6:b5:74  txqueuelen 1000  (Ethernet)
        RX packets 134  bytes 37835 (37.8 KB)
        RX errors 0  dropped 56  overruns 0  frame 0
        TX packets 106  bytes 41137 (41.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I see that today my wlan0 interface has gotten the IP address 192.168.0.30 … on Friday it got 192.168.0.4, if that matters. The NX module cannot see the internet unless I delete the bridge br1. So, I must still be mis-configured somehow.

6

Guess what … success! Turns out that having to add a bridge was a red herring. The real answer was using iptables to set up Network Address Translation between wlan0 and eth0. I adapted the working formula from:

https://help.ubuntu.com/community/Internet/ConnectionSharing

and

Here are the winning moves. This assumes the gateway Jetson gets its internet connection on wlan0, and its ethernet interface eth0 has static IP address 192.168.4.1. The other Jetsons on the ethernet network have static IPs in the range 192.168.4.2 to 192.168.4.254.

First, enable IP forwarding on your gateway Jetson by uncomment the following line in /etc/sysctl.conf so that it reads:

net.ipv4.ip_forward=1

Then, configure Network Address Translation (NAT) so that IP packets are correctly routed through the gateway. Do this by executing the following commands on the gateway Jetson:

sudo iptables -A FORWARD -o wlan0 -i eth0 -s 192.168.4.0/24 -m conntrack --ctstate NEW -j ACCEPT
sudo iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -t nat -F POSTROUTING
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

Now save the iptables configuration to a file /etc/iptables.sav:

sudo iptables-save | sudo tee /etc/iptables.sav

Then reload iptables.sav on subsequent reboots by creating an /etc/rc.local file that restores the iptables configuration. Note that /etc/rc.local does not exist by default on Ubuntu 18.04, so we’ll create one:

sudo nano /etc/rc.local

Enter the following contents:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
# Restore iptables configuration for internet connection sharing
# on interface eth0:
iptables-restore < /etc/iptables.sav
# By default this script does nothing.
exit 0

Make sure your /etc/rc.local file is executable.

sudo chmod+x /etc/rc.local

Then reboot. Your gatway Jetson’s internet connection should now be shared to all others on the ethernet network.

2 Likes
7

Glad it worked out!

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.