Jetson Orin Issues Reconnecting To Switch

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

I am experiencing an extremely odd issue with our Jetson Orin, we often disconnect and reconnect our robot to a switch, but on disconnecting the ethernet and reconnecting it it is extremely slow to reconnect and sometimes won’t even reconnect till we connect it to a different switch or router. I’ve had this issue with three different switches so I don’t think its the switch. It’s very critical that we are able to ssh as quickly as possible when re-attaching our tether, so really hoping we can solve this. We have a Jetson xavier on the exact same network and it is not experiencing this issue, I did a brand new L4T install just incase, and this did not help.

nvidia@ubuntu:~$ sudo dmesg | grep nvether
[sudo] password for nvidia: 
[   12.954871] nvethernet 6810000.ethernet: Adding to iommu group 52
[   12.961473] nvethernet 6810000.ethernet: failed to read skip mac reset flag, default 0
[   12.969638] nvethernet 6810000.ethernet: failed to read MDIO address
[   12.976179] nvethernet 6810000.ethernet: setting to default DMA bit mask
[   12.983103] nvethernet 6810000.ethernet: failed to read UPHY GBE mode- default to 10G
[   13.381226] nvethernet 6810000.ethernet: Ethernet MAC address: 48:b0:2d:7d:e5:20
[   13.389485] nvethernet 6810000.ethernet: Macsec not enabled
[   13.395221] nvethernet 6810000.ethernet: Macsec: Reduced MTU: 1466 Max: 16383
[   13.406190] nvethernet 6810000.ethernet: eth0 (HW ver: 31) created with 10 DMA channels
[   21.726892] nvethernet 6810000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   43.227662] nvethernet 6810000.ethernet eth0: Link is Down
[   47.326362] nvethernet 6810000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   63.707653] nvethernet 6810000.ethernet eth0: Link is Down
[   96.478458] nvethernet 6810000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[  105.692204] nvethernet 6810000.ethernet eth0: Link is Down
[  129.246681] nvethernet 6810000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:70:79:82:4d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1466
        inet 10.42.43.116  netmask 255.255.255.0  broadcast 10.42.43.255
        inet6 fe80::bb37:9756:7480:50d  prefixlen 64  scopeid 0x20<link>
        ether 48:b0:2d:7d:e5:20  txqueuelen 1000  (Ethernet)
        RX packets 676  bytes 43234 (43.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 295  bytes 38910 (38.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

nvidia@ubuntu:~$ sudo ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether d6:d6:4b:5d:01:27 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1466 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 48:b0:2d:7d:e5:20 brd ff:ff:ff:ff:ff:ff
4: l4tbr0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b6:7e:7c:20:c8:b1 brd ff:ff:ff:ff:ff:ff
5: rndis0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master l4tbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether b6:7e:7c:20:c8:b1 brd ff:ff:ff:ff:ff:ff
6: usb0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master l4tbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether b6:7e:7c:20:c8:b3 brd ff:ff:ff:ff:ff:ff
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:70:79:82:4d brd ff:ff:ff:ff:ff:ff

nvidia@ubuntu:~$ ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:	on
RX:		on
TX:		on
RX negotiated:	on
TX negotiated:	on

Anyone experience this same issue or can think of any solution? Much appreciated!

2

Ethernet is a negotiated link and involves both ends of the connection. This means it is more “warm swap” than “hot swap”. Warm swap implies there is no electrical issue with sudden disconnect or reconnect, but the software might not behave correctly (with hot swap there is software to detect failures and perhaps restore some or part of software function upon reconnect).

For example, unless all ends have a statically assigned IP address (even the switch or router, not just the Jetson), a DHCP address query must be sent and replied to, but a simple disconnect and reconnect does not do this. It is a case of luck if reconnect just happens to have everything still operating smoothly.

Have you tried purposely dropping eth0 before pulling the connector, and manually telling it to bring up eth0 once reconnected? Depending on software release the command might differ, but here are some methods of doing this (if available I prefer “sudo ifconfig eth0 down” or “up”; avoid systemctl methods except as a last resort; “sudo ip link set eth0 down” would be a newer version of “ifconfig” and also a good choice for temporary changes). See:
https://linuxhint.com/ubuntu-enable-and-disable-the-network-interface/

3

Thank-you for your response. I do currently have static ip’s for everything on the network, but our switches are un-managed so I cannot assign static ip via the switch.

If i manually ifconfig up and down the issue is resolved, but unfortunately I am not able to do that as this is for a robot in an enclosed hull, ssh is the only way to access the robot. So if i do eth0 down, theres no way for be to get it back up other then reboot the entire computer. It’s still very strange that the Orin is the only device with this issue, i agree it seems to be luck whether or not it reconnects, but the orin has especially bad luck.

4

You will probably need to add a service (or just a simple script) to repeatedly attempt to bring up the ethernet upon failure. A script would detect failure, and periodically run down/up until success. Ethernet by itself is not designed to do this, and it is “normal” for physically removing and adding the cable to not be hot plug. If this were USB, then unplug and replug would result in a hot plug event, and udev would “do the right thing”. Ethernet has no such plug-n-play (USB layer is in addition to whatever device the USB connects…ethernet has no such layer).

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.