Hi NVIDIA Team,
We encountered an unexpected system restart issue while using a USB network card to test network speed on the Jetson AGX Thor development kit. We hope to receive your help.
Platform:
NVIDIA Jetson AGX Thor Developer Kit
OS / BSP:
Ubuntu 24.04.3 LTS
Jetson Linux R38.4
Kernel: 6.8.12-rt-tegra
NIC:
eth0
driver: cdc_ncm
firmware-version: CDC NCM (NO ZLP)
bus-info: usb-a80aa10000.usb-3.1
USB speed: 10000M
Test:
Thor:
iperf3 -s -i 1 -p 5202
Peer:
iperf3 -c 192.168.2.100 -p 5202 -i 1 -t 100 -b 10G -P 4 -R
Failure:
During reverse mode, Thor acts as TCP sender.
Throughput is around 6.9~7.0 Gbit/s.
Kernel reports repeated RCU stall.
RCU stall task is irq/176-xhci-hc.
Call trace involves cdc_ncm_fill_tx_frame, usbnet_start_xmit, net_tx_action, xhci_queue_bulk_tx and xhci_ring_ep_doorbell.
In severe cases serial/SSH become unresponsive and system coldboots.
MB1 reports rst_source=0x33, rst_level=0x1.
systemd watchdog is enabled with NVIDIA Tegra186 WDT, timeout 2min.
Mitigations tried:
- iommu.strict=0: effective, IOMMU lazy mode confirmed, still reproduces.
- Disabled Jetson USB gadget network: still reproduces.
- IRQ 176 affinity set to CPU8-13: still reproduces.
- CPU isolation with isolcpus/nohz_full/rcu_nocbs/irqaffinity: RCU stall still occurs, reboot still occurs.
- Further tuning including IRQ/task CPU separation, offload disable, netdev budget and queue tuning: still cannot eliminate the issue.
- Key call traces:
-
arm_smmu_cmdq_issue_cmdlist → __iommu_dma_unmap → usb_hcd_unmap_urb_for_dma → xhci_unmap_urb_for_dma → irq_thread
-
cdc_ncm_fill_tx_frame → cdc_ncm_tx_fixup → usbnet_start_xmit → net_tx_action → irq_thread
-
xhci_ring_ep_doorbell → xhci_queue_bulk_tx → xhci_urb_enqueue → usb_submit_urb → usbnet_start_xmit → net_tx_action → irq_thread
thor_uart_08_tx.log (562.9 KB)
-
Any guidance would be greatly appreciated. Thanks!