25G QSFP on AGX Thor DevKit not working

[kguo1]

Hi:

I have a THOR DevKit (38.4). I enabled the 25G QSFP referring to this doc.

Enable 25 Gigabit Ethernet on QSFP Port — NVIDIA Jetson Linux Developer Guide

After I re-flashed the device, 25G QSFP ports do not work expected. I observed the same issue as this topic:

Enable 25G QSFP on AGX Thor DevKit

Hi, We have some THOR DevKit ( R38.2.1 ) in our office, and I tested the 10G QSFP successfully. I follow DOC to enable 25G QSFP, but something 25G QSFP can’t work correctly. #1. Even I didn’t connect cable on THOR DevKit, it still shows “Connected“ as following picture. #2. I setup ipv4, and connect two THOR DevKit, I can’t ping another THOR. And the connection status is unstable. [image]

@kayccc , you mentioned it will be fixed in 2026/Jan. I ran “./source_sync.sh -k -t jetson_38.4” and rebuild the code last week but the issue still persists. Can you confirm the fix has been in the branch?

This topic mentioned that I need to convert the dtb file in my BSP directory Linux_for_Tegra/bootloader and update it.

Enable 25 Gigabit Ethernet on QSFP Port but can't find tegra264-bpmp-3834-0008-4071-xxxx.dts

Enable 25 Gigabit Ethernet on QSFP Port discusses making change to Update the BPMP device tree by patching Linux_for_Tegra/bootloader/generic/tegra264-bpmp-3834-0008-4071-xxxx.dtb as follows: --- a/tegra264-bpmp-3834-0008-4071-xxxx.dts +++ b/tegra264-bpmp-3834-0008-4071-xxxx.dts ... To double check after extracting everything in public_sources.tbz2 and also checking git clone -b jetson_38.2 “https://nv-tegra.nvidia.com/r/device/hardware/nvidia/t264-public-dts” I cannot find tegra264-bpm…

I ran “dtc” to convert between dtb and dts. Is that correct?

Thanks

Kevin

[whitesscott]

Did you follow this guide and modify 2 dtb?

enable-25-gigabit-ethernet-on-qsfp-port

[kguo1]

Yes, I did.

When I followed that doc, I found tegra264-bpmp-3834-0008-4071-xxxx.dts is missing from the source code. I followed another thread to convert the device tree binary file to text then apply the patch. Did you have the same issue and how did you patch this file?

Thanks

Kevin

[whitesscott]

This is how I enabled 25gb mgbe:

Update kernel/dtb/tegra264-p4071-0000+p3834-0008-nv.dtb

cd Linux_for_Tegra
mkdir temp
cp kernel/dtb/tegra264-p4071-0000+p3834-0008-nv.dtb temp/
cd temp

# Decompile dtb

dtc -I dtb -O dts -o 0008-nv.dts tegra264-p4071-0000+p3834-0008-nv.dtb 

# edit 0008-nv.dts  I've only included a guide to changes for mgbe0 below. 
Change these entries for all 4 mgbe.
mgbe0: ethernet@a808a10000; mgbe1: ethernet@a808a10000; mgbe2: ethernet@a808a10000; mgbe3: ethernet@a808a10000

    mgbe0: ethernet@a808a10000 {
      status = "okay";
      nvidia,mac-addr-idx = <1>;
-        nvidia,uphy-gbe-mode = <1>;
-        nvidia,phy-iface-mode = <0x0>;
+        nvidia,uphy-gbe-mode = <2>;
         nvidia,max-platform-mtu = <9000>;
         nvidia,pcs-rx-eq-sw-ovrd = <1>;
         nvidia,pps_op_ctrl = <8>;

         fixed-link {
          full-duplex;
-            speed = <10000>;
+            speed = <25000>;
         };
     };

# Compile the DTB and replace it in the following paths:

dtc -I dts -O dtb -o tegra264-bpmp-3834-0008-4071-xxxx.dtb bpmp.dts

cp tegra264-bpmp-3834-0008-4071-xxxx.dtb ../kernel/dtb/tegra264-p4071-0000+p3834-0008-nv.dtb
cp tegra264-bpmp-3834-0008-4071-xxxx.dtb ../rootfs/boot/tegra264-p4071-0000+p3834-0008-nv.dtb
cp tegra264-bpmp-3834-0008-4071-xxxx.dtb ../bootloader/tegra264-p4071-0000+p3834-0008-nv.dtb

---

Update bootloader/generic/tegra264-bpmp-3834-0008-4071-xxxx.dtb as follows:

cp  ../bootloader/generic/tegra264-bpmp-3834-0008-4071-xxxx.dtb .

dtc -I dtb -O dts -o xxxx.dts tegra264-bpmp-3834-0008-4071-xxxx.dtb

# edit xxxx.dts to make these changes:

  status = "okay";
  uphy0-config = <7>;      // Configure UPHY0-Lane6-7 for UFS
-       uphy1-config = <7>;      // Configure UPHY1-Lane0-3 for C5 to enable mgbe
-       mgbe0-speed = <2>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
-       mgbe1-speed = <2>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
-       mgbe2-speed = <2>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
-       mgbe3-speed = <2>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
+       uphy1-config = <8>;      // Configure UPHY1-Lane0-3 for C5 to enable mgbe
+       mgbe0-speed = <3>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
+       mgbe1-speed = <3>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
+       mgbe2-speed = <3>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
+       mgbe3-speed = <3>;       /* 0 for 2.5G, 1 for 5G, 2 for 10G, 3 for 25G */
};

# Compile the BPMP DTB and copy it to the following locations:

dtc -I dts -O dtb -o tegra264-bpmp-3834-0008-4071-xxxx.dtb xxxx.dts

cp tegra264-bpmp-3834-0008-4071-xxxx.dtb ../bootloader/generic/tegra264-bpmp-3834-0008-4071-xxxx.dtb
cp tegra264-bpmp-3834-0008-4071-xxxx.dtb ../bootloader/tegra264-bpmp-3834-0008-4071-xxxx.dtb

---

Reflash Thor

[kguo1]

Thanks.

That is the same as what I did.

Did you successfully set up the 25G connection and actually use the network?

In my case, Thor shows the speed of the ports are 25G after I re-flashed it.

But I failed to get a connection when I connected it to the other end of 25G port.

And I observed the same issue of this ticket:

Enable 25G QSFP on AGX Thor DevKit

Hi, We have some THOR DevKit ( R38.2.1 ) in our office, and I tested the 10G QSFP successfully. I follow DOC to enable 25G QSFP, but something 25G QSFP can’t work correctly. #1. Even I didn’t connect cable on THOR DevKit, it still shows “Connected“ as following picture. #2. I setup ipv4, and connect two THOR DevKit, I can’t ping another THOR. And the connection status is unstable. [image]

Thanks

Kevin

[balaji.wanole]

Any updates on this? As I am also experiencing the same

[shantian.huang]

我使用Thor DevKit 也遇到了同样的状况。 现在有更新了吗？

[jesse75]

@kguo1 The carrier state can be misleading here: the kernel DTB can make the MGBE interface look configured while the BPMP-side PHY config is still not the state you think it is. On our Thor devkit work, I treated “ethtool says 25000 / NetworkManager says connected” as only a DTB-load check, not proof that the link partner path is usable.

The two things I would separate are:

1. Confirm the running device tree, not just the BSP files you edited:

sudo dtc -I fs -O dts /sys/firmware/devicetree/base | grep -A20 -E 'ethernet@a808a10000|ethernet@a808b10000|ethernet@a808d10000|ethernet@a808e10000'
dmesg | grep -iE 'mgbe|uphy|bpmp|nvethernet'

For the official 25G path, the kernel DTB change and the BPMP DTB change both matter. In the BSP those are the kernel DTB under:

Linux_for_Tegra/kernel/dtb/tegra264-p4071-0000+p3834-0008-nv.dtb

and the BPMP DTB under:

Linux_for_Tegra/bootloader/generic/tegra264-bpmp-3834-0008-4071-*.dtb

What I do not know is whether NVIDIA’s jetson_38.4 source branch already contains the fix Kay mentioned. From the outside, I would not infer that from source_sync.sh -k -t jetson_38.4, because this issue crosses kernel DTB, BPMP DTB, and flashed bootloader artifacts.

2. Check for the separate runtime issue we hit after the DTBs looked right. On our unit, one MGBE interface had a duplicate MAC address, which made “link is up” misleading for actual IP traffic. Quick check:

for i in mgbe0_0 mgbe1_0 mgbe2_0 mgbe3_0; do
  echo "$i $(cat /sys/class/net/$i/address)"
  ethtool $i | grep -E 'Speed|Link detected'
done

If the MACs are duplicated, fix that before trusting ping results. We also had to enable threaded MGBE processing at runtime for usable testing:

sudo jetson_clocks
for port in a808a10000 a808b10000 a808d10000 a808e10000; do
  echo 1 | sudo tee /sys/devices/platform/bus@0/${port}.ethernet/net/mgbe*/threaded
done

So my read is: if “connected” appears with no cable, I would first verify the actually-loaded BPMP/kernel DTBs and dmesg, then separately check MAC uniqueness and threaded mode. I would not treat branch sync alone as confirmation that the BPMP artifact flashed into the board contains the January fix.