Nvethernet driver: TX byte counter integer underflow (petabyte spikes) when TSO is enabled on Jetpack 7.2 (Orin AGX)

user100132 · July 17, 2026, 4:59am

System Configuration:

Module: Jetson Orin AGX 64GB on devkit
Jetpack Version: Jetpack 7.2 (L4T r36.x / r38.x equivalent)
Kernel Version: 6.8.12-1021-tegra
Interface: eth0 (1G EQOS controller, 2310000.ethernet)
Driver: nvethernet

Description of the Issue:

When monitoring network traffic on a Jetson Orin AGX running Jetpack 7.2, the kernel software statistics in /sys/class/net/eth0/statistics/tx_bytes (and /proc/net/dev) occasionally reporting mathematically impossible values (on the order of 159 Petabytes).

This causes monitoring agents like Prometheus node_exporter to report active transmit rates of over 1 TB/s (7 TiB/s).

The issue is caused by an integer underflow in the nvethernet driver’s software byte-accounting when TCP Segmentation Offload (TSO) is active. The physical MAC registers count the transmitted bytes correctly, but the driver occasionally passes a negative delta to the kernel’s network statistics struct, wrapping the unsigned 64-bit integer to a massive value.

How to Reproduce:

Ensure TCP Segmentation Offload is enabled (default state):

Bash
```
sudo ethtool -K eth0 tso on gso on gro on
```
Generate active TCP transmit traffic (e.g., via iperf3 or transferring large files).
Check the kernel’s reported transmit bytes:

Bash
```
cat /sys/class/net/eth0/statistics/tx_bytes
```
Expected Behavior: Realistic byte count matching physical traffic. Actual Behavior: The counter jumps to an astronomical 18-digit number (e.g., 159400201039313555 bytes).

Technical Evidence (Debugging Logs):

1. Corrupted Kernel Counter (Software):

Bash

$ cat /sys/class/net/eth0/statistics/tx_bytes
159400201039313555

2. Accurate Hardware Registers (`ethtool -S`):

The physical hardware counters on the Orin NIC prove that the physical MAC is counting normally and has only actually sent ~473 GB, not 159 PB:

Bash

$ sudo ethtool -S eth0 | grep -i tx
     mmc_tx_octetcount_gb: 473348121972
     mmc_tx_framecount_gb: 340560245
     tx_tso_pkt_n: 37212641

(Note the high tx_tso_pkt_n count, confirming heavy TSO usage during the bug occurrence).

3. Driver Information:

Bash

$ sudo ethtool -i eth0
driver: nvethernet
version: 6.8.12-1021-tegra
bus-info: 2310000.ethernet

Workaround:

Disabling hardware offloads immediately stops the thrashing/underflow and stabilizes the /sys/ counters:

Bash

sudo ethtool -K eth0 tso off gso off gro off

Request to NVIDIA Engineering:

Please review the nvethernet driver source code (specifically where it handles DMA ring buffer completion and updates the kernel socket buffer skb statistics). A calculation logic error during TSO packet segmentation is causing a negative byte subtraction, resulting in this uint64 underflow.

user100132 · July 17, 2026, 5:04am

You may see “end0” on your system. We used kernel boot command line “net.ifnames=0” so we can keep the old name “eth0”.

WayneWWW · July 17, 2026, 7:29am

Just to clarify. Orin AGX devkit does not have any port for EQOS 2310000.

Only the MGBE on it.

Please check if MGBE on a true NV devkit could reproduce issue with your method first.

user100132 · July 17, 2026, 4:09pm

Yes, this reproduced on both custom carrier board and devkit.

On devkit

user@localhost:~$ cat /sys/class/net/end0/statistics/tx_bytes
169004023406449163
user@localhost:~$ sudo ethtool -i end0
driver: nvethernet
version: 6.8.12-1021-tegra
firmware-version: 
expansion-rom-version: 
bus-info: 6800000.ethernet
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

carlos.quiros · July 17, 2026, 7:59pm

Hi @user100132,
I hope you’re doing well.

I looked into the issue you reported, I’m using a Jetson AGX Orin 64GB running JetPack 6.2.1 (L4T R36.4.4). To better understand the problem, I compared the nvethernet driver between the different releases to identify any relevant differences. Here’s what I found:

I compared the nvethernet driver between Jetson Linux R36.4.4 (JetPack 6.2.1) and R38.2.1 (JetPack 7.x) to see whether there were any changes related to the tx_bytes accounting.

One interesting finding is that the accounting model is essentially the same in both versions. In both drivers, tx_bytes is updated directly from swcx->len inside osd_transmit_complete().

R36.4.4 (osd.c, around line 767):

unsigned int len = swcx->len;

ndev->stats.tx_bytes += len;

R38.2.1 (osd.c, around line 738):

unsigned int len = swcx->len;

if ((ULLONG_MAX - ndev->stats.tx_bytes) <= len) {
    ndev->stats.tx_bytes = len;
} else {
    ndev->stats.tx_bytes += len;
}

The only change here is a protection against a 64-bit counter overflow. The driver still trusts swcx->len without validating whether it contains a reasonable packet length.
Another difference I found is in ether_linux.c, where both versions still use -1 as a sentinel value for a TX software context entry.

R36.4.4 (ether_linux.c, around line 3219):

tx_swcx = tx_ring->tx_swcx + cur_tx_idx;
if (tx_swcx->len) {
    return 0;
}

tx_swcx->len = -1;

R38.2.1 (ether_linux.c, around line 3803):

tx_swcx = tx_ring->tx_swcx + cur_tx_idx;
if (tx_swcx->len) {
    ret = 0;
    goto exit_func;
}

tx_swcx->len = -1;

Later, during TX completion, that same field is read as:

unsigned int len = swcx->len;

If a completion path ever processes a software context entry whose len is still -1, it would be interpreted as:

0xffffffff = 4294967295

and that value would be added directly to tx_bytes.

I also noticed that R38.2.1 adds several validation checks around TSO header and payload calculations that are not present in R36.4.4. For example, before computing:

tx_pkt_cx->payload_len = skb->len - tx_pkt_cx->total_hdrlen;

R38.2.1 verifies that the calculated header length does not exceed skb->len, while R36.4.4 performs the subtraction without those validation checks.

Based only on the Linux wrapper sources, I cannot conclude that this is the root cause because the RM/OSI implementation that manages descriptor completion. Therefore, I cannot verify whether a context descriptor can actually reach osd_transmit_complete() with len == -1.

However, after comparing both driver versions, this appears to be one of the most interesting areas to investigate, since the byte accounting depends entirely on swcx->len, and that field is also used as a sentinel value elsewhere in the TX path.

I’m not sure if you’ve already considered this, but I thought these differences might be relevant to your investigation. If you can provide a bit more information (for example, additional logs, traces, or details about how the issue is reproduced), I’d be happy to take another look and compare it against these driver versions.

Best Regards
Carlos Quiros Gomez
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com

user100132 · July 17, 2026, 11:10pm

After downloaded the source code from https://developer.nvidia.com/downloads/embedded/L4T/r39_Release_v2.0/sources/public_sources.tbz2 and checked the Linux_for_Tegra/source/nvidia-oot/drivers/net/ethernet/nvidia/nvethernet/ , here are some findings from AI:

The Anatomy of the Bug

When TCP Segmentation Offload (TSO) is active, the Linux kernel hands a massive payload to the nvethernet driver. Before the driver can send the data, it must first send a special “Context Descriptor” to the physical hardware. This tells the MAC hardware how big the TCP headers are and what the Maximum Segment Size (MSS) is so the hardware can chop the packet up.

Step 1: The Trap is Set (ether_linux.c) In ether_linux.c, look at how ether_tx_swcx_alloc prepares this context descriptor:

	if (((tx_pkt_cx->flags & OSI_PKT_CX_VLAN) == OSI_PKT_CX_VLAN) ||
	    ((tx_pkt_cx->flags & OSI_PKT_CX_TSO) == OSI_PKT_CX_TSO) || ...) {
		tx_swcx = tx_ring->tx_swcx + cur_tx_idx;

		/* Clear the fields in the tx_swcx structure and set len to -1 */
		CLEAR_TX_SWCX(tx_swcx, flags, 0);
		CLEAR_TX_SWCX(tx_swcx, buf_virt_addr, NULL);
		CLEAR_TX_SWCX(tx_swcx, buf_phy_addr, 0);
		tx_swcx->len = OSI_INVALID_VALUE; // <--- THE SMOKING GUN
		cnt++;
		INCR_TX_DESC_INDEX(cur_tx_idx, pdata->osi_dma->tx_ring_sz);
	}

Because this descriptor holds no actual network data, NVIDIA sets its length to OSI_INVALID_VALUE (which evaluates to -1, or 0xFFFFFFFF in unsigned 32-bit math, equivalent to 4.29 Gigabytes).

Step 2: The Trap is Sprung (osd.c) When the hardware finishes transmitting, the OSI layer calls back into osd_transmit_complete() in osd.c for every descriptor it processed.

Look at the very top of osd_transmit_complete():

void osd_transmit_complete(void *priv, const struct osi_tx_swcx *swcx,
			   const struct osi_txdone_pkt_cx
			   *txdone_pkt_cx)
{
    // ... setup vars ...
	unsigned int len = swcx->len;

	if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
		ndev->stats.tx_bytes = len;
	} else {
		ndev->stats.tx_bytes += len;
	}

The driver blindly adds swcx->len to tx_bytes without checking if it is a context descriptor!

Every single time a TSO packet is sent, the hardware processes this dummy descriptor, triggers osd_transmit_complete, and adds 0xFFFFFFFF (4.29 GB) to your kernel’s tx_bytes counter.

How to Fix the Source Code

You can patch this directly in your local kernel source. Open osd.c and modify osd_transmit_complete() to ignore OSI_INVALID_VALUE when doing network statistics accounting.

Change osd.c from this:

	unsigned int len = swcx->len;

	if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
		ndev->stats.tx_bytes = len;
	} else {
		ndev->stats.tx_bytes += len;
	}

#ifdef BW_TEST

To this:

	unsigned int len = swcx->len;

	/* BUG FIX: Do not count dummy context descriptors (like TSO headers) */
	if (len != OSI_INVALID_VALUE) {
		if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
			ndev->stats.tx_bytes = len;
		} else {
			ndev->stats.tx_bytes += len;
		}
	}

#ifdef BW_TEST

Nvidia team, please review above AI comment to see if it makes sense and include in next patch.

user100132 · July 18, 2026, 12:16am

Here is how to reproduce on devkit.

Do a fresh reboot.
Check the tx bytes: cat /sys/class/net/end0/statistics/tx_bytes

In my case, immediately after reboot:

$ cat /sys/class/net/end0/statistics/tx_bytes
176094056452

For a system that literally just booted up, 176 GB of actual data transmission is impossible.

If you check what AI found above: Each dummy TSO context descriptor adds exactly OSI_INVALID_VALUE (0xFFFFFFFF or 4,294,967,295 bytes) to the counter.

If we divide the post-reboot counter by that exact number:

176094056452 / 4,294,967,295 = 41

This means that during the boot sequence, while the Orin was initializing its network stack and grabbing an IP address, it transmitted exactly 41 TSO packets. The driver glitched on every single one of them, instantly inflating the “fresh” boot counter to 176 GB before we even had a chance to log in.

WayneWWW · July 20, 2026, 7:03am

Hi,

We tried to reproduce issue on NV devkit with jp7.2 and we don’t see such value as yours even with TSO enabled.

BTW, jp7.2 is r39.2 but not any r38.x release. Are you sure you are using r39?

*$ ethtool -K end0 tso on
$ ethtool -k end0 | grep tcp-segmentation-offload
tcp-segmentation-offload: on

First boot:
$ cat /sys/class/net/end0/statistics/tx_bytes
616236*

This is just about 600KB.

user100132 · July 20, 2026, 6:20pm

@WayneWWW I am sure we are running jetpack 7.2 on this devkit. /etc/nv_tegra_release says “R39 (release), REVISION: 2.0, GCID: 45755727, BOARD: generic, EABI: aarch64, DATE: Mon Jun 1 09:28:48 PM UTC 2026” . Is your “ethtool -i end0” showing “driver: nvethernet”?

@carlos.quiros or anybody else with devkit and jetpack7.2, can you check what’s your output of “ethtool -i end0” and “cat /sys/class/net/end0/statistics/tx_bytes”?

user100132 · July 20, 2026, 7:46pm

@WayneWWW Are you using wifi on your test devkit? If so, can you disable wifi so the packets are going via the wired port? In order to trigger the bug:

make sure TSO is actually triggered. This means you need to send packets larger than mtu (1500). If the wired NIC is not your main interface, it probably only generates small dhcp or icmp packets that aren’t triggering TSO.

If you can’t generate larger packets to trigger TSO, there are two other ways (VLAN tagging or PTP) to trigger the bug.

In ether_linux.c, the ether_tx_swcx_alloc() function doesn’t just blindly create a dummy context descriptor for every packet. It explicitly checks for three specific packet flags:

	if (((tx_pkt_cx->flags & OSI_PKT_CX_VLAN) == OSI_PKT_CX_VLAN) ||
	    ((tx_pkt_cx->flags & OSI_PKT_CX_TSO) == OSI_PKT_CX_TSO) ||
	    (((tx_pkt_cx->flags & OSI_PKT_CX_PTP) == OSI_PKT_CX_PTP) && ... )) {
		tx_swcx = tx_ring->tx_swcx + cur_tx_idx;
		...
		tx_swcx->len = OSI_INVALID_VALUE;
        ...

The dummy descriptor containing OSI_INVALID_VALUE (which causes the integer underflow in osd.c) is only allocated if the network packet is flagged for VLAN tagging, Precision Time Protocol (PTP), or TCP Segmentation Offload (TSO).

WayneWWW · July 21, 2026, 2:45am

Just want to clarify again.

Please give us a method that you are sure you could reproduce issue and see the abnormal behavior on NV devkit.

Previously you only told us for something like

Do a fresh reboot.
Check the tx bytes: cat /sys/class/net/end0/statistics/tx_bytes

Now it sounds like your method needs extra step to send larger packets.

There is wifi card present on NV devkit but we are not using it. It does not connect to any AP during the test. If you have concern, then we could remove it and test again.

whitesscott · July 21, 2026, 4:50am

AGX Orin 32gb dev kit with jetpack 7.2. Immediately after reboot:

cat /sys/class/net/end0/statistics/tx_bytes
8589976881

sudo ethtool -i end0
driver: nvethernet
version: 6.8.12-1021-tegra
firmware-version:
expansion-rom-version:
bus-info: 6800000.ethernet
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

head -n1 /etc/nv_tegra_release
# R39 (release), REVISION: 2.0, GCID: 45755727, BOARD: generic, EABI: aarch64, DATE: Mon Jun  1 09:28:48 PM UTC 2026

uname -r
6.8.12-1021-tegra

And perhaps this is how to get a correct value?

sudo ethtool -S end0 | grep mmc_tx_octetcount_gb
     mmc_tx_octetcount_gb: 264346
     mmc_tx_octetcount_gb_h: 0

user100132 · July 21, 2026, 6:35am

Thanks @whitesscott for showing that you could reproduce the issue.

Sorry @WayneWWW that I assumed your system will at least have one TSO transmit. To make sure there is TSO, you can try to scp a file (say 10MB size) to another machine to trigger TSO transmit. “cat /sys/class/net/end0/statistics/tx_bytes” will then show tens of GB instead of 10MB.

whitesscott · July 21, 2026, 7:08am

Tue Jul 21 12:01:16 AM PDT 2026

cat /sys/class/net/end0/statistics/tx_bytes
425207030421

du -h cudnn_samples_v9 99M
scp -r cudnn_samples_v9 scott@chithor:~/temp/

cat /sys/class/net/end0/statistics/tx_bytes
57415234020981

Tue Jul 21 12:02:40 AM PDT 2026


sudo ethtool -S end0 | grep -E 'mmc_tx_octetcount_gb'
     mmc_tx_octetcount_gb: 116046875
     mmc_tx_octetcount_gb_h: 0

WayneWWW · July 21, 2026, 8:13am

Also want to clarify. Is this issue happened in every Jetpack version?

I mean, jp7.2 is new release. Want to identify if it is a regression or it has been there for a while.

user100132 · July 21, 2026, 7:06pm

@whitesscott Thanks for your test. copying a 99MB file triggers tx_bytes increase of 57TB is definitely showing the bug. The whole storage of the Orin is unlikely reaching 57TB. I tried the below patch and with the new nvethernet.ko, the issue goes away. If you want, you can give it a try too.

--- nvidia-oot/drivers/net/ethernet/nvidia/nvethernet/osd.c.orig	2026-06-01 20:08:43.000000000 +0000
+++ nvidia-oot/drivers/net/ethernet/nvidia/nvethernet/osd.c	2026-07-17 23:14:38.344445767 +0000
@@ -711,10 +711,13 @@
 	unsigned int chan, qinx;
 	unsigned int len = swcx->len;
 
-	if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
-		ndev->stats.tx_bytes = len;
-	} else {
-		ndev->stats.tx_bytes += len;
+	/* BUG FIX: Do not count dummy context descriptors (like TSO headers) */
+	if (len != OSI_INVALID_VALUE) {
+		if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
+			ndev->stats.tx_bytes = len;
+		} else {
+			ndev->stats.tx_bytes += len;
+		}
 	}
 
 #ifdef BW_TEST

@WayneWWW I don’t have an Orin devkit running Jetpack 6 anymore to test, but looking at the source codes Linux_for_Tegra/source/nvidia-oot/drivers/net/ethernet/nvidia/nvethernet/ from https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v5.0/sources/public_sources.tbz2 the bug is definitely there. In fact, reviewing this older code reveals something even more interesting—Jetpack 6 is actually missing a “band-aid” fix that NVIDIA attempted to add in Jetpack 7, making the bug even more raw in this version. In the Jetpack 7 source code we looked at earlier, osd_transmit_complete() had an extra if/else block checking (ULLONG_MAX - ndev->stats.tx_bytes) <= (len).

Jetpack 6 completely lacks this check. It seems that between Jetpack 6 and Jetpack 7, an NVIDIA engineer noticed that the tx_bytes counter was rolling over ULLONG_MAX and causing issues. They added the if/else block to prevent the absolute rollover, but they completely missed the root cause—that the driver was feeding a 4.29 GB context descriptor into the math in the first place!

WayneWWW · July 22, 2026, 1:56am

Thanks for pointing out this. will look into this.

whitesscott · July 22, 2026, 5:29am

@WayneWWW

Issue appears on Thor too:

cat /sys/firmware/devicetree/base/model |tee osdc.txt
NVIDIA Jetson AGX Thor Developer Kit

head -n1 /etc/nv_tegra_release |tee -a osdc.txt
R39 (release), REVISION: 2.0

uptime |tee -a osdc.txt
 22:06:24 up  2:55,  3 users,  load average: 0.83, 2.93, 6.13
date |tee -a osdc.txt
Tue Jul 21 10:06:24 PM PDT 2026

cat /sys/class/net/mgbe2_0/statistics/tx_bytes | tee -a osdc.txt
4698904582261

sudo reboot

uptime |tee -a osdc.txt
 22:09:25 up 0 min,  3 users,  load average: 2.91, 0.78, 0.26
date |tee -a osdc.txt
Tue Jul 21 10:09:25 PM PDT 2026
cat /sys/class/net/mgbe2_0/statistics/tx_bytes | tee -a osdc.txt
281842

du -h NVIDIA_VPI-4.0-samples/
19M NVIDIA_VPI-4.0-samples/
scp NVIDIA_VPI-4.0-samples scott@chiorin:~/temp/

uptime
 22:12:04 up 3 min,  3 users,  load average: 0.97, 0.92, 0.41
Tue Jul 21 10:12:05 PM PDT 2026

cat /sys/class/net/mgbe2_0/statistics/tx_bytes | tee -a osdc.txt
2701554571195

date
Tue Jul 21 10:12:18 PM PDT 2026

sudo ethtool -S mgbe2_0 | grep -E 'mmc_tx_octetcount_gb' | tee -a osdc.txt
     mmc_tx_octetcount_gb: 21113866
     mmc_tx_octetcount_gb_h: 0

user100132 · July 22, 2026, 5:50am

The issue is with nvethernet driver. The “mgbe2_0” you mentioned seems to also use nvethernet

$ ethtool -i mgbe2_0
driver: nvethernet
version: 6.8.12-1021-tegra
firmware-version: 
expansion-rom-version: 
bus-info: a808d10000.ethernet
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

WayneWWW · July 23, 2026, 9:06am

please try with this patch.

diff --git a/drivers/net/ethernet/nvidia/nvethernet/osd.c b/drivers/net/ethernet/nvidia/nvethernet/osd.c
index 7fab204..7d6290d 100644
--- a/drivers/net/ethernet/nvidia/nvethernet/osd.c
+++ b/drivers/net/ethernet/nvidia/nvethernet/osd.c
@@ -711,12 +711,6 @@
 	unsigned int chan, qinx;
 	unsigned int len = swcx->len;
 
-	if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
-		ndev->stats.tx_bytes = len;
-	} else {
-		ndev->stats.tx_bytes += len;
-	}
-
 #ifdef BW_TEST
 	if (pdata->test_tx_bandwidth == OSI_ENABLE) {
 		return;
@@ -756,6 +750,17 @@
 			netif_tx_wake_queue(txq);
 			netdev_dbg(ndev, "Tx ring[%d] - waking Txq\n", chan);
 		}
+
+		/* Update TX byte counter. Only descriptors with valid SKB
+		 * (buf_virt_addr) should contribute to stats. Context
+		 * descriptors have NULL skb and don't transmit actual data.
+		 */
+		if ((ULLONG_MAX - ndev->stats.tx_bytes) <= (len)) {
+			ndev->stats.tx_bytes = len;
+		} else {
+			ndev->stats.tx_bytes += len;
+		}
+
 		if (ndev->stats.tx_packets == ULLONG_MAX) {
 			ndev->stats.tx_packets = 0;
 		} else {

Topic		Replies	Views
Jetpack 6.0: mgbe_payload_cs_err increasing after MACsec removal on orin Jetson AGX Orin ethernet	28	543	April 9, 2025
Solve the R35.3.1 Jetson agx orin LAN7800 USB to Gigabit PHY bandwidth rate is too low, only about 250M Jetson AGX Orin board-design , ethernet	21	588	June 26, 2025
Jetson Xavier NX & lan78xx Jetson Xavier NX ethernet , yocto	6	782	November 21, 2023
Nvethernet PTP bug Jetson AGX Orin kernel , time-synchronization	28	3967	January 16, 2024
Multiple prx_desc errors on TX2 ethernet Jetson TX2	9	950	July 22, 2019
Package loss on bandwidth reduced network - AGX Orin Jetson AGX Orin ethernet , networking	19	2443	March 28, 2023
10G ethernet for jetson tx1 using pci-e x4 Jetson TX1	55	9850	September 11, 2017
Kernel panic when the pcie connecting with x86 and jetson orin nano Jetson Orin Nano pcie	5	239	July 5, 2024
Ethernet speed issue for 4K USB capture and streaming Jetson TX1	18	2689	August 3, 2017
Nvidia AGX Orin USB connection issues Jetson AGX Orin usb , nvbugs	22	8332	May 9, 2023

Nvethernet driver: TX byte counter integer underflow (petabyte spikes) when TSO is enabled on Jetpack 7.2 (Orin AGX)

System Configuration:

Description of the Issue:

How to Reproduce:

Technical Evidence (Debugging Logs):

1. Corrupted Kernel Counter (Software):

2. Accurate Hardware Registers (ethtool -S):

3. Driver Information:

Workaround:

Request to NVIDIA Engineering:

The Anatomy of the Bug

How to Fix the Source Code

Related topics

2. Accurate Hardware Registers (`ethtool -S`):