Xavier AGX ethernet truncates RX frames

[jakub.piskorz]

Hello,

I am using Xavier AGX on a custom carrier board.
Jetson Linux release: R35.4.1
JetPack release: 5.1.2

I’m unable to correctly receive packets from the switch connected via RGMII (BCM53125S), they are being truncated by the AGX module somewhere in HW.

Here is what it looks like on tcpdump (command used: tcpdump -e -XX -i end0 -Q in):

11:37:11.270243  [|brcm-tag]
        0x0000:  ffff ffff ffff f0a7 313f af4c 0000       ........1?.L..

Here is what data looks like on RGMII lines:

55 55 55 55 55 55 55 D5 FF FF FF FF FF FF F0 A7 
31 3F AF 4C 00 00 20 00 81 00 00 00 88 99 25 00 
00 00 F0 A7 31 3F AF 4C 00 00 00 00 00 F0 A7 31 
3F AF 4C FF 07 A3 D9 28 42 D4 D4 B2 42 4B 74 7E 
77 45 3C 9B AE 65 BA C3 A9 EC 8A E3 3C 88 8D E7 

(attached picture that shows captured signals:

Switch is configured to include Broadcom tag 4octetes which are inserted before ethertype L2 field (here on packet dump val 0x00002000).

I was trying to investigate where this truncation happened, and my conclusion is that it is very early, probably in hardware. First software place that mentions rx packet size I’ve found is in nvidia/drivers/net/ethernet/nvidia/nvethernet/nvethernetrm/osi/dma/osi_dma_txrx.c, function osi_process_rx_completions, I’ve added a print after line rx_pkt_cx->pkt_len = rx_desc->rdes3 & RDES3_PKT_LEN; and it outputs 14 for every received packet.

I think the problem is with this broadcom tag added, eqos is performing some automatic filtering based on ethertype tag, and this frame has 0x0000 there and it fails to handle it correctly, maybe it is expecting a VLAN tag there or something else.

How can I force ethernet (eqos) to pass the entire packet to the kernel?

[kayccc]

Sorry for the late response.
Is this still an issue to support? Any result can be shared?

[jakub.piskorz]

Hi @kayccc,

yes, this is still an issue.

I’ve tried to prevent filtering by disabling interface capabilities reported by ethtool -k end0 like:

rx-vlan-offload: on
tx-vlan-offload: on
rx-vlan-filter: on

by setting them to off but it was not it, packets still were truncated.

[jonathan.wang]

Hi @jakub.piskorz,

I’ve managed to get DSA tagging working on the xavier eqos by forcibly disabling the IP checksum offloading which does not take into account the DSA tag when calculating an offset:

Index: nvidia/drivers/net/ethernet/nvidia/nvethernet/ether_linux.c
===================================================================
--- nvidia.orig/drivers/net/ethernet/nvidia/nvethernet/ether_linux.c
+++ nvidia/drivers/net/ethernet/nvidia/nvethernet/ether_linux.c

@@ -6211,10 +6163,15 @@ static void ether_set_ndev_features(stru
        features |= NETIF_F_GSO_UDP_L4;
 #endif

+#if (!defined(CONFIG_NET_DSA) && !defined(CONFIG_NET_DSA_MODULE))
+    /* Force IP checksum offloading off when DSA tags are used as the
+     * nvidia eqos checksum offloading engine cannot handle the custom
+     * offsets for calculating checksums from DSA tag insertion */
    if (pdata->hw_feat.tx_coe_sel) {
        features |= NETIF_F_IP_CSUM;
        features |= NETIF_F_IPV6_CSUM;
    }
+#endif

this patch snippet may not apply directly as there are a lot more modifications involved in getting the mdio to function when the port is in admin down state as you have encountered but hope this helps.