TX2 Ethernet RX performance on custom carrier board

Hello,
I have a performance issue with Ethernet on a TX2.
I have a custom carrier card on which a Marvell 7-port GbE switch is installed. This is directly connected to the TX2 via 10nF/50V AC coupling capacitors.
The routing from the Jetson to the switch seems correct: differential pairs in controlled impedances @ 100Ohm, track lengths < 30mm.
We do not use any of the led ACK signals.

The negotiation is done correctly between the TX2 and the switch:

root@tegra-ubuntu:/home/nvidia# dmesg |grep eth
[    3.396759] eqos 2490000.ether_qos: Setting local MAC: 0 4 4b c5 44 84
[    3.432089] usbcore: registered new interface driver cdc_ether
[    8.175803] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   10.024088] using random self ethernet address
[   10.044435] using random host ethernet address
[   10.503465] using random self ethernet address
[   10.521289] using random host ethernet address
[   11.442465] eqos 2490000.ether_qos eth0: Link is Up - 1Gbps/Full - flow control off
[   11.464860] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   11.493700] eqos 2490000.ether_qos: EQOS ISO BW 81920 reservation failed with 0

The link seems stable and I have no apparent disconnection or error.

eth0      Link encap:Ethernet  HWaddr 00:04:4b:c5:44:84
          inet addr:192.168.42.49  Bcast:192.168.42.255  Mask:255.255.255.0
          inet6 addr: fe80::204:4bff:fec5:4484/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:187911 errors:0 dropped:0 overruns:0 frame:0
          TX packets:41867 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:210992912 (210.9 MB)  TX bytes:2150358443 (2.1 GB)
          Interrupt:42

root@tegra-ubuntu:/home/nvidia# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: ug
        Wake-on: g
        Link detected: yes

But when I perform a speed test, I am capped at 100~200Mbits/s in RX, while in TX I am very close to Gbits/s (this one is my best)

root@tegra-ubuntu:/home/nvidia# iperf -c 192.168.42.40 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.42.40, TCP port 5001
TCP window size:  280 KByte (default)
------------------------------------------------------------
[  5] local 192.168.42.49 port 41120 connected with 192.168.42.40 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec  1.00 GBytes   861 Mbits/sec
[  4] local 192.168.42.49 port 5001 connected with 192.168.42.40 port 33848
[  4]  0.0-10.0 sec   246 MBytes   207 Mbits/sec

(I’ve been doing tests for quite some time)

The test topology is as follows:
TX2 <-> Marvell switch <-> PC

If I do my tests with the following topology, I have no worries:
PC <-> Marvell switch <-> PC

I was confronted with a first sleight of hand regarding the activation of USB on a Custom card (dtb modification), and I wonder if the same kind of thing is possible in the case of Ethernet.

The only thing that could put me on the hint is the following trace:

[   11.493700] eqos 2490000.ether_qos: EQOS ISO BW 81920 reservation failed with 0

But I have no idea what this is about, “BW” may be related to BandWidth…

The Marvell switch is known and has been used in other design without issues.

Thank you for your help,

Thomas.

Do you mean you replace the PHY of BCM89610? The iso bw (bandwidth) seems an issue.
Could you debug into “kernel/kernel-4.4/drivers/net/ethernet/nvidia/eqos/drv.c” and see which reason that tegra_isomgr_reserve is not working?

Hello and thank you for the answer,

No, not at all, I didn’t modify the TX2 module.
I simply put an ethernet switch on my carrier board, connected to the Jetson via its available MDI GbE ports on the main connector.

I have just modified the TX2 link <-> Switch Marvel by replacing the AC decoupling capacitors with a 1:1 GbE transformer.
The results are identical, no degradation in TX or improvement in RX.

So, I’ll try to debug the ethernet driver, but if it’s a serious clue it’s worth a look.