[Failed to get PCS block lock] on custom board's AQR113C

[ghdlghdl0558]

hello

We are making a Jetson orin custom board and using AQR113C.
However, there is one issue regarding the ethernet connection.

When I used marvell’s AQR113C firmware, ethernet did not work, so I read and used devkit’s AQR113C firmware according to below link.

Loading Firmware for 10GbE AQR113C on bootup

So the ethernet started working after a lot of troubleshooting of the AQR113C’s firmware. We had to change a couple of registers of Nvidia’s version of the firmware that we took from the flash connected to the dev board’s AQR113C. Marvel’s standard firmware didn’t work probably because there is a section on the driver that checks for specific firmware to be loaded into the flash.

Then two strange phenomena occurred.

1. When connecting ethernet to another ethernet device during booting, booting does not proceed.
   There was a delay in the UEFI firmware part before, but booting proceeded after about 2 minutes, even when the AQR113C firmware was not written to flash, .
   After use devkit’s AQR113C firmware, booting does not proceed at all on that part.
   If it is not connected to another ethernet device, booting proceeds normally.

2. When connected to another ethernet device after booting, the ethernet connector connection is recognized by the other device.
   However, ethernet is not recognized on the orin side of the custom board.
   At the same time, the following log appears in the kernel.

[ 38.333041] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][470][type:0x4][loga-0x0] Failed to get PCS block lock

When checking the status with mdio, it appears that link up has been recognized in AQR113C.

Our custom board has an EEPROM, but no data has been written. Could this be the cause?
We are using Jetpack 5.1.2 BSP.
What causes it?

thank you

[WayneWWW]

When connecting ethernet to another ethernet device during booting, booting does not proceed.

Share the log then.

Our custom board has an EEPROM, but no data has been written. Could this be the cause?

Does not matter. You should always disable carrier board eeprom read from bootloader.

[ghdlghdl0558]

I tested it again, and i found that it booted after a very long time.

attached is log
booting_log_ethernet_connected.log (96.2 KB)

It took very long time between

E/TC:?? 00 get_rpc_alloc_res:645 RPC allocation failed. Non-secure world result: ret=0xffff0000 ret_origin=0
E/LD:   init_elf:486 sys_open_ta_bin(bc50d971-d4c9-42c4-82cb-343fb7f37896)
E/TC:?? 00 ldelf_init_with_ldelf:131 ldelf failed with res: 0xffff000c

and

Jetson UEFI firmware (version 4.1-33958178 built on 2023-08-01T19:34:02+00:00)
ESC   to enter Setup.
F11   to enter Boot Manager Menu.
Enter to continue boot.

I also tried phy reset as in the following answer, but it still didn’t work.

AQR113C PHY firmware update corrupted

Dear @abelton, Could you check trying a Hardware reset by writing 0x1 to register 1E.2681 on AQR113C via MDIO to see if it recovers it from a bad state. Please see Orin can not read the PHY reg data - #15 by WayneWWW and try if phytool write mgbe1_0/0:0x1e/0x2681 0x1 helps?

And after the [xps_lane_bring_up] error appears in the kernel log, ethernet-related logs no longer appear even if i unplug and plug the cable.

Is there anything else we can try next?

Thanks

[ghdlghdl0558]

hello.

I would like to inquire further about other things we have tried.

On another custom board we made, we confirmed that AQR113C operates normally with the AQR113C firmware from nvidia devkit.

So, to check if it was a module problem, I tried switching modules on a board that was operating normally.
However, the same [Failed to get PCS block lock] error occurred in the module that was operating normally.
The module that had an error on the first custom board worked normally when moved to the second custom board.

So I thought it was a problem with AQR113C, so I replaced the IC with another one, but [Failed to get PCS block lock] still occurred.

When I reset phy by executing 0x1e/0x2681 0x1 with MDIO, the connection returned normally as shown in the capture below, but then an error occurred again.
The corresponding kernel log is as follows.

[   362.535614] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][470][type:0x4][loga-0x0] Failed to get PCS block lock
[   367.657784] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][477][type:0x4][loga-0x0] PCS block lock SUCCESS
[   368.677783] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][477][type:0x4][loga-0x0] PCS block lock SUCCESS
[   369.701783] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][477][type:0x4][loga-0x0] PCS block lock SUCCESS
[   370.727611] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][470][type:0x4][loga-0x0] Failed to get PCS block lock
[   371.749774] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][477][type:0x4][loga-0x0] PCS block lock SUCCESS
[   372.773781] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][477][type:0x4][loga-0x0] PCS block lock SUCCESS
[   373.799618] nvethernet 6810000.ethernet: [xpcs_lane_bring_up][470][type:0x4][loga-0x0] Failed to get PCS block lock

Is there anything else we can see?

thank you