GMSL2 link lock failure for MAX96712 (part 2)

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hi. I had previously posted here about a link lock failure for MAX96712. I did not receive an answer from Nvidia before my post was automatically closed. Can I please get more information from Nvidia about possible mitigations for this issue?

Thank you.

3

Dear @jmanning2,
We notice rebooting the target has fixed the issue. As asked last time, can you share the nvsipl_camera and syslog.

4

We notice rebooting the target has fixed the issue

So Nvidia has done independent testing and is able to replicate this issue? Rebooting isn’t a resolution, it’s a workaround.

As mentioned in my last post, I dug a bit into the MAX96712 deserializer driver in the SDK and I see a TODO comment (cdi_max96712.c line 2583) mentioning that the link lock may take >100ms intermittently. I wonder if this may have something to do with this issue?

As posted last time, here is the syslog:

Apr 01 12:58:51 tegra-ubuntu  [5257459663] Module_id 30 Severity 2 : devices/MAX96712DeserializerDriver/cdi_max96712.c 2604
Apr 01 12:58:51 tegra-ubuntu  [5257464454] Module_id 30 Severity 2 : MAX96712: GMSL2 link lock not detected on link 0
Apr 01 12:58:51 tegra-ubuntu  [5257465052] Module_id 30 Severity 2 : devices/MAX96712DeserializerDriver/CNvMMax96712.cpp 254
Apr 01 12:58:51 tegra-ubuntu  [5257465216] Module_id 30 Severity 2 : MAX96712: MAX96712SetDefaults failed with NvMedia error 7
Apr 01 12:58:51 tegra-ubuntu  [5257465559] Module_id 30 Severity 2 : src/devblk/common/ddi/CNvMDevice.cpp 71
Apr 01 12:58:51 tegra-ubuntu  [5257465723] Module_id 30 Severity 2 : Driver failed Init() call 10
Apr 01 12:58:51 tegra-ubuntu  [5257466563] Module_id 30 Severity 2 : src/devblk/common/core/CNvMDeviceBlock.cpp 793
Apr 01 12:58:51 tegra-ubuntu  [5257466720] Module_id 30 Severity 2 : Deserializer init failed with SIPL error 10
Apr 01 12:58:51 tegra-ubuntu  [5257467027] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 616
Apr 01 12:58:51 tegra-ubuntu  [5257467235] Module_id 30 Severity 2 : Device block init failed. Status: 10
Apr 01 12:58:51 tegra-ubuntu  [5257467921] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 244
Apr 01 12:58:51 tegra-ubuntu  [5257468646] Module_id 30 Severity 2 : Device block action failed for action 1
Apr 01 12:58:51 tegra-ubuntu  [5257468906] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 246
Apr 01 12:58:51 tegra-ubuntu  [5257469361] Module_id 30 Severity 2 : Device block action failed for block 0
Apr 01 12:58:51 tegra-ubuntu  [5257469663] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 278
Apr 01 12:58:51 tegra-ubuntu  [5257469932] Module_id 30 Severity 2 : Failed to initialize device blocks
Apr 01 12:58:51 tegra-ubuntu  [5257470140] Module_id 30 Severity 2 : src/core/CNvMCamera.cpp 1142
Apr 01 12:58:51 tegra-ubuntu  [5257470332] Module_id 30 Severity 2 : DeviceBlock manager init failed
Apr 01 12:58:51 tegra-ubuntu  [5257795614] Module_id 30 Severity 2 : src/core/pipelineMgr/spmgr/pipeline/blocks/CNvFusaISPBlock.cpp 450
Apr 01 12:58:51 tegra-ubuntu  [5257798515] Module_id 30 Severity 2 : FusaISP: Skiping Deinit as block was not initialized 1
Apr 01 12:58:51 tegra-ubuntu  [5267229908] Module_id 30 Severity 2 : src/devblk/common/core/CNvMDeviceBlock.cpp 1235
Apr 01 12:58:51 tegra-ubuntu  [5267231855] Module_id 30 Severity 2 : CNvMDeviceBlock must be in the initialized state or the stopped state before attempting to deinitialize
Apr 01 12:58:51 tegra-ubuntu  [5267232216] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 638
Apr 01 12:58:51 tegra-ubuntu  [5267232378] Module_id 30 Severity 2 : Device block deinit failed. Status: 6
Apr 01 12:58:51 tegra-ubuntu  [5267233226] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 244
Apr 01 12:58:51 tegra-ubuntu  [5267234330] Module_id 30 Severity 2 : Device block action failed for action 4
Apr 01 12:58:51 tegra-ubuntu  [5267234567] Module_id 30 Severity 2 : src/core/CNvMDeviceBlockMgr.cpp 246
Apr 01 12:58:51 tegra-ubuntu  [5267234830] Module_id 30 Severity 2 : Device block action failed for block 0
Apr 01 12:58:51 tegra-ubuntu  [5276615257] Module_id 30 Severity 2 : src/devblk/common/core/CNvMDeviceBlock.cpp 1235
Apr 01 12:58:51 tegra-ubuntu  [5276616740] Module_id 30 Severity 2 : CNvMDeviceBlock must be in the initialized state or the stopped state before attempting to deinitialize

Here is the nvsipl_camera output:

$ sudo /opt/nvidia/drive-linux/samples/nvmedia/nvsipl/test/camera/nvsipl_camera --platform-config HZKJ_IMX728_ES2_V2_070FOV_CPHY_x4 --link-enable-masks "0x0001 0x0000 0x0000 0x0000" --showfps --enableRawOutput
Pipeline: 0 ISP Output: 0 is using YUV 420 SEMI-PLANAR UINT8 BL REC_709ER
Pipeline: 0 ISP Output: 1 is using YUV 420 SEMI-PLANAR UINT8 BL REC_709ER
Pipeline: 0 ISP Output: 2 is using RGBA PACKED FLOAT16 PL SENSOR_RGBA
nvsipl_camera: ERROR: NvSIPLCamera Init failed

nvsipl_camera: ERROR: Master initialization failed. status: 10
5

No. I meant the issue was resolved after reboot. We could not reproduce the issue.

I am checking internally on this issue and update you.

7

Hi, it’s been almost two weeks so I’m worried this post will be auto-closed again, is there any update from Nvidia on this issue?

8

Hi @jmanning2,
Just confirming if You received the camera drivers from quanta?

A bug is filed internally for tracking the issue. I will keep you updated here and seek the requested info from you.
The issue does not look like related to link lock time. We already using 500ms.
If you can share any repro code/steps it would help the to debug.

9

Yep, the drivers are from Quanta.

Sounds good, thanks!

And gotcha, I was just assuming that could be related given the syslog entry about link lock not detected.

As far as repro, I haven’t been able to nail down criteria to reproduce. It seems to happen very intermittently, we’re working to try and track the frequency/conditions better.

10

Bumping again to avoid auto-close, does Nvidia have any update?

11

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Dear @jmanning2,
Can you check the power of deserializer(use i2cget) when this issue occur. It could be problem with that.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.