49.7 day bug: Camera RTCPU gone bad

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

There have been many reports of the timer overflow that makes the camera grabbing crash after 49.7 days with errors like

Apr  1 07:30:49 rc-visard-ng-1422324319939 kernel: [4295001.509376] [RCE] BUG: common/rtcpu-queue.c:22 [safe_queue_recv] "ERROR 0x2000000a: safe_queue_recv failed"
Apr  1 07:30:49 rc-visard-ng-1422324319939 kernel: [4295001.545218] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!

We are still encountering this severe but on our Orin Nano and tried many versions, none of which fixed the issue.
Affected L4T releases with rce-fw sha1:

  • L4T 35.5.0 with rce-fw sha1=55ecd57df677fd722e795b49ccb283f065b222a1
  • L4T 35.5.0 with rce-fw sha1=761d9d48563b4131a5af3b34f2c9f19a713a222d (rce-fw from fused-orin-nano-35-5-0-camera-rtcpu-gone-bad )
  • L4T 35.6.4 with rce-fw sha1=738ae3ed66400a5562093b958d218bf596c1f623
  • L4T 36.4.3 with rce-fw sha1=e2238c99959d2df9350d393f04e1f44e5bef98cb

It is not even listed under known issues in the release notes!!
Is there a rce-fw that fixes this for L4T 35.X? Or when will it be available?

Is it really fixed with 36.5.0 as https://forums.developer.nvidia.com/t/jetson-agx-orin-camera-failure-safe-queue-recv-0x2000000a-camera-rtcpu-gone-bad-no-recovery-until-reboot-connect-tech-carrier/358992/15 suggests?
I can start another long-term test with 36.5.0 now, but it will obviously take another 49 days until I can be sure!

We really need a fix for this!

5

JP6.2.2 should include the patch.

6

And what about a patch for the JetPack 5.x/L4T 35.x series?

7

Sorry to tell no patch for r35 release.

It’s better to upgrade to r36.5 and later release.

Thanks

8

What is the timeline for a patch for 35.x? Is there a rce-fw binary available that fixes this, even if it is not yet in any 35.x release?
Moving to r36 involves a lot of time/work and also a lot of changes for our customers (e.g. new base container images, etc).

9

The RCE firmware code base is much different to JP6 to apply the patch to JP5.

Thanks