Camera connected to Jetson Orin Nano stops after long test: Seeking solutions for nvargus-daemon errors

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
, ,
1

I am using the Jetson Orin Nano, and when I run the camera for an extended period, it stops. The time until it stops can be as short as 1 hour or as long as about 48 hours.

—Error location —
The error occurs in gstnvarguscamerasrc.cpp, around line 405:

  UniqueObj<Frame> frame(iFrameConsumer->acquireFrame(ACQUIRE_FRAME_TIMEOUT, &frame_status));

The frame_status returns as Argus::STATUS_TIMEOUT.

— nvargus-daemon log —

Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: Module_id 30 Severity 2 : (fusa) Error: InvalidState Status syncpoint signaled but status value not updated in:/capture/src/fusaViHandler.cpp 817
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: Module_id 30 Severity 2 : (fusa) Error: InvalidState  propagating from:/capture/src/fusaViHandler.cpp 759
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: === python3[2634]: CameraProvider initialized (0xffff75392cf0)SCF: Error InvalidState:  Corr Error Received for sensor 0 .. Continuing!
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]:  (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 643)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: Module_id 30 Severity 2 : (fusa) Error: ResourceAlreadyInUse All captures are already pending, no idle captures available in:/capture/src/fusaViHandler.cpp 633
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: Module_id 30 Severity 2 : (fusa) Error: ResourceAlreadyInUse  propagating from:/capture/src/fusaViHandler.cpp 475
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse:  (propagating from src/services/capture/FusaCaptureViCsiHw.cpp, function startCaptureInternal(), line 866)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse:  (propagating from src/services/capture/CaptureRecord.cpp, function doCSItoMemCapture(), line 536)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse:  (propagating from src/services/capture/CaptureRecord.cpp, function issueCapture(), line 483)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1530)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1359)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error ResourceAlreadyInUse: Worker thread CaptureScheduler frameStart failed (in src/common/Utils.cpp, function workerThread(), line 133)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error Timeout:  (propagating from src/api/Buffer.cpp, function waitForUnlock(), line 644)
Oct 23 13:00:32 ubuntu nvargus-daemon[1046]: SCF: Error Timeout:  (propagating from src/components/CaptureContainerImpl.cpp, function returnBuffer(), line 426)
Oct 23 13:00:33 ubuntu nvargus-daemon[1046]: SCF: Error Timeout:  (propagating from src/components/amr/Snapshot.cpp, function waitForNewerSample(), line 91)
Oct 23 13:00:33 ubuntu nvargus-daemon[1046]: SCF_AutocontrolACSync failed to wait for an earlier frame to complete.
Oct 23 13:00:33 ubuntu nvargus-daemon[1046]: SCF: Error Timeout:  (propagating from src/components/ac_stages/ACSynchronizeStage.cpp, function doHandleRequest(), line 126)
Oct 23 13:00:33 ubuntu nvargus-daemon[1046]: SCF: Error Timeout:  (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 137)
Oct 23 13:00:33 ubuntu nvargus-daemon[1046]: SCF: Error Timeout: Sending critical error event for Session 0
Oct 23 13:00:33 ubuntu nvargus-daemon[1046]:  (in src/api/Session.cpp, function sendErrorEvent(), line 1039)
Oct 23 13:00:41 ubuntu nvargus-daemon[1046]: SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Oct 23 13:00:41 ubuntu nvargus-daemon[1046]: Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc for sensor guid 0 count 2136195840

— Environment —

  • Jetson Orin nano module: P3767-0003 (No SD card slot model)
  • J401 Carrier Board: J401 Carrier Board from seeed studio.
  • Camera: IMX477 4K Camera from Arducam
  • Jetpack: 5.1.2
  • L4T: 35.4.1

— Additional Information —

  • I feel that the chance of occurrence is higher when the system is under heavy load, but it sometimes occurs even with a simple GStreamer command.
  • When I tested the same camera with Jetson Xavier NX under the same conditions, the camera did not stop. (I used Jetpack: 5.0.2)

— Question —

  • What is the cause of this error?
  • Are there any measures to prevent this error and stabilize camera running?
3

Do you run single camera or two cameras?

Thanks

4

Thank you for replay.

I am using a single camera. It is connected to the CAM0 port.

6

Do you have any advice about this topic? @ShaneCCC

After that, I tried boost all the VI/CSI/ISP clocks by command below refered from this thread.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

However, after 30 hours, camera stopped.
The error message is the same as described in the question.

8

Hi,
This’s know issue still on going. Will update to you once get fix.

Thanks

9

I understand that you are working on the issue.
Thank you for your reply.

10

Is JetPack 6.0 DP include fix of this issue?
Or is there any update for this issue?

11

Not sure if J6.0DP have the same issue or not.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.