Detecting / Gracefully Failing from Bad CSI/Camera State

VB2K · December 22, 2025, 8:49pm

Hello,

Sometimes my Orin gets into a state where when I try to stream cameras w/ argus I get a lot of the following warnings/errors:

Module_id 30 Severity 2 : (fusa) Error: Timeout propagating from:/capture/src/fusaViHandler.cpp 820
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 3, capture sequence ID = 0 draining session frameStart events 3
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 529)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 5, capture sequence ID = 0 draining session frameEnd events 3
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Sensor 3 already in same state
(in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 3, capture sequence ID = 0 draining session frameEnd events 3
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 0 draining session frameEnd events 3
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
Module_id 30 Severity 2 : (fusa) Error: Timeout propagating from:/capture/src/fusaViHandler.cpp 820
Module_id 30 Severity 2 : (fusa) Error: Timeout propagating from:/capture/src/fusaViHandler.cpp 820
SCF: Error InvalidState: Sensor 0 already in same state
(in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Sensor 5 already in same state
(in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 5, capture sequence ID = 0 draining session frameStart events 3
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 529)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 0 draining session frameStart events 3
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 529)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)

Sometimes this causes my program that calls into argus to fully crash, and sometimes it limps along in a degraded state (non-deterministic).

I think its introduced when the cameras are ‘hotplugged’, or streaming is suddenly stopped from under the Orin, or something is putting the VI/RTCPU system into a bad state (my guess). As far as I can tell, the only way to recover from this is a full reboot (sudo reboot) of the Orin system.

Is there any way to detect that the Orin is in this state beforehand so I can raise a warning and not even attempt to stream, since trying to stream seems to be non-deterministic in how it fails?
Is there any way to recover from this and reset RTCPU/VI without reboot? I have tried all the naive things of restarting argus, unloading/reloading my camera kernel drivers, etc. But nothing as far as I know is doing a proper reset of the hardware that manages the actual MIPI CSI Busses which I think is the real problem.

ShaneCCC · December 23, 2025, 1:47am

What’s the BSP version?

cat /etc/nv_tegra_release

VB2K · December 23, 2025, 1:51am

# R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan 8 01:49:37 UTC 2025

# KERNEL_VARIANT: oot

TARGET_USERSPACE_LIB_DIR=nvidia

TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

ShaneCCC · December 23, 2025, 1:53am

Apply below changes to verify,

[Argus] Long run stability issue.
https://forums.developer.nvidia.com/t/318999/5

ShaneCCC · January 28, 2026, 3:41am

The libs resolved some memory leak for camera long run stability.

Thanks

ShaneCCC · January 28, 2026, 6:40am

My bad the libs is fix possible segment fault in multi-camera streaming. There is possible concurrent access to moduleString in running multi cameras.

The memory leak fix should be below.

[gpu/host1x-fence] Significant kernel memory leak when using Argus camera
https://forums.developer.nvidia.com/t/325399/4/

VB2K · February 12, 2026, 3:06am

Hi ShaneCCC,

I have some updates from our side:

I have now tried the updated binaries you mentioned as well as the following kernel patches:

[Argus] Long run stability issue.
https://forums.developer.nvidia.com/t/318999/5

[gpu/host1x-fence] Significant kernel memory leak when using Argus camera
https://forums.developer.nvidia.com/t/325399/4/

[Camera][VI] Add semaphore to avoid multi-cam race condition
https://forums.developer.nvidia.com/t/337341/12/

[Camera][VI] Memory leakage fix with error recovery
https://forums.developer.nvidia.com/t/337609/11/

However, I still get the following style error:

During moments of high memory pressure (loading in ML models using pytorch concurrently uses between 30-60GB of RSS RAM), cameras will fail with this message (similar to my original forum post):

Module_id 30 Severity 2 : (fusa) Error: InvalidState Status syncpoint signaled but status value not updated in:/capture/src/fusaViHandler.cpp 861 

Module_id 30 Severity 2 : (fusa) Error: InvalidState propagating from:/capture/src/fusaViHandler.cpp 803 

SCF: Error InvalidState: Corr Error 8 Received for sensor 6 .. Continuing!

And will then cascade down to all the sensors.

Sometimes this is recoverable with a restart of our argus application, and other times requires a reboot.

I notice that usually on the times it requires a reboot, the same CSI bus sensors keep failing (we have GMSL2 so up to 4 sensors on 1 CSI bus using virtual channels). I feel like these would be resolvable without a reboot if we had a way to reset VI/NVCSI hw blocks.

(We also see this same error sometimes when we aren’t explicitly loading in models/experiencing high pressure)
We see this with all the aforementioned patches.

I can recreate failures like this even with gstreamer like so:

run the following command:

gst-launch-1.0 -e \
nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=1 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=2 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=3 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=4 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=5 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=6 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \
nvarguscamerasrc sensor-id=7 ! ‘video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=17/1’ ! fakesink sync=false \

(In our case these are the resolution for each sensor)

And in a separate terminal, create memory pressure (RSS memory only, not nvmap or CPU pressure)

stress --vm 1 --vm-stride 4096 --vm-bytes 30G

(Depending on what else is running on your system, you may need to increase vm-bytes)

Notably, we don’t get an OOM so this isn’t an extremely traumatic allocation, at the most it should result in dropped frames, not failed streaming. (although ideally Argus should pre-allocate all the memory it needs and not require dynamic allocations that fail with pressure)

I think there may be other reasons than just memory pressure that causes us to fail streaming, but this is just the one I can re-create easily.

Do you have any suggestions?

I have also tried the updated libnvfusacap.so you mentioned, and it did not affect this.

libnvfusacap.so.r36.4.dis_notify.zip (89.9 KB)

Can you help provide any more fixes to stability, or help us identify what parts of argus are susceptible to the errors mentioned?

Thank you!

ShaneCCC · February 12, 2026, 4:25am

Please if still the same error.

Suppose repelace the libnvfusacap.so.r36.4.dis_notify.zip should not show the same error.

VB2K:

Module_id 30 Severity 2 : (fusa) Error: InvalidState Status syncpoint signaled but status value not updated in:/capture/src/fusaViHandler.cpp 861 

Module_id 30 Severity 2 : (fusa) Error: InvalidState propagating from:/capture/src/fusaViHandler.cpp 803 

SCF: Error InvalidState: Corr Error 8 Received for sensor 6 .. Continuing!

VB2K · February 19, 2026, 10:42pm

Hi ShaneCCC, the error with the new binary is of this form:

We get 2 types of errors:

Argus keeps going for a bit with lots of drops, then fails with this message (and subsequent AquireBuffer calls fail until restart):

SCF: Error InvalidState: Corr Error 8 Received for sensor 6 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
SCF: Error InvalidState: Corr Error 8 Received for sensor 5 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
[CAPTURE_TIMING] /dev/cam_boom_rear_narrow: frame_delta_ms=117.69 expected_ms=58.82
SCF: Error InvalidState: Corr Error 8 Received for sensor 7 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
SCF: Error InvalidState: Corr Error 8 Received for sensor 4 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
SCF: Error InvalidState: Corr Error 8 Received for sensor 1 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
[CAPTURE_TIMING] /dev/cam_boom_rear_narrow: frame_delta_ms=18446744073591.89 expected_ms=58.82
SCF: Error InvalidState: Corr Error 8 Received for sensor 2 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
SCF: Error InvalidState: Corr Error 8 Received for sensor 0 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
SCF: Error InvalidState: Corr Error 8 Received for sensor 0 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
SCF: Error InvalidState: Corr Error 8 Received for sensor 3 .. Continuing!
(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 654)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)

Argus fails much quicker with this type of message

(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
SCF: Error ResourceAlreadyInUse: (propagating from src/services/capture/FusaCaptureViCsiHw.cpp, function startCaptureInternal(), line 877)
SCF: Error ResourceAlreadyInUse: (propagating from src/services/capture/CaptureRecord.cpp, function doCSItoMemCapture(), line 547)
SCF: Error ResourceAlreadyInUse: (propagating from src/services/capture/CaptureRecord.cpp, function issueCapture(), line 490)
SCF: Error ResourceAlreadyInUse: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1565)
SCF: Error ResourceAlreadyInUse: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueBubbleFillCapturesIfNeeded(), line 721)
SCF: Error ResourceAlreadyInUse: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1406)
SCF: Error ResourceAlreadyInUse: (propagating from src/common/Utils.cpp, function workerThread(), line 125)
SCF: Error ResourceAlreadyInUse: Worker thread CaptureScheduler frameStart failed (in src/common/Utils.cpp, function workerThread(), line 144)
SCF: Error InvalidState: Capture Scheduler not running (in src/services/capture/CaptureServiceDevice.cpp, function addNewItemToSchedule(), line 1039)
SCF: Error InvalidState: (propagating from src/services/capture/CaptureService.cpp, function addRequest(), line 413)
SCF: Error InvalidState: (propagating from src/components/stages/MemoryToISPCaptureStage.cpp, function doHandleRequest(), line 148)
SCF: Error InvalidState: (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
SCF: Error InvalidState: Sending critical error event for Session 6
(in src/api/Session.cpp, function sendErrorEvent(), line 1039)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
SCF: Error InvalidState: Capture Scheduler not running (in src/services/capture/CaptureServiceDevice.cpp, function addNewItemToSchedule(), line 1039)
SCF: Error InvalidState: (propagating from src/services/capture/CaptureService.cpp, function addRequest(), line 413)
SCF: Error InvalidState: (propagating from src/components/stages/MemoryToISPCaptureStage.cpp, function doHandleRequest(), line 148)
SCF: Error InvalidState: (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
SCF: Error InvalidState: Sending critical error event for Session 5
(in src/api/Session.cpp, function sendErrorEvent(), line 1039)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
(Argus) Error Timeout: (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 264)
SCF: Error InvalidState: Capture Scheduler not running (in src/services/capture/CaptureServiceDevice.cpp, function addNewItemToSchedule(), line 1039)

ShaneCCC · February 23, 2026, 2:39am

There is no update from you for a period, assuming this is not an issue anymore.
Hence, we are closing this topic. If need further support, please open a new one.
Thanks ~0325

The new log tell unable get frame from the sensor.

Could you verify on two cameras with the same condition.

Thanks

Topic		Replies	Views
Argus Streaming STATUS_CANCELLED Jetson AGX Orin camera	37	2679	December 7, 2023
Argus Multi-Camera Capture Stops (9 IMX390 via UB960 on AGX Orin) Jetson AGX Orin camera	21	172	July 15, 2026
AGX Orin + IMX390 (UB960) – When one camera disconnects at runtime, Argus enters bad state and all other streams fail Jetson AGX Orin camera	11	106	February 22, 2026
Intermittent camera failure on Orin Jetson AGX Orin camera	7	1195	January 5, 2023
AGX Orin camera issues Jetson AGX Orin camera , vi	25	1704	June 17, 2024
Nvargus crashes with unreliable CSI camera connections on Jetpack 5.1.2 Jetson Orin NX camera , nvbugs	14	2315	December 14, 2023
Argus capture instability : ViCsiHw frameCo: syncpoint id xx (progress) stuck waiting Jetson Orin NX camera	12	238	October 13, 2025
Camera failed: InvalidState Status syncpoint signaled but status value not updated in:/capture/src/fusaViHandler.cpp 817 Jetson AGX Orin camera	3	1077	November 21, 2022
Argus outputs the image successfully, but waiting tens of seconds and then opening it results in no image being displayed DRIVE AGX Orin General camera	2	22	July 18, 2026
Argus timeout when capturing frames followed by unrecoverable state Jetson Orin Nano camera , nvbugs	23	489	July 20, 2026

Detecting / Gracefully Failing from Bad CSI/Camera State

Related topics