Infinite timeout support

MarkusHess · December 15, 2023, 10:49am

Hi,

I tried to enabled infinite timeout support on Orin NX. Somehow it does not work. I still getting timeout messages and the camera crashes:

SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 1 draining session frameStart events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 532)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 2 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 532)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 0 draining session frameEnd events 3
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 1 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 2 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)
SCF: Error Timeout: Sending critical error event for Session 1 
 (in src/api/Session.cpp, function sendErrorEvent(), line 1039)
PowerServiceCore:handleRequests: timePassed = 2581
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 1 draining session frameStart events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 532)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 2 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 532)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 3 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 532)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 0 draining session frameEnd events 4
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 1 draining session frameEnd events 3
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 2 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 3 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 635)

Is this feature supported on Orin NX (l4t 35.4.1)?

JerryChang · December 18, 2023, 2:13am

hello MarkusHess,

may I know what’s real use-case?
there’s also an internal timeout value (i.e. 2500ms) for the camera software stack, may I also know how long is each SOF arrived to CSI capture channel

MarkusHess · December 18, 2023, 9:05am

We want to trigger the camera from external for example by a light barrier or a button. Wait times between two triggers are random. The camera will be directly triggered by the electrical signal. So Argus should just wait infinite for a new image from the camera.
Is it somehow possible to adjust the timeout value? I thought, enableCamInfiniteTimeout=1 would already do that!?

JerryChang · December 18, 2023, 9:20am

hello MarkusHess,

please update pre-built with the attachment, Topic276217_Dec18.zip (2.4 MB)

and… you may refer to developer guide, Device Properties.
please configure set_mode_delay_ms for settings maximum wait time for the first frame after capture start.
thanks

MarkusHess · December 19, 2023, 12:40pm

Hi JerryChang,

thanks, but it is still not working. I did not change the set_mode_delay_ms parameter in the device tree. Instead, I triggered the camera with a higher frequency after capture start to capture some images. As I understand, the parameter is only for the first frame, so I hope this should work also!? After capturing the first frames successful, I reduced the trigger frequency but I still get the timeout errors:

OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
CAM: serial no file already exists, skips storing againLSC: LSC surface is not based on full res!
---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
CAM: serial no file already exists, skips storing againLSC: LSC surface is not based on full res!
SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
CAM: serial no file already exists, skips storing againLSC: LSC surface is not based on full res!
JC: createEvent: SetModeDelayMS= 0 Line(582)
....
>>> capture some images here <<<
....
SCF: Error Timeout: Sending critical error event for Session 1 
 (in src/api/Session.cpp, function sendErrorEvent(), line 1039)
PowerServiceCore:handleRequests: timePassed = 2703
SCF: Error Timeout: Sending critical error event for Session 1 
 (in src/api/Session.cpp, function sendErrorEvent(), line 1039)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 37 draining session frameStart events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 37 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 36 draining session frameEnd events 3
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 36 draining session frameEnd events 3
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 37 draining session frameEnd events 3
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 38 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 37 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 38 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 39 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 39 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)

JerryChang · December 20, 2023, 2:38am

hello MarkusHess,

it’s actually the timeout value for each capture request.
this configuration, set_mode_delay_ms will update SOF and also EOF timeout values.

MarkusHess · January 11, 2024, 11:09am

Hi JerryChang,

I set the set_mode_delay_ms in the device tree to 300000 (5min) but unfortunately I still get the timeout after some seconds. What else could I try?

JerryChang · January 12, 2024, 3:12am

hello MarkusHess,

I’ve also include some bug fixes, please give it another try with this attachment, Topic276217_Jan12.zip (2.4 MB)

MarkusHess · January 15, 2024, 1:41pm

Hi JerryChang,

I tried the files but still get timeouts:

SCF: Error Timeout: Sending critical error event for Session 1 
 (in src/api/Session.cpp, function sendErrorEvent(), line 1039)
SCF: Error Timeout: Sending critical error event for Session 1 
 (in src/api/Session.cpp, function sendErrorEvent(), line 1039)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 30 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 30 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 29 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 29 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 30 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
PowerServiceCore:handleRequests: timePassed = 4596
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 31 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 30 draining session frameEnd events 2
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 1 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 1, capture sequence ID = 31 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error InvalidState: Sensor 0 already in same state 
 (in src/services/capture/CaptureServiceDeviceSensor.cpp, function setErrorState(), line 100)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 31 draining session frameEnd events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 646)
SCF: Error InvalidState: Timeout!! Skipping requests on sensor GUID 0, capture sequence ID = 31 draining session frameStart events 1
 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 538)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)

JerryChang · January 16, 2024, 2:13am

hello MarkusHess,

may I know what’s your test approaches, may I review your test steps.
besides, please try enable Infinite Timeout Support for double check.

MarkusHess · January 16, 2024, 8:55am

Oh sorry, my fault. I forgot to set enableCamInfiniteTimeout=1 in the last test. We will make some further tests, but it seems to work now. Thanks.

Will this patch be included in a next release?

JerryChang · January 16, 2024, 9:08am

hello MarkusHess,

thanks for status update, we’re currently on the code review progress.

MarkusHess · February 21, 2024, 1:38pm

Hi JerryChang,

i noticed, that the first two frames seem to be dropped. Is there any option for forcing Argus to also forward these frames to the application?
When using Video4Linux, we were able to solve this by enabling the low-latency-mode control. But this seems to have no effect for Argus.

MarkusHess · February 21, 2024, 2:23pm

I also noticed something else: With the patch you provided, the timeout parameter of the function Argus::IEventProvider::waitForEvents does not seem to have any effect. I would expect, that the function returns after the timeout expired, but it blocks forever until the an event occurs. So I think this is a bug, can you confirm?

JerryChang · February 22, 2024, 3:08am

hello MarkusHess,

it is correct that first two frames has dropped by Argus.
FYI, it’s Argus FIFOs; from Argus side when ISP is in use, our user space driver internally takes care of initiating extra 2 captures for sensor exposure programming when Argus and hence underneath driver receives first capture request from client.
these two internal captures were ignored at driver level and are NOT sent to Argus or client, so this way client receives the same output captures which was requested.

we do have some code update, bug fixes to address this issue.
please wait for next JP-5 public release, and moving forward to have verification. thanks

MarkusHess · February 22, 2024, 8:11am

Ok, thanks for clarification.

From the Roadmap it seems that the next JP5 release is planned for April, correct? We need to test this feature for our new product, because it has some impact of our hardware design. As always, time schedules are tight, so April would be too late. May is it somehow possible to get a pre-release of the libraries as you provided earlier? So I could test the behavior and we can start with the hardware development.

JerryChang · February 23, 2024, 2:45am

hello MarkusHess,

could you please reiterate your real use-case,

BTW,

it blocks forever until the an event occurs.

IIUC, you have random external trigger for sending camera frames, and enableCamInfiniteTimeout works.
don’t it be expected since you’ve enable infinite timeout?

JerryChang · February 23, 2024, 3:00am

BTW2, JetPack 5.1.3/ l4t-r35.5.0 is now available.
please moving forward to the latest release version if that’s possible.

MarkusHess · February 26, 2024, 9:29am

IIUC, you have random external trigger for sending camera frames, and enableCamInfiniteTimeout works.
don’t it be expected since you’ve enable infinite timeout?

The use case is correct. We have random external triggers and Argus should not crash due to internal timeouts. enableCamInfiniteTimeout=1 together with your patches helps in this case.

But timeouts that are set externally via the API should still be respected. E.g. we are reading the events from Argus using the waitForEvents function. A timeout parameter can be passed to this function but when enableCamInfiniteTimeout is set, the timeout provided to this function is somehow not respected. Instead, the function call blocks forever. This is not what the caller would expect. enableCamInfiniteTimeout should only make internal timeouts infinite and should not override user provided timeouts from the API.

We will try to update to r35.5.0 and see how it works there.

JerryChang · February 27, 2024, 1:40am

please moving forward to r35.5.0 for testing, let’s have a new discussion thread for following-up if issue persist.

Topic		Replies	Views
Infinite Timeout Support #2 Jetson Orin Nano camera	11	1882	April 3, 2024
Hardware Triggering with enableCamInfiniteTimeout=1 Jetson Xavier NX camera	13	2123	March 14, 2023
waitForEvents() timeout ignored if enabelCamInfiniteTimeout=1 Jetson Orin Nano camera , nvbugs	9	400	August 20, 2024
Nvargus enableCamInfiniteTimeout not working in TX2-NX Jetson TX2 camera	9	239	February 3, 2025
Infinite Timeout Jetson AGX Orin camera	3	307	June 28, 2024
SCF timeout error Jetson Orin NX	21	608	July 7, 2025
Infinite timeout support with GStreamer R36.4.4 Jetson Orin NX camera	9	203	October 5, 2026
Infinite Timeout Support #3 Jetson Orin Nano camera , nvbugs	11	933	June 12, 2024
SCF: Timeout! with external trigger mode in libargus — session crashes before frames arrive Jetson Orin NX camera	35	504	April 29, 2026
Nvarguscamerasrc TIMEOUT - JetPack 5.1.4 Jetson Xavier NX camera	34	1153	February 6, 2025

Infinite timeout support

Related topics