Frame Drop Issue on Orin when Image Capture is enabled

Please provide the following info (tick the boxes after creating this topic):
Software Version
[*] DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[*] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
[*] DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
[*] native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
I am trying to capture png images for each frame using sample_camera and observe that there is too much frame loss in camera streaming as well as image capture. I have enabled m_frameGrab = true; before onRenderHelper() gets invoked. Attaching the logs for reference. How to minimize the frame drops? Also, can you please let me know how to configure the resolution for camera; currently it is using 8MP i.e 3848x2168 as default for streaming and image capture.

Logs
Frame_drop_error.txt

Please paste the complete application log here. If there are multiple logs, please use multiple text box

frame_drop_error.txt (171.0 KB)

Hi Team,

Any update on this issue?

Dear @PA_GN ,
My apologies for missing this topic. Is this issue still néeds support. I notice several other related topics are created.

Hi @SivaRamaKrishnaNV,

Yes, it would be helpful if you address this query. I observed that reducing the resolution to 640x480 in rig.json there was a massive reduction in frame loss (identified by debug logs NOTIF_WARN_ICP_FRAME_DROP) but want to achieve same functionality with higher resolution.
Also, can you please help to configure the encoder parameters like resolution, bit-rate, frame-rate from rig.json and hardcoded from application as well.

Dear @PA_GN ,
Please check the suggestions at https://forums.developer.nvidia.com/t/entron-camera-frame-are-getting-dropped-using-sample-camera-application/309086/6y

Hi @SivaRamaKrishnaNV,

What does below prints signify & Under what conditions these prints are observed? How can they be avoided?
SIPLNotifier: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DISCONTINUITY

Hi @SivaRamaKrishnaNV ,

Gentle Reminder.

Hi Arjav,

If you are saving images into PNG files this would definitely slow you down since SoCs and Arm architecture are not the greatest in reading from and writing to disks. If you want to log your camera sensors, you should check out camera serialization. This can serialize your raw camera images into H264/H265 format byte streams and you can decode that using ffmpeg or CUDA ffmpeg afterwards. Hope this can help you with your development.

Did you check NVIDIA DRIVE OS Linux SDK API Reference: nvsipl::NvSIPLPipelineNotifier Class Reference | NVIDIA Docs

As suggested here, did you check the disk write BW, hard coding GMSL params in code?
Thanks @extern.ray.xie for your suggestion to help the community. Yes. if write BW is bottleneck, we recommend to encode it before saving to storage.

Hi @extern.ray.xie

Understood your point. But as the support was provided in the sample app provided by Nvidia & as the sample executes on GPU so I assumed that the performance part would have been taken care of.

@SivaRamaKrishnaNV These explains the type of error code but the query still remains persist that why this messages appear; under what circumstances as I am just using the sample_object_detector_tracker application without any custom modifications.
I have configured the resolution as 640x480 in rig.json.

Yes, the BW captured is approx 600 ~ 610 MB/s.

Hi @SivaRamaKrishnaNV

Please share your inputs on this.

Dear @PA_GN ,
For NOTIF_WARN_ICP_FRAME_DISCONTINUITY
the frame drop was found in RCE traces because of capture request starvation on RCE, which means not receiving capture requests from SoC, so RCE dropped the frame. It is possible SoC is too busy not to issue the capture in time. (even before RCE, maybe in VI)