syncSensor crashed on full res(3840x2160)

Hi Nvidia,

I found problem.
syncSensor from samples crashed when resolution is 3840x2160 after 3-5 seconds. RAM is over too.
Cameras are 2x LI imx274.

argus_daemon errorlog:

Growing thread pool to 1 threads
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 18, SyncPointValue = 0)
SCF: Error IoctlFailed: (propagating from src/services/capture/CaptureServiceDeviceIsp.cpp, function startCapture(), line 461)
SCF: Error IoctlFailed: (propagating from src/services/capture/CaptureServiceCore.cpp, function doMEMtoISPCapture(), line 538)
SCF: Error IoctlFailed: (propagating from src/services/capture/CaptureServiceCore.cpp, function issueCapture(), line 346)
SCF: Error IoctlFailed: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1087)
SCF: Error IoctlFailed: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 913)
SCF: Error IoctlFailed: (propagating from src/common/Utils.cpp, function workerThread(), line 183)
SCF: Error IoctlFailed: Worker function failed (in src/common/Utils.cpp, function workerThread(), line 199)
SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc

On 1920x1080 syncSensor works fine.
2x argus_camera works fine at full res in the same time.

Best regards, Viktor.

@vsw
Did you check any time out message form the kernel message?

Here is dmesg last messages:

[ 100.797696] imx274 7-001a: imx274_set_frame_rate: val: 50331648
[ 100.797706] imx274 7-001a: imx274_set_frame_length length = 23076
[ 100.798357] imx274 7-001a: imx274_set_frame_length: val: 23076
[ 100.800199] imx274 7-001a: imx274_set_gain - val = 92274688
[ 100.800207] imx274 7-001a: input gain value: 5632
[ 100.800213] imx274 7-001a: imx274_set_gain: gain 1600 val: 07a3
[ 100.801509] imx274 7-001a: imx274_set_exposure: val: 349525
[ 100.801516] imx274 7-001a: imx274_set_coarse_time
[ 100.801521] imx274 7-001a: imx274_set_coarse_time: input val: 2080
[ 100.801526] imx274 7-001a: imx274_set_coarse_time: set val: 20996
[ 100.816559] imx274 8-001a: imx274_set_frame_rate: val: 50331648
[ 100.816569] imx274 8-001a: imx274_set_frame_length length = 23076
[ 100.817257] imx274 8-001a: imx274_set_frame_length: val: 23076
[ 100.819126] imx274 8-001a: imx274_set_gain - val = 92274688
[ 100.819133] imx274 8-001a: input gain value: 5632
[ 100.819140] imx274 8-001a: imx274_set_gain: gain 1600 val: 07a3
[ 100.820375] imx274 8-001a: imx274_set_exposure: val: 349525
[ 100.820381] imx274 8-001a: imx274_set_coarse_time
[ 100.820388] imx274 8-001a: imx274_set_coarse_time: input val: 2080
[ 100.820394] imx274 8-001a: imx274_set_coarse_time: set val: 20996
[ 104.099379] isp isp.0: nvhost_ioctl_channel_submit: failed with err -12

[ 104.107911] isp isp.0: nvhost_ioctl_channel_submit: failed with err -22

Looks like the same error:
https://devtalk.nvidia.com/default/topic/1018971/memory-leaks-reading-images-from-native-buffer/

Did you modify the sample code? I can’t repo the problem with two sensors with resolution 5632x3168

Yes. I changed capture time to 15s and stream size to 4k.

From argus_camera MultiSession 3x cameras on 3840x2160@60 are working, but from syncSensor(2x cameras) are not.
But i think capture to gl buffer from sensors is not the same as in syncSensor EGLfarme capturing:

ScopedCudaEGLStreamFrameAcquire left(m_cuStreamLeft);
        ScopedCudaEGLStreamFrameAcquire right(m_cuStreamRight);

I wrote to LeopardImaging support too.
Summary:
syncSensor with 1920x1080 and LI imx274 - OK
syncSensor with 3840x2160(any fps) and LI imx274 - Crashed after few seconds due to a memory leaks up to 100%.
It is Nvidia libargus bug or camera/driver provider bug?

@vsw
Could you help to check if any memory leak with 1080p. Make the APP run in a while loop to check if the memory is increased by the time.

Yes, of course. But it is already done.
2x constructor ScopedCudaEGLStreamFrameAcquire left(m_cuStreamLeft); execute in while loop. After 50-100 calls the Tx1 RAM ends.
If i remove frame accuire constructors then all is ok. And it is only at 4k.
I am right?
I wrote to LI support too.

@vsw
I am not understand “If i remove frame accuire constructors then all is ok. And it is only at 4k.”
Could you have more explanation and post what you modify here.

I mean that if the code with frame acquiring is commented then syncSensor works;

while (true)
    {
<b>continue;</b>
        ScopedCudaEGLStreamFrameAcquire left(m_cuStreamLeft);
        ScopedCudaEGLStreamFrameAcquire right(m_cuStreamRight);

        if (!left.hasValidFrame() || !right.hasValidFrame())
            break;

        // Calculate histograms.
        float time = 0.0f;
        if (left.generateHistogram(histogramLeft, &time) &&
            right.generateHistogram(histogramRight, &time))
        {
            // Calculate KL distance.
            float distance = 0.0f;
            Size2D<uint32_t> size = right.getSize();
            float dTime = computeKLDistance(histogramRight,
                                            histogramLeft,
                                            HISTOGRAM_BINS,
                                            size.width() * size.height(),
                                            &distance);
            CONSUMER_PRINT("KL distance of %6.3f with %5.2f ms computing histograms and "
                           "%5.2f ms spent computing distance\n",
                           distance, time, dTime);
        }
    }