Memory leak in camera reading using Argus

I am using the Jetson MMAPI sample 13, to receive images from a camera. The code works fine, until the end of the frame capturing progress.
When the code wants to finish the capture request, there are a few lines of code as follows:

for (uint32_t i = 0; i < streamCount; i++)
{
        ICaptureSession *iCaptureSession =
            interface_cast<ICaptureSession>(captureHolders[i].get()->getSession());
        iCaptureSession->waitForIdle();
}
/* Destroy the capture resources */
for (uint32_t i = 0; i < streamCount; i++)
{
      captureHolders[i].reset();
}
/* Wait for the rendering thread to complete */
PROPAGATE_ERROR(consumerThread.shutdown());
/* Shut down Argus */
g_cameraProvider.reset();
/* Cleanup EGL Renderer */
delete g_renderer;

This code stucks at captureHolders[i].reset(); for a very long time until it throws the timeout error. I have found if I comment the iCaptureSession->waitForIdle(); part and send some extra triggers to the camera at the final stage, this object gets released and the whole ConsumerThread object and CaptureHolder will be released.

However my main program is open and I need to make instances of these two classes multiple times (I need to access to the camera with multiple resolution change so I need to close all the argus-related objects and open again). When I do that with the changes in the code that I mentioned, I get a memory leakage. The amount of leakage is about 3 times of the image size at each iteration. What can be the reason of the leak?

I checked all the allocations and deallocations in the sample 13 and my code exactly mimic them, so I am suspecting the closing part.

Please reference to argus_camera for changing the sensor resolution process.

Thanks

The 09_argus_camera_jpeg you mean? Or jetson_multimedia_api/argus/apps/camera?
If you mean argus/apps/camera, which part of it should check?
Thank you

Yes, it’s argus/apps/camera you can run it to change sensor mode and check if any memory leak and check the code follow to compare.

Thanks

1 Like

Thanks, I will check that one as well, but the behaviour of the sample 13 is a point of question. Is it expected to have trouble closing this app (sample 13)? I don’t understand the reason behind the long waiting before the captureHolder reset.

And since the needed deallocations are done:

if (m_compositedFrame)
{
        NvBufSurf::NvDestroy(m_compositedFrame);
        m_compositedFrame = 0;
}

for (uint32_t i = 0; i < m_streams.size(); i++)
{
    if (m_dmabufs[i]) {
         NvBufSurf::NvDestroy(m_dmabufs[i]);
         m_dmabufs[i] = 0;
    }
}

Wheresles can we expect the memory leak?

@ShaneCCC @DaneLLL @kayccc Sorry to mention again, any idea about the above question? This is highly critical for our case, thank you.

Does the sensor streaming off before waitForIdle()? Suppose the sensor should streaming to have driver to consume the capture request to avoid timeout.

@ShaneCCC
Can you elaborate a bit? You mean the trigger stream? If so then yes, the trigger is continuing until all camera related objects will be released.
In the software side, the Consumer thread is also up in that time, (waiting for leftover frames I guess). If I wait a very long time, the consumer will throw the timout error and be released and the program will be closed finally, but in real case, not possible to wait this much.

Suppose the waitForIdle() is waiting all capture request in the queue if the sensor streaming early may cause the timeout.

Still didn’t get it. The sensor streaming late can be related, but how can it stream early when we are trying to finish?
I didn’t understand anything from the answers.

Does this problem only happen on multiple cameras?

No, I tried with 1 Camera and 2 Cameras, both were the same.

I didn’t see the problem with 13_argus_multi_camera sample code on JP5.x
What’s your release?

This is my environment:

Model: NVIDIA Jetson Xavier NX Developer Kit - Jetpack 5.1 [L4T 35.2.1]
Hardware:
 - Module: NVIDIA Jetson Xavier NX (16GB ram)
Platform:
 - Distribution: Ubuntu 20.04 focal
 - Release: 5.10.104
Libraries:
 - CUDA: 11.4.315
 - cuDNN: 8.6.0.166

Can you share your test code as a stress test on sample 13? The original sample doesn’t repeat the initialization and closing. It only runs it one time.

Yes, I run the original sample one time.
How do you stress test it?

I mean just running everything in a loop (initial->capture->destroy) to see the potential memory leak. But anyway even for one time run I still stuck at captureHolders[i].reset();
Is there anything out of the sample code that I can check to see everythins is working fine?

What’s the sensor?
You may check the argus-daemon log. Run below command the start the test.

journalctl -u nvargus-daemon -f

1 Like

The sensor is Sony IMX531

And here is the log of the command, while I was running Sample13:

journalctl -u nvargus-daemon -f
– Logs begin at Thu 2024-10-24 15:51:57 JST. –
Oct 29 10:44:57 s02v4dev2 nvargus-daemon[516794]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
Oct 29 10:44:57 s02v4dev2 nvargus-daemon[516794]: === argus_multi_camera[516880]: CameraProvider initialized (0xffff886e26c0)SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
Oct 29 10:44:57 s02v4dev2 nvargus-daemon[516794]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
Oct 29 10:44:57 s02v4dev2 nvargus-daemon[516794]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
Oct 29 10:44:57 s02v4dev2 nvargus-daemon[516794]: E/ libnvphs: Error: NvPHSSendThroughputHints[usecase=camera, hint=MinCPU, value=4294967295, timeout_ms=1000]: queue_or_send() failed
Oct 29 10:44:59 s02v4dev2 nvargus-daemon[516794]: CAM: serial no file already exists, skips storing againCAM: serial no file already exists, skips storing againPowerServiceCore:handleRequests: timePassed = 1398
Oct 29 10:46:03 s02v4dev2 systemd[1]: Stopping Argus daemon…
Oct 29 10:46:03 s02v4dev2 systemd[1]: nvargus-daemon.service: Succeeded.
Oct 29 10:46:03 s02v4dev2 systemd[1]: Stopped Argus daemon.
Oct 29 10:46:03 s02v4dev2 systemd[1]: Started Argus daemon.
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections…=== argus_multi_camera[517189]: Connection established (FFFFAE281900)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: NvPclHwGetModuleList: No module data found
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: NvPclHwGetModuleList: No module data found
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Oct 29 10:46:14 s02v4dev2 nvargus-daemon[517139]: NvPclHwGetModuleList: No module data found
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: CAM: serial no file already exists, skips storing again---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: CAM: serial no file already exists, skips storing again---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: (NvCamV4l2) Error ModuleNotPresent: (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: (NvOdmDevice) Error ModuleNotPresent: (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 107)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: NvPclStartPlatformDrivers: Failed to start module drivers
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: NvPclStateControllerOpen: Failed ImagerGUID 2. (error 0xA000E)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: NvPclOpen: PCL Open Failed. Error: 0xf
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 708)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: SCF: Error BadParameter: (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 453)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: SCF: Error BadParameter: (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 333)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: SCF: Error BadParameter: (propagating from src/api/CameraDriver.cpp, function getSource(), line 505)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: === argus_multi_camera[517189]: CameraProvider initialized (0xffffa86af110)SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
Oct 29 10:46:15 s02v4dev2 nvargus-daemon[517139]: E/ libnvphs: Error: NvPHSSendThroughputHints[usecase=camera, hint=MinCPU, value=4294967295, timeout_ms=1000]: queue_or_send() failed
Oct 29 10:46:17 s02v4dev2 nvargus-daemon[517139]: CAM: serial no file already exists, skips storing againCAM: serial no file already exists, skips storing againPowerServiceCore:handleRequests: timePassed = 1403

And this is the situation of the sample executive:

Could you upgrade to r35.6 to verify.

Thanks

Alright. Only MMAPI or the whole Jetpack?