We have been tracing what appears to be a memory leak in the kernel on the Jetson Orin Nano Developer Kit, running both the original JetPack 6.0 release and the “rev 2” release that showed up in the SDK Manager a week or so back.
The odd thing about the memory leak is that the memory is not released when our application is terminated. Our application, which is running in a container, is based on one of the Deepstream sample apps, and runs without memory leaks on x86 platforms. On the Jetson Nano, however, we see the kmalloc-128 pool from the slab allocator increase steadily in size as the application runs.
It starts at around 3000 slabs, and will remain relatively steady while nothing is running. When our application starts (processing 4 1080p 30fps RTSP feeds) we see a jump to just under 4000, and then it just grows slowly until all the memory is used after a period of several days.
Right now, 3 hours after a reboot, I see about 30,000 slabs and previously after about 21 hours running with just one feed, it was showing a little over 140,000 slabs.
The only way to recover this memory that I have found is to reboot the board. Stopping the app does not release the memory from this slab pool. Restarting docker also doesn’t release it.
I did find another post which seemed to be referring to a similar issue, but with the triton server being used:
The thread under there refers to a patch (and pre-dates the “rev 2” release showing up in the SDK manager), but it does not appear that this patch has made it into a release yet. Is there any way to get hold of that patch and see whether it addresses the issue for us?