Enabling Cuda Backend/Target in OpenCV Increases CPU Memory Usage

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, ,
1

Hello everyone,
Currently I am developing an object detection script using OpenCV (python) and a Tiny-Yolo (Darknet) DNN on a Jetson Nano 2GB. I read about how you can re-build OpenCV with CUDA support for performance improvements, and was able to do so successfully. I was comparing the memory usage, though, after adding the necessary lines to my python script (cv.net.setpreferabletaget & cv.net.setpreferablebackend) and noticed that the CPU memory usage goes up when I enable CUDA in OpenCV, along with the GPU memory usage. I’m really just confused why the CPU memory usage would go up after enabling CUDA support for the neural network inferences, and was wondering if anyone could help me understand why I’m noticing this increase. I attached my jtop outputs before/after enabling the CUDA target/backend in OpenCV. If this memory increase is normal/expected then that’s fine; I just wanted to make sure, and am having trouble finding information on why this is happening online.

jtop before enabling CUDA
jtop before enabling CUDA1280×960 294 KB

jtop after enabling CUDA
jtop after enabling CUDA1280×960 298 KB

2

Hi @aobrien, on Jetson the CPU & GPU share the same physical memory, so the Mem bar in jtop shows the memory usage for the whole system. The GPU bar in jtop shows the GPU processor utilization (not memory). So it makes sense for the GPU utilization to go up since you are now using GPU for DNN inference in OpenCV.

3

Hi @dusty_nv, thank you for the response. Yeah I saw that the GPU usage is higher, which I agree is expected. I was mainly just surprised because it looked like in the Mem bar the amount of physical memory used by the CPU increased. I expected the amount of physical memory used by the GPU to go up, since I enabled it for DNN inference, but not the amount of physical memory used by the CPU.

4

That mem bar shows the usage of physical memory across the system, including CPU and GPU memory. Unlike discrete GPUs that you plug into your PC over PCIe, the Jetson’s integrated GPU shares physical memory with the CPU and the rest of the system.

5

Thanks for the response. I’m pretty new to CUDA in general, and did some researching after your first answer on how the shared physical memory system works. What you said in both answers makes sense now and answers my question; thanks again!