Inquiry Regarding GPU Utilization Monitoring in Docker Containers on Xavier NX

[1397921243]

hello,

I am currently exploring ways to monitor GPU utilization within each Docker container running on the Jetson Xavier NX.

I have attempted to use the Nsight Systems tool, but it seems to fall short in providing detailed GPU utilization information for tasks within Docker containers(the tasks inside these containers involve PyTorch-based YOLOv5 inference).

Therefore, I am reaching out to inquire if there are alternative methods that can offer more comprehensive GPU utilization insights, especially at the granularity of each Docker container(or GPU utilization per process, similar to what the nvmlDeviceGetProcessUtilization API does).

Thanks in advance.

[AastaLLL]

Hi,

Do you want to get the GPU utilization info or profile an app?

You can run the sudo tegrastats outside of the container to get the GPU utilization.
The info is system-level data and doesn’t separate usage for each application.

Thanks.

[1397921243]

Hi,

Thanks for your reply. I appreciate your suggestion to use tegrastats. However, I’m specifically interested in obtaining GPU utilization information at the process level. Is there another method or tool that provides more granular details about GPU usage for each individual process?

Thanks again.

[AastaLLL]

Hi,

GPU scheduler picks up tasks from the pool.
So it doesn’t have application-level information.

Maybe you can check if the profiler can provide the execution duration as the info.

Thanks.