Get tensor core usage through nvml

Is there a way to monitor real time usage of tensor cores through some API? I couldn’t find anything on the nvml api, with the only option being nsight, which isn’t able to do real time monitoring.

1 Like

The CUPTI Metric API seems to have some tensor utilization functions.

https://docs.nvidia.com/cupti/Cupti/r_main.html#r_host_raw_metrics_api

You can do this with Data Center GPU Manager (DCGM)

https://docs.nvidia.com/datacenter/dcgm/latest/dcgm-user-guide/feature-overview.html#profiling

Download from here:

.