# Get tensor core usage through nvml

**URL:** <https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571>\
**Category:** System Management and Monitoring (NVML)\
**Created:** [April 22, 2020, 1:52am UTC](https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571 "2020-04-22T01:52:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![igormp](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/igormp/32/17128_2.png) [@igormp](https://forums.developer.nvidia.com/u/igormp)\
**Post date:** [April 22, 2020, 1:52am UTC](https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571/1 "2020-04-22T01:52:15Z")

</div>

Is there a way to monitor real time usage of tensor cores through some API? I couldn’t find anything on the [nvml api](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceQueries.html), with the only option being nsight, which isn’t able to do real time monitoring.

---

<div class="post-metadata">

**Author:** ![lilcore](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lilcore](https://forums.developer.nvidia.com/u/lilcore)\
**Post date:** [July 4, 2021, 7:53am UTC](https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571/3 "2021-07-04T07:53:06Z")

</div>

The CUPTI Metric API seems to have some tensor utilization functions.

[https://docs.nvidia.com/cupti/Cupti/r\_main.html#r\_host\_raw\_metrics\_api](https://docs.nvidia.com/cupti/Cupti/r_main.html#r_host_raw_metrics_api)

---

<div class="post-metadata">

**Author:** ![BrentS](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@BrentS](https://forums.developer.nvidia.com/u/BrentS)\
**Post date:** [November 5, 2021, 2:37pm UTC](https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571/4 "2021-11-05T14:37:13Z")

</div>

You can do this with Data Center GPU Manager (DCGM)

[https://docs.nvidia.com/datacenter/dcgm/latest/dcgm-user-guide/feature-overview.html#profiling](https://docs.nvidia.com/datacenter/dcgm/latest/dcgm-user-guide/feature-overview.html#profiling)

Download from here:

> **[NVIDIA DCGM](https://developer.nvidia.com/dcgm)**
>
> Manage and Monitor GPUs in Cluster Environments

.

---

<div class="post-metadata">

**Author:** ![hexexpert5](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hexexpert5](https://forums.developer.nvidia.com/u/hexexpert5)\
**Post date:** [December 15, 2022, 10:17pm UTC](https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571/5 "2022-12-15T22:17:33Z")

</div>

Profiling Tools!? That is a far lower level than basic obvious system monitoring stuff that is usually released with a product. So if nvml says my 4090 is 0% busy even though all my Tensor cores are 100% busy how is that right is any way shape or form? Shouldn’t the tools provide accurate data. How many years have Tensor cores been present?

---

<div class="post-metadata">

**Author:** ![rs277](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@rs277](https://forums.developer.nvidia.com/u/rs277)\
**Post date:** [December 17, 2022, 7:57am UTC](https://forums.developer.nvidia.com/t/get-tensor-core-usage-through-nvml/120571/6 "2022-12-17T07:57:11Z")

</div>

Documentation for Nsight Compute CLI lists a couple of tensor metrics:

sm\_\_pipe\_tensor\_op\_hmma\_cycles\_active.avg.pct\_of\_peak\_sustained\_active  
sm\_\_pipe\_tensor\_op\_imma\_cycles\_active.avg.pct\_of\_peak\_sustained\_active (SM 7.2+)