Reading performance counters vs. disabling frequency throttling

Developer Tools Nsight Compute
1

Is “reading performance counters” related to “frequency throttling”? Is it true that “one can not read perf counters without disabling frequency throttling”? Why?

Appreciated : )

2

Nsight Compute locks the clock frequency in order to create reproducible performance results. There is an option to not lock clocks as well. I’m not aware of any issues related to frequency throttling and the ability to collect performance counters. Did you read this somewhere or have a link to where that quote came from? There may be something I’m missing or perhaps I can try to clarify what this was in reference to.

3

Thank you for response! @jmarusarz : )

I believe you are correct. Reading perf counters is not related to frequency throttling but is related to dynamic frequency which might result in overheat and reproducibility issues. For reproducibility, Nsight tools indeed should disable dynamic frequency and lock clock frequency to a constant.

Pardon my follow-up questions as below:

  1. Is my understanding above correct?
  2. What clock frequency will Nsight tools lock to? Or just using the default clock frequency after disabling dynamic frequency?
  3. Is it true that only during the profiling the clock frequency will be adjusted and locked by Nsight tools? Or even after profiling the clock frequency is still locked?

Really appreciated,

4
  1. Is my understanding above correct?

Yes. Many of the more complicated metrics take multiple passes to collect all of the counters. If the GPU ran at 1.8 GHz in the first pass and 1.4 GHz in the second pass the accuracy of the metric will be poor.

  1. What clock frequency will Nsight tools lock to? Or just using the default clock frequency after disabling dynamic frequency?

Nsight locks to the base clocks. If you want to lock to a different clock rate then the recommend method is to set Nsight Compute to not lock clocks and use nvidia-smi to lock the clock rates.

The Nsight Compute command line to control clocks is

 --clock-control arg (=base)           Control the behavior of the GPU clocks during profiling. Allowed values:
                                          base
                                          (Lock GPU clocks to base)
                                          none
                                          (Don't lock clocks)
                                          reset
                                          (Reset GPU clocks and exit)
5

Really appreciated! @Greg : )

Just a minor followup – how about question 3?

Thanks again : )

6

@jmarusarz will have to ask the Nsight Compute team. I don’t recall if Nsight Compute sets the clock only for the profiling session or if it sets it globally. A few years ago it was set globally and it was up to the user to rest to unlock the clocks.