I am looking for a performance counter monitor that will monitor at runtime hardware metrics of an NVIDIA GPU. I need runtime monitoring (during the GPU application) and not monitoring after the end of the application like what nvidia visual profiler or nvprof do.
I have used nvidia-smi that monitors its metrics at runtime, but I need the metric of GPU utilization that refers to spatial percentage of total SMs/cores are used and not to time percentage (like what nvidia-smi refers to).
Is there any tool, that monitors spatial GPU utilization and other hardware metrics like cache misses during the CUDA application?
Any ideas? Thank you