I tried to profile ResNets(50;101;152) under MPS on A100.
Nsys works, however, since I need the utilization info, I tried NCU.
Command I used is :
ncu -o profile --target-processes all --set full python ../resnet/resnet/resnet50.py
Error returned is:
==ERROR==ERR_NVGPUCTRPERM - The user does not have permission to access NVIDIA GPU Performance Counters on the target device 0.
Warning returned is:
==WARNING== No kernels were profiled.
Is this caused by the lack of permission to access NV GPU Perf Counters?
I saw this solution before, and I find it already created on the server. However, it doesn’t work for me.
NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 NCU version: 2021.2.2.0 (build 30282580) (public-release)