profiling metrics in cuda 10.0 with nvprof

I use nvprof to profile the program in ubuntu 18.04 with CUDA 10.0.But when I run the following command in 940 Maxwell or 1070 Pascal, getting below error.

    nvprof --profile-from-start off --metrics all --log-file log.csv --csv --print-gpu-trace myprogram

    no event/metrics were profiled

This command works with GT630m and CUDA 8.0 in ubuntu 16.04.

From changelog:
* Updated the driver to restrict use of the GPU performance counters to
system administrators by default. The previous more permissive behavior
can be restored by setting NVreg_RestrictProfilingToAdminUsers=0 when
loading nvidia.ko.

Hi khorshidian.hamid

  1. Does your application control the profiling granularity using the API cuProfilerStart/cudaProfilerStart? nvprof switch “–profile-from-start off” doesn’t start the profiling until it encounters the API cuProfilerStart/cudaProfilerStart.

  2. Does removal of the option --print-gpu-trace make any difference?

  3. Does option “–profile-from-start on” result in the expected output?

If this doesn’t help, would it be possible for you to provide the reproducer?