Nsight Compute can't export the report

I could’t export the .ncu-rep report, and I find this manual: NVIDIA Development Tools Solutions - ERR_NVGPUCTRPERM: Permission issue with Performance Counters | NVIDIA Developer
However I couldn’t finish all the order without ERROR, such as “nvidia_modeset is in use”, and so on …
So I have to use sudo ncu xxx, which solve problem for some cases but not for ROS, the error is as follows:

$ sudo ncu --export profile --target-processes all rosrun turtlesim turtlesim_node
rospack: error while loading shared libraries: librospack.so: cannot open shared object file: No such file or directory
==ERROR== The application returned an error code (2).
==WARNING== No kernels were profiled.

if I don’t use sudo, the error is as follows:

==WARNING== No kernels were profiled.

Enviroment:
OS: Ubuntu2004
CUDA: 11.4(also CUDA11.6 also not work)
Night Compute: 2021.2.2
GPU: RTX4000(also test RTX2080Ti)

Hello,

However I couldn’t finish all the order without ERROR, such as “nvidia_modeset is in use”

Yes, sometimes this can be a bit tricky to do. One simple way to do this if you have permissions for it is to just make the change persistent by running something like this (the /etc/modprobe.d directory is distribution dependent but that should work on Ubuntu):

$ echo 'options nvidia "NVreg_RestrictProfilingToAdminUsers=0"' | sudo tee /etc/modprobe.d/ncu-profiling-normal-users.conf
$ sudo update-initramfs -c -k all
$ systemctl reboot

The first command will create a file that will cause the nvidia kernel module to be loaded with an additional option allowing profiling for non-root users.

The second one causes your initrd to be regenerated which is probably needed since the nvidia module is often loaded early at boot time. Even if this is not needed, the command should be harmless.

When the machine reboots, you should be able to profile applications as your normal user. Of course, this allows all users to access performance counters on the GPU which can be a security issue as it can reveal information about what is running on the GPU.

Regarding the issues when using rosrun with sudo, it seems like the issue you are encountering is caused by sudo not passing on something in your environment which helps rosrun to find a library it needs (probably the LD_LIBRARY_PATH variable).

Does the rosrun command work without the profiler but with sudo?

$ sudo rosrun turtlesim turtlesim_node

Do you use any steps to setup the rosrun environment (maybe something in your $HOME/.bashrc)?

Here are a few things you might be able to try out to fix this:

  • Tell sudo not to create a new environment by adding the -E option so your command would look like this:

    $ sudo -E ncu --export profile --target-processes all rosrun turtlesim turtlesim_node
    

    Note that keeping a previous environment when running commands as a super user may have security implications and may be disabled by your local system administrator.

  • Launch an actual shell as the root user instead of just launching one command. If there is any setup you usually do for the rosrun command you can then do it as the root user directly.

    $ sudo -s
    #        # do any setup you need to do here and then run the profiler ...
    # ncu --export profile --target-processes all rosrun turtlesim turtlesim_node
    
1 Like

Do you mean update-initramfs?

Yes, sorry, I made a typo. I meant:

sudo update-initramfs -c -k all

I also edited my previous post to fix it. Thank you for letting me know.

Did that help you solve your problem?

1 Like

These commands work well, which solve my problem, Thank you very much!

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.