Hello.
I’ve got a DGX-A100 system at my university. I’ve got a docker container in it. I’m learning CUDA this way.
My host machine is MacOS. So far I’ve managed to remote profile a CUDA program with Nsight Systems.
And then I’m trying to profile it with Nsight Compute selecting a kernel from the Nsight Systems timeline and choosing Analyze with Nsight Compute.
And then Nsight Compute fails to profile. Here’s what it’s saying in the Progress log:
Launching: /tmp/var/target/linux-desktop-glibc_2_11_3-x64/ncu
Process launched
==PROF== Attempting to connect to ncu-ui at 127.0.0.1:50152…
==PROF== Attempting to connect to ncu-ui at 172.20.10.2:50152…
==PROF== Attempting to connect to ncu-ui at 10.211.55.2:50152…
==PROF== Attempting to connect to ncu-ui at 10.37.129.2:50152…
And none of these attempts succeed.In the end I get a window that remote profile failed.
Here’s a screenshot of the Connection dialog in which I choose Profile:
What am I doing wrong?
Thanks!
The ncu CLI on the target is not able to connect back to the ncu-ui UI process on your host system on the specified IP addresses and port ranges. You should check that your host system is reachable from the docker instance using these connection parameters.
Note that you can also use the ncu CLI directly on the target to profile your application and create a report file that can then be opened later in your host UI, or in the CLI itself. You can also use the UI’s Interactive Profile activity to interactively connect and step through your target application, selecting kernels to profile manually.
Please also let us know the exact Nsight Compute version you are using.
Note that you
- have to use the same version on the target and host
- can (and should) use the latest version, even when profiling applications with older CUDA toolkit or driver versions. I recommend to use version 2021.3, which has many improvements for remote profiling to better support SSH configurations.