Hello, I am trying to run the Nsight Systems profiler on some code I have inside a Docker container. I am not sure what’s the correct way to do that.
I tried to run the profiler on the nano inside the container but the nano just freezes, is the nano supposed to be able to run it? or is it a problem in my code?
I also saw someone suggesting to run the profiler on a remote host and run the code on the nano remotely, but how do I do that if the code is inside a container?
If anyone can help me I would really appreciate it
There is also documentation on running in docker at User Guide :: Nsight Systems Documentation (that’s a direct link to the section on running in Docker, this interface just munges the name poorly).
Hey, thanks a lot for the links. I tried reading through them, however, I didn’t find a way to connect Nsight from a remote desktop to a container that is running on the Nano (that might be because I didn’t fully understand everything, I’m pretty new to using Docker and Nsight). Is that possible? I’m asking this because when I run Nsight on the Nano inside the container the Nano freezes, or takes a few hours to finish.
Hello,
I need some help with running my project using Docker. I’m trying to use the nsys profiler, but I’m having trouble combining the two commands into one. Here’s what I’ve tried so far : nsys profile
-w true
-t cuda,nvtx,osrt,cudnn,cublas
-s cpu
–capture-range=cudaProfilerApi
–capture-range-end=stop-shutdown
–cudabacktrace=true
-x true
-o my_profile
docker run --shm-size 2g --gpus all --rm …
When you do docker run ..., you are using the docker CLI to request that the Docker engine start a container on your behalf. Your command, as you currently have it structured, would be profiling the docker call itself, not the process in the container.
You need to have nsys available inside the container, and then run your container like:
docker run <docker options> \
<image>:<tag> \
nsys profile <nsys optons> \
<your application and options>
This also means that the profiling report will be generated inside the container as well, so you’ll want to map a volume into docker for the output, and then use the nsys '-o` option to make sure you are writing to that mapped volume.
Thank you for your response. I tried this earlier, but it didn’t work. I can’t even launch the container with -it because it launches the simulation and finishes immediately. The problem is that my Docker command is personalized to launch simulations, so I have to enter the data repository name “simulation_test” and the driver I used “local,” along with the Docker image name “compute.” I have tried several times to run NVTX inside the container, but it doesn’t work. That’s why I want to know if there’s a solution to combine these commands. I’ll give you the complete commands I use so that it might be clearer for you:
I use docker build to build my Docker image.
Then, I run docker run --shm-size 2g --gpus all --rm -v $(pwd)/data/simulation_test:/app/data/simulation_test my_compute simulation_test local.
Additionally, I have the NVTX command I wrote before with all the options I want.
To respond to your question, I’m not using Tegra. I just want to profile my PyTorch model with NSYS, but the Docker specificity is blocking me.
With respect to your comment about -it, You could try supplying --entrypoint=/bin/bash to your docker arguments to override an ENTRYPOINT directive in your Dockerfile, and do not supply the program arguments simulation_test local. That should allow you to be able to get into a shell in the container, and you can then attempt to run nsys manually.
Again, you’ll need to have nsys available within the container’s filesystem.