Nvidia Nsight Compute 2020.1 CLI, free():invalid pointer error


I am trying to run the nsight compute profiler for a pytorch model inside nvidia pytorch docker container, an instance of the nvcr.io/nvidia/pytorch:20.03-py3 image using the following command:

nv-nsight-cu-cli python app.py

And I am still getting the error:
free(): invalid pointer
==ERROR== The application returned an error code (6)
==WARNING== No kernels were profiled
==WARNING== Profiling kernels launched by child processes requires the --target-processes all option

The nvidia driver version is 440.64
CUDA is 10.2

On the https://docs.nvidia.com/nsight-compute/ReleaseNotes/index.html
it does say that the free():invalid pointer" error has been fixed.
And that the nv-nsight-cu-cli command has been replaced by ncu.
But if I use ncu command, I get ncu command not found error.
What should be done to get rid of this error.

The fact that ncu cannot be found seems to imply that you are not actually using Nsight Compute 2020.1, but still the 2019.5. You should be able to verify that by running

nv-nsight-cu-cli --version

in the same context that you run it to profile the python application.

Yes, you are right. After running
nv-nsight-cu-cli --version,
I get
NVIDIA ® Nsight Compute Command Line Profiler
Copyright © 2012-2019 NVIDIA Corporation
Version 2019.5.0 (Build 27346997)

My bad.
nvcr.io/nvidia/pytorch:20.03-py3 says it has Nsight Compute 2019.5. I confused it with Nsight systems.

So, is there a way I should get rid of this error. Is there any nvidia pytorch image that has latest Nsight compute.

I am not certain if there already is any such container. However, if that’s possible, you can setup Nsight Compute 2020.1 on your host system and map it into the container, and it should just work the same way.

Thanks a lot. I installed Nsight Compute 2020.1 and mapped it to the container.