Profiling Python Scripts with NSight Systems

I’m having a lot of trouble profiling Python scripts within an Anaconda environment with NSight Systems. I’m on Windows 10.

The issue is specifically with a Python script using tensorflow-gpu. My NVIDIA driver version is 441.68 (from nvidia-smi), so it should be able to support up to CUDA 10.2.

When I run the Python script using tensorflow-gpu via NVIDIA Nsight using the execution command:
I get the following error:
tensorflow.python.framework.errors_impl.InternalError: cudaGetDevice() failed. Status: CUDA driver version is insufficient for CUDA runtime version.

On the other hand, if I run the same python script using the same command in a regular command prompt, it runs successfully.

The anaconda environment I’m using has Python 3.7.x, tensorflow-gpu=1.13.x, and cudatoolkit=10.0.x. (all should be compatible with each other).

Does NVIDIA Nsight Systems somehow run it in a different context?

I’ve contacted the Windows team to try to get an answer for you.