Profile command cannot be used more than once with the same agent

I want to profile the process of my CNN model inference, But I got this info:

Dynamic exception type: std::runtime_error
std::exception::what: Profile command cannot be used more than once with the same agent

even when I use a very simple code in test.py file like:

print(1+1)

And I use the command “nsys profile python test.py”, the info is still the same

If I write a simple C file ‘test.c’ below and then generate a binary ‘test.out’

#include<stdio.h>
int main(){
	printf("%s\n", "test");
	return 0;
}

use this command to profile:

nsys profile ./test.out

It has no such runtime_error and runs successfully:

Importing the qdstrm file using /opt/nvidia/nsight-systems/2019.4.2/host-linux-x64/QdstrmImporter.

Importing...

Importing [==================================================100%]
Saving report to file "/home/edge/Workspace/nsys/report1.qdrep"
Report file saved.
Please discard the qdstrm file and use the qdrep file instead.

Removed /home/edge/Workspace/nsys/report1.qdstrm as it was successfully imported.
Please use the qdrep file instead.

Is there any problems with python script? I see an example in document like:

nsys profile --trace=cuda,cudnn,cublas,osrt,nvtx --delay=60 python my_dnn_script.py

Hi IronRoyGrey,

What version of Nsight Systems are you using? You can determine the version of nsys you are using with the “nsys --version” command.

I used the latest version, “NVIDIA Nsight Systems version 2019.6.1.101-8582829”, shipped in mid-December and available at https://developer.nvidia.com/gameworksdownload#?tx=$gameworks,developer_tools. With the latest version, I could not reproduce your issue using the test.py script you provided.

The command line you provided, “nsys profile --trace=cuda,cudnn,cublas,osrt,nvtx --delay=60 python my_dnn_script.py”, looks fine. It should work.

Bob Knight

Hi, I encountered the same problem.

my config:
NVIDIA Nsight Systems version 2019.6.1.101-8582829
Driver Version: 440.33.01 CUDA Version: 10.2
(those marked as gconfig below)

In my case, IronRoyGrey’s test.c report the same problem.
And I even can not reproduce this problem in another machine with the same OS, gconfig and GPUs.

Does anyone have some advice?

Did this issue get resolved?

Upgrading to NVIDIA Nsight Systems version 2020.3.1.72-e5b8014 resolved this issue for me.