Report file is corrupted

Development Tools Nsight Systems Profiling Linux Targets
1

I am profiling a Python application on an Ubuntu server using the command:

nsys profile  \
    --output ${OUTPUT_FILE} \
    --trace-fork-before-exec true \
    --trace cuda,nvtx,cublas,cudnn,opengl \
    --delay 120 \
    --duration 360 \
    --kill sigkill \
    ${PYTHON_EXECUTABLE} ${PYTHON_SCRIPT}

The python application doesn’t stop after 360 minutes (isn’t it the purpose of the duration option?) so I kill it with a kill -9 pid. After that nsys seems to run for a few minutes, and finally it generates a report.qdrep file.
I download it to my Mac where I try to open the file.

Error details:

Failed to load report from: /Users/matteoreport.qdrep
File is corrupted.

Details:
/Users/devtools/buildAgent/work/20a3cfcd1c25021d/QuadD/Common/StreamSections/StreamWithSections.cpp(374): Throw in function int64_t QuadDCommon::StreamSectionsManager::readSectionTableOffset()
Dynamic exception type: boost::exception_detail::clone_impl<QuadDCommon::CorruptedDataException>
std::exception::what: CorruptedDataException
[QuadDCommon::tag_message*] = Section Table Reference magic number mismatch.
[QuadDAnalysis::tag_report_file_name*] = "/Users/matteo/Documents/Fusus/Coding/fususcore-ai-detector-nvidia/nsys/local/report.qdrep"
[boost::errinfo_api_function_*] = parseSectionTable()

I tried multiple times but I always get a corrupted file. I also checked and I am running the latest Nvidia Nsight System version.

2

The arguments of delay and duration are in seconds, you may also check nsys profile --help.
Also, the duration time will be added to the delay time, i.e., the above command should be expected to run for 480 seconds. Is the application still running after that period?

Which versions of nsys are you using to create the report and to view it? 2021.4.1.73-08591f7 on both devices?

3

Good catch. Running sudo find / -name nsys showed me that on Ubuntu the default nsys was the one that came with cuda 11.1 and not the latest I installed.

1 Like