- 8 gpus machine with 2080ti
- cuda 11.4
- using the docker image
dlprof --reports=detail --mode=tensorflow2 <COMMAND>
Model trains fine, limited the number of steps such that it trains for a couple of minutes only.
Creating final output files... Processing [===============================================================100%] Saved report file to "/tmp/nsys-report-e176-9bcd-0e6a-6a6a.qdrep" Exporting 26015306 events: [===============================================100%] Exported successfully to /tmp/nsys-report-e176-9bcd-0e6a-6a6a.sqlite Report file moved to "/app/./nsys_profile.qdrep" Report file moved to "/app/./nsys_profile.sqlite" [DLProf-17:26:01] DLprof completed system call successfully [DLProf-17:26:04] Initializing Nsight Systems database [DLProf-17:28:34] Reading System Information from Nsight Systems database [DLProf-17:28:34] Reading Domains from Nsight Systems database [DLProf-17:28:34] Error Occurred: [DLProf-17:28:34] Nsight Systems did not detect any NVTX traces. Please check your script and try again.
nsys_profile.sqlite are created correctly.