I am brand new to this, and I was trying out the pytorch DCGan Tutorial here: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html. Based upon the task manager, it seemed to be cpu bound and hardly use any GPU at all. So I wanted to see what NVidia’s profiler says. It normally takes 5-10 minutes to run this tutorial, which I launch the python application from the terminal. So to profile it, I launched:
& “C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0\nv-nsight-cu-cli.bat” -o profile C:/ProgramData/Anaconda3/python.exe d:/DCGAN/tutorial.py
The program appears to run as normal, but when it gets to the training portion, it slows to a crawl, never getting beyond the first hundred images, and the compute exe taking almost an entire core of cpu. Is this expected behavior? I would have thought that the profiler would add 1-10% overhead, not essentially freeze the application.
& “C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0\nv-nsight-cu-cli.bat” --version
NVIDIA ® Nsight Compute Command Line Profiler
Copyright © 2012-2019 NVIDIA Corporation
Version 2019.5.0 (Build 27346997)
I installed CUDA Toolkit 10.2 for the compute program, but pytorch is using 10.1. Could this cause the issue?