Profiling and Optimizing Deep Neural Networks with DLProf and PyProf

Originally published at: https://developer.nvidia.com/blog/profiling-and-optimizing-deep-neural-networks-with-dlprof-and-pyprof/

Software profiling is key for achieving the best performance on a system and that’s true for the data science and machine learning applications as well. In the era of GPU-accelerated deep learning, when profiling deep neural networks, it is important to understand CPU, GPU, and even memory bottlenecks, which could cause slowdowns in training or…

We have been using these profiling tools for my deep learning models frequently and were taking notes about my experiences here and there. This blog is a good summary of those experiences. Please feel free ask any questions you might have and/or share your feedback/comments with us.