Tensor Core Performance on NVIDIA GPUs: The Ultimate Guide

GTC 2020 S21929
Presenters: Valerie Sarge, NVIDIA
Abstract
Learn what’s needed to achieve optimal performance on NVIDIA Tensor Core GPUs, including the brand-new A100 GPU based on the NVIDIA Ampere architecture. We’ll review the fundamentals of GPU performance, explain how Tensor Core-accelerated operations work, and use this knowledge to infer how to structure and size neural network operations (layers) to achieve ideal performance. We’ll also provide a cheat sheet of Tensor Core performance guidelines. The talk aims to provide tools to understand why neural networks perform a certain way on Tensor Core GPUs and to enable changes to network architecture to further improve performance.

Watch this session
Join in the conversation below.