Measuring the GPU Occupancy of Multi-stream Workloads

Originally published at: https://developer.nvidia.com/blog/measuring-the-gpu-occupancy-of-multi-stream-workloads/

NVIDIA GPUs are becoming increasingly powerful with each new generation. This increase generally comes in two forms. Each streaming multi-processor (SM), the workhorse of the GPU, can execute instructions faster and faster, and the memory system can deliver data to the SMs at an ever-increasing pace. At the same time, the number of SMs also…