Batching

Hi everyone,

I would like to understand in technical terms why batching images improves the throughput of the GPU.

Thank you very much.

Hi maycondouglasd, please see the following whitepaper which includes a discussion on batching inference:

https://www.nvidia.com/content/tegra/embedded-systems/pdf/jetson_tx1_whitepaper.pdf

In short, batching more data in parallel helps keep the GPU’s pipeline full and increased occupancy/utilization.