How to understand the specs 450image/sec using AlexNet on Titan X?

Accelerated Computing CUDA CUDA Programming and Performance
1

Hi Guys

According to the benchmarks:

Forward time of AlexNet on Titan X Maxwell is 7.55ms that means 132.45 images/sec? Then how to achieve the 450images/sec performance?

Thanks.

2

Where did the number 450 image/sec come from? Does it pertain to a Pascal-based Titan Xp by any chance? Are you using the same configuration settings, e.g. image size, as is used by the benchmark?

3

one word: batching

4

Hi Njuffa, thanks for the reply. The 450 images/sec is from GTC conference where Jen-Hsun Huang announced the NVIDIA Drive PX2. Please refer to the image I attached. The Titan X in the comparison is still Maxwell. I’m not doubting the benchmarks as I also got the same results (7.55ms that means 132.45 images/sec). I’m asking the question because our benchmarks seems have a big performance drop from what NVIDIA announced (132 images/sec VS. 450 images/sec). I’m asking what configuration am I supposed to use to achieve the performance mentioned by NVIDIA.

2560×1440 170 KB

5

Hi Txbob, thanks for the reply. I roughly get what you meant. Could you please elabrate more on the details why the batching does the tricks? Is the 450 images/sec talking about training or inference? Thanks.

6

Sorry, I believe the 450 images/sec number for Titan X is a non-batched inferencing number. So ignore my previous remark.

A similar number is also reported here:

[url]https://devblogs.nvidia.com/parallelforall/inference-next-step-gpu-accelerated-deep-learning/[/url]

And the experimental setup is here:

[url]http://developer.download.nvidia.com/embedded/jetson/TX1/docs/jetson_tx1_whitepaper.pdf?autho=1447264273_0fafa14fcc7a1f685769494ec9b0fcad&file=jetson_tx1_whitepaper.pdf[/url]

(the number reported there appears to be 405 images/s, I’m not going to try and explain the difference)

7

See here:

The 7.55ms inference is for batch of 16. So it’s actually 2119 img/s.

8

Hi Janet, thank for the reply. Yes, this seems make sense. However, 2119 image/sec is still not even close to the 450images/sec as NVIDIA stated.

9

That’s because the 450 images/sec is a non-batched number. And I already gave you the references for that 405 images/sec non-batched inference on Maxwell Titan X.

10

Hi Bob, thanks for the links. I m readin the reports and try to figure out the crux. I will update later if I found out. Thanks.

11

Yes, thanks Bob. I’m reading the reports now. Will update later.