Consulting DeepStream SDK 2.0 and TensorRT

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

1、Must the model be caffe model in the DeepStream SDK 2.0?
2、How to run the tensorflow model in the DeepStream SDK 2.0? If TensorRT4.0 optimizes the tensorflow model to INT8 model, can the optimized model run in deepstream?
3、Is The computing power of the Int8 model in the Tesla P4 22T?
4、Is The computing power of the Int8 model in the GTX 1080 Ti 45.2T? The INT8 computing power is 4 times the FP32 computing power in the GTX 1080 Ti.The FP32 computing power in the GTX 1080 Ti is 11.3T,so I think the computing power of the Int8 model in the GTX 1080 Ti is equal to 4*11.3.

#2

Hi,

You can find these information in our document which is located at [DeepStream]/DeepStream_Release/docs/NVIDIA_DeepStream_SDK_on_Tesla_V2.0_User_Guide.pdf.

DeepStrem is kind of wrapper for lot of API and component, one of them is TensorRT.

Currently, TensorRT DeepStream element is only support Caffe interface.
But you can write you own TensorRT element with DeepStream plugin interface.

And since DeepStream is kind of wrapper for TensorRT, you can check its spec for the computing power directly.

Thanks.

#3

1、How to check its spec for the computing power directly？

2、How to the cal_trt4.bin in the /DeepStream_Release/samples/models/Secondary_CarColor is produced?
It is the int8-calib-file。If I use my own model, can I call this cal_trt4.bin directly or do I need to rebuild it?

Thanks.

#4

Hi,

1.
You can check it with CUDA sample directly.
For example.

/usr/local/cuda-9.0/bin/cuda-install-samples-9.0.sh .
cd NVIDIA_CUDA-9.0_Samples/0_Simple/matrixMul
make
./matrixMul

[Matrix Multiply Using CUDA] - Starting...
GPU Device 0: "NVIDIA Tegra X2" with compute capability 6.2

MatrixA(320,320), MatrixB(640,320)
Computing result using CUDA Kernel...
done
<b>Performance= 32.20 GFlop/s, Time= 4.071 msec, Size= 131072000 Ops, WorkgroupSize= 1024 threads/block</b>
Checking computed result for correctness: Result = PASS

NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

2.
Calibration file is model-dependent. Please regenerate it for a custom model.
Here is an tutorial to creat calibration table:
https://devblogs.nvidia.com/int8-inference-autonomous-vehicles-tensorrt/

Thanks.