DeepStream implementation of general YoloV2 and YoloV3 to INT8 precision enginefile

Is there any way to use YoloV2 and YoloV3 darknet models using trtexec

Do you want to measure its perf?
You can build them as TRT engine, and profile their perf referring to https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps#measure-the-inference-perf

Yes I want to measure performance. But now I only need to know how to easily convert YOLOV2 and YOLOV3 to int8 tensorrt engine without any calibration table?

Hi @mchi
Also can you tell me if I use the same yolov3 calibration table on the DeepStream yoloV2 or tiny-yoloV3 application will I be able to atleast generate a INT8 engine so I can calculate the performance??

No!

Hi @mchi
how to easily convert YOLOV2 and YOLOV3 to int8 tensorrt engine without any calibration table???

I have answered this question in previous comments, can you check again?

Hi @mchi,
You mean by using Calibration table only right. But I just need to create a int8 engine without a calibration table for YoloV2 . Same could be done for other models such as Yolov3.onnx using trtexec but could not be done with YoloV2. If there is a sample calibration table for YoloV2 I can create a int8 engine for the same.

HI @GalibaSashi,
Do you have Yolov2.onnx ?
You are not referring to the YoloV2 in DeepStream, right?
Is it ok for you to build a INT8 TRT engine with a dummy INT8 calibration table? If using this TRT engine for inference, it can’t output correct result, but this engine can be used for performance measurement.

Hi @mchi ,
I donot have YoloV2.onnx.
I was refering to YoloV2 in DeepStream.
Yes It is ok to build a INT8 TRT engine with dummy INT8 calibration table as I only need a dummy calibration table to make a model work in DeepStream in INT8.
Can you share the steps.

The YoloV2 in DS is darknet file, TRT does not support darknet model file and DS does not suppport dummy INT8 calibrtion table, so there is not a solution to build INT8 TRT engine with DS YoloV2 config file.

Thanks!

Hi @mchi,
So you mean to say even if I create a dummy calibration table for TensorRT INT8 engine for a particular model unless there is an implementation in DeepStream 5.0 for that model in INT8 itself it would not work right?? Can you confirm the same??

yes

Hi @mchi
When I created a dummy INT8 calibration table for Resnet50 it worked in DeepStream. So are you sure the earlier point you mentioned is correct.

Also can you tell me a easiest ad shortest method to create calibration table.

if you can, why don’t use the method on YoloV2?
I have make this very clear.

Hi @mchi
The reason I could not do was trtexec does not process or is compatible with YoloV2.weights or YoloV2.onnx. So I cannot create engine.

that’s why I said, with DS yolov2, without INT8 calibration, we can’t build the INT8 TRT engine

ok