Migrating INT8 calibration from TensorRT 6 to TensorRT 7 in YoloV3 and YoloV4 failed

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I’m migrating my YoloV3 and YoloV4 code from TensorRT 6 to TensorRT 7 and getting some errors on INT8 calibration.

Both YoloV3 and YoloV4 can infer with FP32 correctly but I infer YoloV3 with INT8 will get the warning like the image below and get the wrong output.

image
image962×274 76.9 KB

(yolo-det is a cutom layer)

When I inferring YoloV4 on with INT8 will get the ERROR and crash like the following image.

image
image1717×212 139 KB

I’m using IInt8EntropyCalibrator. Is there any update of INT8 calibration?

Environment

TensorRT Version : 7.1.3
GPU Type : Jetson TX2 iGPU
Nvidia Driver Version :
CUDA Version : 10.2
CUDNN Version : 8
Operating System + Version : Ubuntu 18.04

2

Hi @jackgao0323,
Can you please share the verbose logs along with the model and script.

Thanks!

3

Hi @AakankshaS,

This is the verbose of the YoloV4 mode run on INT8.

image
image1734×1826 1.23 MB

This is the last part of verbose of the YoloV3 mode run on INT8.

image
image3104×1651 2.27 MB

image
image3195×1657 3.34 MB

image
image3195×1655 3.28 MB

image
image3042×1834 3.31 MB

Will it be better if I save the logs as a file? Sorry I can’t provide our model and script.

4

I have solved the problem of running YoloV3 model in INT8 mode. The reason is I using the wrong size to do the calibration, but I still got the same problem on my YoloV4 model. Any update of this question?

5

Sorry, I’m using Jetson NX not Jetson TX2.

6

I’ve shared my implementation of INT8 TensorRT yolov3/yolov4 models here: Demo #6: Using INT8 and DLA core.

I don’t see the same problem as what you’ve described. But I do see some significant accuracy (mAP) drop on the INT8 yolov4 engine (comparing to the FP16 engine).

8

Actually, my model is not the same as YoloV4. My model is more similar to Implement Yolo-LSTM (~+4-9 AP) for detection on Video with high mAP and without blinking issues · Issue #3114 · AlexeyAB/darknet · GitHub yolo_v3_tiny_pan3.cfg model (https://github.com/AlexeyAB/darknet/files/3580764/yolo_v3_tiny_pan3_aa_ae_mixup_scale_giou.cfg.txt).
I’m trying another workflow from Darknet->Pytorch->ONNX->TensorRT.
But I still get some problem when I use trtexec to convert the model from ONNX to TensorRT.
I got the error message like below.

ERROR: builtin_op_importers.cpp:2179 In function importPad:
[8] Assertion failed: inputs.at(1).is_weights()

Have you met this error before?

Thanks!

9

@jackgao0323 No, I don’t see such a problem when building TensorRT engines for the more regular yolov3/yolov4 models.

I wonder whether the problem is related to the use of “swish” activation. “Swish” is not directly supported by TensorRT, so you’ll likely need to implement it with a plugin.

10

@jkjung13 Thanks for your reply. The problem is not swish activation because I replace it with x * sigmoid(x). I think the problem is the maxpooling. Darknet have a “special” maxpooling padding, so I use a padding before the maxpooling.

    x = torch.nn.functional.max_pool2d(torch.nn.functional.pad(x, (padding3, padding4, padding1, padding2), mode='replicate'),self.size, stride=self.stride)

I am wondering whether it have some limitations of the padding or not.

11

Hi @jackgao0323,
Apologies for delayed response,
Can you pls share your onnx model?

Thanks!