I used trt to speed up caffe model with int8 Quantization. The error occured when the program tried to serialize the trt model.
[TRT] C:\p4sw\sw\gpgpu\MachineLearning\DIT\release\5.1\builder\cudnnBuilder2.cpp (1777) - Misc Error in nvinfer1::builder::createRegionScalesFromTensorScales: -1 (Could not find scales for tensor concat_2_1 copy.)
I did more experiments on other models.I found that this error has relationship the model structure.It occurs when densenet is used.
Calibrator file:
TRT-5102-EntropyCalibration
data: 3ca59f3e
relu1: 3bf98a75
pool1_dense: 3c104387
(Unnamed Layer* 5) [Scale]_output: 3db9b7e9
relu2_1_x1: 3c342e28
relu2_1_x2: 3b54fd55
relu2_1_x3: 3b6d8c4b
(Unnamed Layer* 15) [Scale]_output: 3ce160b0
relu2_2_x1: 3bd8af28
relu2_2_x2: 3b80d876
conv2_2_x2: 3af992a6
Hello,
I can reproduce the error if using IInt8EntropyCalibrator, but no issue with IInt8EntropyCalibrator2.
The scales for tensor concat_2_1 could be generated as follows.
Calibrator2 file:
TRT-6000-EntropyCalibration2[url][/url]
data: 3f7e11c4
conv1: 40ece7e9
(Unnamed Layer* 1) [Scale]_output: 4018aab7
(Unnamed Layer* 2) [Scale]_output: 3e21382e
conv1/bn: 3e0be0ed
pool1: 3eadb4c4
(Unnamed Layer* 5) [Scale]_output: 40046324
(Unnamed Layer* 6) [Scale]_output: 3eb329a3
conv2_1/x1/bn: 3f04f4be
conv2_1/x1: 3eef9290
(Unnamed Layer* 9) [Scale]_output: 4001073b
(Unnamed Layer* 10) [Scale]_output: 3f2919e2
conv2_1/x2/bn: 3e8e9df2
conv2_1/x2: 3eadb4c4
concat_2_1: 3eadb4c4
…
IEntropyCalibratorV2[url]https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Int8/EntropyCalibrator2.html[/url] is the preferred calibrator, please have a try.