Batch Size Failure in Custom YOLOv3 INT8


I have been trying to use the examples at /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/ and was able to successfully load a custom YOLOv3 model in all 3 precision modes (FP32, FP16 & INT8) when batch-size=1. When I tried changing the deepstream_app_config_yoloV3.txt & config_infer_primary_yoloV3.txt to set batch-size=4 with 4 input streams, the deepstream app builds model_b4_fp32.engine & model_b4_fp16.engine and runs when precision is set to FP32 & FP16. But when I set the precision to INT8 I see the below error:

Building the TensorRT Engine…
0:00:07.041273045 11552 0xf81eb90 ERROR nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Assertion failed: !empty()
0:00:07.044651085 11552 0xf81eb90 ERROR nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): …/builder/intInterval.cpp (25) - Assertion Error in max: 0 (!empty())
Building engine failed!

I couldn’t find much on the web with the above error messages. I will appreciate any thoughts on what the problem could be and how it can be fixed.


I can not repro your issue, here is my environments on Jetson Xavier:
1 get yolov3 files.
wget -q --show-progress
wget -q --show-progress
2 build yolo library
export CUDA_VER=10.0
make -C nvdsinfer_custom_impl_Yolo/
3 modify deppstream config batch-size of source, streammux, pgie to 4,
4 run
deepstream-app -c deepstream_app_config_yoloV3.txt

build success log:
(102) conv-bn-leaky 256 x 76 x 76 128 x 76 x 76 61311070
(103) conv-bn-leaky 128 x 76 x 76 256 x 76 x 76 61607006
(104) conv-bn-leaky 256 x 76 x 76 128 x 76 x 76 61640286
(105) conv-bn-leaky 128 x 76 x 76 256 x 76 x 76 61936222
(106) conv-linear 256 x 76 x 76 255 x 76 x 76 62001757
(107) yolo 255 x 76 x 76 255 x 76 x 76 62001757
Output blob names :
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine…
Building complete!
0:07:08.359672621 8988 0x1b0b8950 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/model_b4_int8.engine

can you specify how can we repro your issue?

1 Like

Hello @amycao,

Thank you so much for your reply. I was able to make some headway with your comments:

  • You were right with regards to INT8 batch-size4 model for yolov3. I was able to build this successfully as well. The error I reported earlier was happening with a custom yolov3 model.
  • I verified that I was unable to build my INT8 model with custom calibration table even with batch size = 1. I think I can rule out batch size as a factor.
  • Like I said earlier, I was able to build FP32 & FP16 modes of my custom model with batch size=4.

With the above information I tried building my custom yolov3 model with INT8 precision and batch size 4, but used the default yolov3-calibration.table.trt5.1 and it worked. So it appears my custom calibration table was somehow incompatible with the yolo deepstream example.

I used a modified version of in /usr/src/tensorrt/samples/python/yolov3_onnx to produce an INT8 trt engine and a custom calibration table. I couldn’t use that int8 engine in the yolo deepstream sample and it looks like I can’t use the calibration table either. The documentation provided in says we can use our own calibration table for custom model. But I don’t know why I was seeing the above error message with a custom calibration table, the model (yolov3 config+weights) seems to be fine.

I noticed the default calibration table was generated by TensortRT 5.1 but the calibration table I created was with TensorRT 6.0. I can see the formats being different:

TensorRT 5.1 calibration table:
data: 40000889
(Unnamed Layer* 0) [Constant]_output: 4000890a
(Unnamed Layer* 1) [ElementWise]_output: 3c008912
(Unnamed Layer* 2) [Convolution]_output: 3c543871…

TensorRT 6.0 calibration table:
000_net: 3c010a14
001_convolutional_bn: 3e51ceba
001_convolutional_lrelu: 3dd30ae2
002_convolutional_bn: 3da8337a…

Can you confirm if a calibration table produced by tensorRT 6.0 is compatible with the yolo deepstream sample ?


Hi @dilip.s,
You need to use the calibration table generated by the TensorRT release which is supported by DeepStream.
You can find which TensorRT is supported by DS in DS release note.


1 Like

Hello mchi,

Thanks for the clarification. I will refer the TRT version compatibility with DeepStream in the release notes.