TAO Toolkit - unable to create Int8 engine

[ColinPs26kt]

• Hardware : RTX2070
• Network Type: yolo_v4
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here): tao-toolkit-tf:v3.21.11-tf1.15.5-py3 docker
• How to reproduce the issue ? Please see below

Hi,

I’m running into this error.

[ERROR] 1: Unexpected exception _Map_base::at
[ERROR] Unable to create engine

I use the command below:

tao converter -k tlt_encode /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080.etlt -e /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.tao -c /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.calib -t int8 -d 3,416,416 -p Input,1x3x416x416,16x3x416x416,16x3x416x416 -o BatchedNMS -s -w 1000000000 -b 16 -m 16

I refered to the answer in the ticket (Tao-converter export int8 engine core dump on Xavier NX jetpack 4.6 - #9 by olivier.brousse1) but can’t seem to solve my issue.

I’m able to convert fp32 and fp16 engines just fine.

Thanks.

[Morganh]

@ColinPs26kt

/workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.calib

Could you share the /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.calib?

[ColinPs26kt]

Hi @Morganh,

I think I may have misunderstood the steps. I thought the calibration file was to be generated in the export step.

So now I’m doing this. I define the int8 right at the start when exporting to etlt. The calibration file “yolov4_cspdarknet53_epoch_080_int8.calib” is created in this step (in link below).

tao yolo_v4 export -m /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080.tlt -o /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.etlt -k tlt_encode -e /workspace/tao-experiments/specs/combined_training_config_yolov4_fromchkpt.txt --gen_ds_config --cal_image_dir /workspace/tao-experiments/data/calibration --cal_data_file /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.calib --data_type int8

Then I call the convert.

tao converter -k tlt_encode /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.etlt -e /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.tao -t int8 -d 3,416,416 -p Input,1x3x416x416,16x3x416x416,16x3x416x416 -o BatchedNMS -c /workspace/tao-experiments/data/yolov4_detector/weights/yolov4_cspdarknet53_epoch_080_int8.calib -b 16 -m 16 -s -w 1000000000

The error message is the same.

Here’s a link to the calib file generated at the export step.

TaoDebug

[Morganh]

See YOLOv4 — TAO Toolkit 3.21.11 documentation
When you run "tao yolo_v4 export ", thereis an option --cal_cache_file , it will generate /export/cal.bin . This cal.bin is the correct file for generating engine when you run “tao converter”.