Not able to deploy int8 engine file

[clara.goncalves]

When attempting to deploy the resnet18_detector.trt.int8 model using DetectNet_v2 on an EC2 instance with thw following command:

!sed -i “s|/workspace/tao-experiments/data/training|/workspace/tao-experiments/data/training/image_2|g”

$LOCAL_SPECS_DIR/detectnet_v2_retrain_resnet18_kitti.txt
!tao deploy detectnet_v2 gen_trt_engine
-m $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.onnx
–data_type int8
–batches 10
–batch_size 4
–max_batch_size 64
–engine_file $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.trt.int8
–cal_cache_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.bin
-e $SPECS_DIR/detectnet_v2_retrain_resnet18_kitti.txt
–results_dir $USER_EXPERIMENT_DIR/experiment_dir_final
–verbose

!sed -i “s|/workspace/tao-experiments/data/training/image_2|/workspace/tao-experiments/data/training|g” $LOCAL_SPECS_DIR/detectnet_v2_retrain_resnet18_kitti.txt

I encountered the following issue:
ouput_log.txt (1.2 MB)

[Morganh]

@clara.goncalves

–max_batch_size 64

Please try to set lower max_batch_size. For example, --max_batch_size 4.
I am afraid the error is due to out-of-memory.

[clara.goncalves]

After changing the max_batch size I got the same error.
output_log.txt (1.5 MB)

[Morganh]

The error as Error while converting .etlt model to .trt model - #9 by Morganh.
Please try to generate tensorrt engine against the unpruned onnx file.

[clara.goncalves]

I don’t have an unpruned onnx file. The unpruded weights output is in hdf5.

[Morganh]

You can export the unpruned hdf5 file to onnx file as well.