How to use models for object detection in python

[thomas.gigout]

• Hardware: Tesla V100
• Network Type: Bodypose Net trained on COCO dataset
• TAO Version: v3.21.08-py3

Hi,
I finally managed to finish the notebook provided on bpnet and I have several models files (unpruned has .tlt .ckzip and .pbtxt, pruned has only a .tlt, retrain (from pruned if I understood well) has also the same as unpruned, and the final has fp16.engine fp32.engine int8.deploy.engine .etlt .deploy.etlt .bin and .tensorfile).

My goal is to do the inference on a video for each steps (unpruned, pruned, int8, fp16, fp32) to compare the times.

So when I have these files, what is the next step? Can I use them with only TensorRT in Python? Because I searched in the python samples on the nvidia Github but it is focused on how to convert a onnx model to tensorrt model, not on how to use it on a video or something.

Also, are the .tlt files ready to use with TensorRT or do I have to export them before in .etlt (tao export) and then to .engine (tao converter) ?

Because the first time this command is done is in step 9.4 of bpnet notebook. A parameter -p is set, but the example is for FP32 FP16 and INT8. If I want to do this command on .etlt from unpruned model, can I use the same parameters as below? Or does it need to change?

!tao converter $USER_EXPERIMENT_DIR/models/exp_m1_final/bpnet_model.etlt \
                               -k $KEY \
                               -t fp16 \
                               -e $USER_EXPERIMENT_DIR/models/exp_m1_final/bpnet.model.$IN_HEIGHT.$IN_WIDTH.fp16.engine \ 
                               -p ${INPUT_NAME},1x$INPUT_SHAPE,${OPT_BATCH_SIZE}x$INPUT_SHAPE,${MAX_BATCH_SIZE}x$INPUT_SHAPE

(??? → Don’t know what to replace it with / bold → Does it need to change ?)

!tao converter $USER_EXPERIMENT_DIR/models/exp_m1_unpruned/bpnet_model.etlt \
                               -k $KEY \
                               -t ????? \
                               -e $USER_EXPERIMENT_DIR/models/exp_m1_unpruned/bpnet.model.?????.engine \ 
                               -p ?????

[Morganh]

To generate tensorrt engine, there are two ways.

1. Using “export”. See Body Pose Estimation — TAO Toolkit 3.22.05 documentation
   https://developer.nvidia.com/blog/training-optimizing-2d-pose-estimation-model-with-tao-toolkit-part-2/

2. Using tao-converter. Refer to Body Pose Estimation — TAO Toolkit 3.22.05 documentation . Or refer to TLT CV Inference Pipeline Quick Start Scripts — Transfer Learning Toolkit 3.0 documentation Leverage the code inside tlt_cv_compile.sh from TLT CV Inference Pipeline to convert .etlt to TensorRT Engine .

To run inference, you can also refer to deepstream_tao_apps/apps/tao_others/deepstream-bodypose2d-app at release/tao3.0 · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub