Possibility of QAT training for Jetson devices for yolov4_tiny model with pruned etlt model

[sarvatmak]

• Hardware GeForce RTX 4070 Ti
• Network Type (Detectnet_v2)
• TLT Version (format_version: 2.0, toolkit_version: 4.0.1)

This is link to my previous question for more reference

---

Very low precision while Training detectnet_v2 model using custom data in TAO

Now I have got considerable accuracy of about 0.79 mAP and my aim is to deploy this model on jetson xavier NX device in deepstream.

I have generated following files in export folder

• cal.bin -------- calibration cache file
• cal.tensorfile ---------tensorfile
• trt.engine ------- fp32 engine file
• trt.engine.fp16 ----------fp16 engine file
• trt.engine.fp8 ----------fp8 engine file
• labels.txt ----------------- name of classes
• nvinfer_config.txt ----------------- it generates incomplete deepstream related config file( not sure though in detectnet_v2 it is mentioned.)

Now i want to deploy it on jetson for that QAT training is required but i assumed for training it will be using pretrained .etlt but i just observed that it is using downloaded .hd5 model

---

pretrain_model_path: “/workspace/tao-experiments/yolo_v4_tiny/pretrained_cspdarknet_tiny/pretrained_object_detection_vcspdarknet_tiny/cspdarknet_tiny.hdf5”

---

• so is there a way i can use .etlt model instead for training model with pretrained model and train from scratch?
• Also i want to ask if i use int8 model trained with TAO (and not QAT trained model) directly on jetson then is it compulsory to use TAO converter?? also will it give same accuracy after conversion?
• finally for deployment on jetson should i train QAT trained int8 model or convert the model on jetson as i stated earlier?

[Morganh]

@sarvatmak

so is there a way i can use .etlt model instead for training model with pretrained model and train from scratch?

You can use the tlt model in the existing result folder. Set it as pretrained model and trigger new training.

@sarvatmak

Also i want to ask if i use int8 model trained with TAO (and not QAT trained model) directly on jetson then is it compulsory to use TAO converter?? also will it give same accuracy after conversion?

User can deploy non-QAT model or QAT model.
TAO-converter is an option which is mentioned in previous TAO version. It can be still used.
Currently, tao-deploy can also generate the tensorrt engine as well.

To deploy in deepstream, please use GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream
You can deploy the .etlt model and key directly. Then deepstream will help generate the engine.
https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/blob/master/configs/yolov4-tiny_tao/pgie_yolov4_tiny_tao_config.txt#L31-L32

You can also comment out above two lines if you already use tao-converter or tao-deploy to generate engine. Then, just set https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/blob/master/configs/yolov4-tiny_tao/pgie_yolov4_tiny_tao_config.txt#L29