Using int-8 and dla for tlt PeopleNet model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

• Hardware Platform Jetson
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4

Hi, so we have deployed the tlt peoplenet model on a site with 11 cameras and it is running a bit slow. We have played with lowering the resolution of the cameras, decreasing the fps to 8-10, setting the interval in pgie config. Obviously the greater the interval there is a decrease in the accuracy we observed above 2. Our next steps is to look into using int8 and dla as we are using the Xavier.

So, how do we create an int8-calibration file for the PeopleNet tlt model? Then i know you can specify enable-dla and that in config for deepstream app but how do i use dla and gpu, do i have to create multiple pgie’s and assign cameras to each?

Thanks :)

#3

Then i know you can specify enable-dla and that in config for deepstream app but how do i use dla and gpu, do i have to create multiple pgie’s and assign cameras to each?

yes, for DLA and GPU, you need have seperate TRT engines.

how do we create an int8-calibration file for the PeopleNet tlt model?

Will have my colleague to answer this.

#4

@gabe_ddi
Please trigger tlt 2.0_dp docker and start jupyter notebook. Under example/detectnet_v2, you can see below command

!tlt-int8-tensorfile detectnet_v2 -e $SPECS_DIR/detectnet_v2_train_resnet18_kitti.txt
-m 10
-o $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.tensor

and

!tlt-export detectnet_v2
-m $USER_EXPERIMENT_DIR/experiment_dir_retrain/weights/resnet18_detector_pruned.tlt
-o $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.etlt
-k $KEY
–cal_data_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.tensor
–data_type int8
–batches 10
–batch_size 4
–max_batch_size 4
–engine_file $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.trt.int8
–cal_cache_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.bin
–verbose

The calibration.bin will be generated.
Note that,

  1. For peoplenet model, the key is tlt_encoded
  2. For peoplenet model, it has three classes. Person, Bag,Face
    You may need this to generate a dummy training spec.
  3. You may need to prepare your own data. The tool tlt-int8-tensorfile will generate a calibration tensorfile according to your training data
#7

Is using the pruned PeopleNet ResNet34 .etlt possible instead of generic detectnet_v2 tlt? I need to run PeopleNet on the DLA cores (for that I need to export it using the --force-ptq flag) but it seems NGC does not have the pruned .tlt and PeopleNet was trained on some proprietary dataset.

#8

Hi abrar.shahriar,

Please help to open a new topic with more details of your issue. Thanks

1 Like