PeopleNet on xavier nx using deepstream-5.0 DP

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hi guys. @Mor
I want to test Peoplenet on xavier nx and I downloaded tlt_peoplenet_pruned_v2.0.zip and In that zip file included these files :

labels.txt
resnet18_peoplenet_int8.txt
resnet18_peoplenet_int8_dla.txt
resnet18_peoplenet_pruned.etlt
resnet34_peoplenet_int8.txt
resnet34_peoplenet_int8_dla.txt
resnet34_peoplenet_pruned.etlt

Q1- I want to run INT8 mode, but doesn’t exist calibration file, Where I must to download that file?

Q2- How I can to see the DLA is active after running inference? and which DLA?

Q3- When I run the model without calibration file, It’s correctly work, and I get such logs:

]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files
INFO: [TRT]:
INFO: [TRT]: --------------- Layers running on DLA:
INFO: [TRT]:
INFO: [TRT]: --------------- Layers running on GPU:

That’s mean these passed some layers to run on DLA? I don’t have calibration file.

3
  1. For int8 mode, it is necessary to run with calibration file.
  2. You can see in the log there are some layers running on DLA.
  3. It is running on GPU. If you want to let DLA run inference, need to set below in the DS config file.
    enable-dla = 1
    use-dla-core = 1
4

Thanks so much. @Morganh

1- I usually see the calibration file is in the .bin file, but for peoplenet are .txt file, these are calib file?
2- The log shows some layers run on DLA and some runs in GPU? why all layers doesn’t run on DLA? In my opinion, the default;t set of config file for DLA is enable, right? because in the config file Isn’t there enable-dla = 1
use-dla-core = 1 for setting.

3- This model use both DLAs or single DLA? How I can to set the specific DLA for running?

4- These params of DLA Where I have to add in the DS config file:
in the property?

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
tlt-model-key=tlt_encode
tlt-encoded-model=…/…/models/tlt_pretrained_models/peoplenet/resnet34_peoplenet_pruned.etlt
labelfile-path=labels_peoplenet.txt
model-engine-file=…/…/models/tlt_pretrained_models/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
input-dims=3;544;960;0
uff-input-blob-name=input_1
batch-size=1
process-mode=1
model-color-format=0

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
interval=0
num-detected-classes=3

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3 = None(No clustering)

cluster-mode=1
interval=0
gie-unique-id=1
output-blob-names=output_bbox/BiasAdd;output_cov/Sigmoid

5
  1. Yes, they are cal files. This is mentioned in https://ngc.nvidia.com/catalog/models/nvidia:tlt_peoplenet

Model versions:

  • unpruned_v1.0 - ResNet34 based pre-trained model. Intended for training
  • pruned_v1.0 - ResNet34 floating point deployment model.
  • unpruned_v2.0 - ResNet18 and ResNet34 based pre-trained model. Intended for training using Quantization aware training (QAT)
  • pruned_v2.0 - ResNet18 and ResNet34 deployment models. Contains calibration cache for GPU and DLA. DLA one is required if running inference on Jetson AGX Xavier or Xavier NX DLA.
  1. Because some layers cannot run on DLA and then fall back to GPU.
  2. use single DLA because “use-dla-core=1” is set.
  3. Yes.
6

Thanks so much, @Morganh
1- use-dla-core=0 is fo DLA0 and use-dla-core=1 for DLA1?
2- Is possible to use both DLA for one inference? If so How I can to set?
3- How I can to know what’s other options there for [property] or others?

7

1- use-dla-core=0 is fo DLA0 and use-dla-core=1 for DLA1 ? ==> yes
2- Is possible to use both DLA for one inference? If so How I can to set? ==> DS now supports to put two models on two DLAs respectively.
3- How I can to know what’s other options there for [property] or others? ==> check the DS nvinfer doc

8

@mchi, Thanks so much,
Is it possible to see source code of each element in the gstream pipeline of DS? I want to know to see the source code of nvinfer element, or tracker element, If yes, Where I can to find the source codes of elements?

9

not all element’s source are open source.
you can find source code under source folder, which includes the source code of nvinfer.
but traceker source is not opened.