Can't get TLT trained model get to work on Deepstream - Jetson (NX)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) DGPU % Xavier NX.Jetson Nano
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) 4.5
• TensorRT Version 72xx and 7130
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Me and my colleague are trying for days to get a TLT 3.0 trained Detectnet-v2 model working on our NX’s using Deepstream 5.1. The model runs (with a warning, see further in text), but not a single detection

This issue was posted also in the TLT category, but as I’m not sure if it’s TLT or Deepstream related I posted it here as well. @moderators, feel free to remove redundant post.

Let me break down the pipeline;
Training machine: X86 - RTX 3070 Using TLT3.0 ( tlt-streamanalytics:v3.0_dp_py3)
If I recall correctly, the tensorrt version in the dockerfile is 7.2xx.
After too many attempts, we went basic and executed the detectnet-V2 example with the kitti dataset.
None of the params were changed and we used the non-qat retrain, export and int-8 conversion.

Copied the resnet18.etlt and calibration.bin to our Xavier NX (multiple) w Jetpack45 (Tensorrt 7130 and Deepstream 5.1)
Ran the following command:

./tlt-converter /resnet18_detector_qat.etlt -k MyAPIkey -c /calibration_qat.bin -o output_cov/Sigmoid,output_bbox/BiasAdd -d 3,768,1024 -m 64 -i nchw -t int8 -e /resnet18_detector.trt -b 4 -w 1000000000

After it finished, copied the model.trt file and labelmap in our runtime folders and modified the PGIE file like this:

[property]
gie-unique-id=1
gpu-id=0
enable-dbscan=true
net-scale-factor=0.0039215697906911373
#model-file=models/primary/resnet10.caffemodel
#proto-file=models/primary/resnet10.prototxt
model-engine-file=models/primary/resnet18_detector.trt
labelfile-path=models/primary/labels.txt
#int8-calib-file=models/primary/cal_trt.bin
force-implicit-batch-dim=1
batch-size=12
network-mode=1
process-mode=1
model-color-format=0
num-detected-classes=3
interval=0
#output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
output-blob-names=output_cov/Sigmoid;output_bbox/BiasAdd
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
pre-cluster-threshold=0.2
eps=0.2
group-threshold=1

In fact, we reproduced everything as in this youtube vid:

Nvidia Transfer Learning Toolkit 3 Models Deployed to Jetson Devices

with the exeption we had to change the #output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid to output-blob-names=output_cov/Sigmoid;output_bbox/BiasAdd to get our deepstream app started, while the guy in the video didn’t change these params.

The warning at startup of our Deepstream app is:

WARNING: [TRT]: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.

Which seems normal to me when building a model on x86, but deploy it on aarch.
I’ve read smething about building a custom nvinfer parser, but the guy in the vid didn’t do that, So it’s not necessary for Detectnet_v2?

Any help would be greatly appreciated!!

2

Duplicated with Can’t get TLT trained model get to work on Deepstream - Jetson (NX) - Intelligent Video Analytics / Transfer Learning Toolkit - NVIDIA Developer Forums

3

Actually, there is some detection going on, depending of the objectsize in the sample vid. However there is a huge difference between the tlt-infer inference and the deepstream inference. The tlt-infer is spot on, even with the smallest objects. Using the default deepstream-app with eg the source30 configfile (leaving all values default except nr of streams and batchsize). Inference performance is affected by the net-scale factor, but I have yet to discover how to calculate the value. In short; why is there a huge difference between the tlt-infer and deepstream infer?

5

Found the answer. There are 2 things going on here;

  1. The TLT detectnet-v2 kitti Jupyter example works quite good with the demo vids in the stream, but the traied model isn’t robust enough to infere on lower quality vids or streams.
  2. In the default specfiles, DBSCAN is enabled. To get my own detectnet v2/resnet34 model to work, I had to change the folloring in the primary GIE configfile;

[class-attrs-all]
pre-cluster-threshold=0.2
group-threshold=1
eps=0.7
minBoxes=3

2 Likes