I am using Xavier NX. And I am testing 2 JP version 5.1.1, 5.1.2 I had qat onnx model. I converted this qat onnx model to qat engine for JP5.1.1 and 5.1.2. I run inference and evaluate 2 qat engine with Python API, mAP of 2 qat engine was same. But when I run inference and evaluate engine with Deepstream (DS6.2, TRT8.5.2 in both JP version, same source code), there was big mAP (~10%) between 2 engine models. mAP with JP5.1.1 is good, but mAP with JP5.1.2 is bad. From my investigation, the issue is inference part with Deepstream (DS6.2, TRT8.5.2). I also checked release note of JP5.1.2 and did not find any strange points. Sorry, I can not share my models.

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks can you try network-mode=0? maybe in8 calibration file is bonding to specific low-level. you can use this method to dump the Inference Input, then you will know if the inference inputs are the same.

please refer to this link Jetson model Platform and OS Compatibility. DeepStream version needs to be consistent with Jetpack’s version.

@fanzh Thanks for quick response. What does GA means in 5.1 GA? I dont see DS version for JP5.1.1 in the above link. In this link, DS6.2 is compatible with JP5.1.2 JetPack SDK 5.1.2 | NVIDIA Developer, but in the above link DS6.3 is compatible with JP5.1.2. It makes me confused.

5.1 GA is one version of 5.1. please install the corresponding version of DS. DS6.2 is compatible with 5.1 GA.

@fanzh I checked DS6.3 with JP5.1.2. mAP is bad. I am not clear the root cause here.

do you mean the same code will get different results on DS6.2+JP5.1.1 and DS6.2+JP5.1.2? from the doc 511 and 512, the TensorRT version is the same.

Yes. DS6.2+JP5.1.1 gives good mAP. DS6.2+JP5.1.2 and DS6.3+JP5.1.2 give bad mAP for COCO dataset. The mAP gap is about 10%. For other dataset, the big mAP gap is not.

what is the whole pipeline? how did you use DeepStream? it is the custom code? can you use deepstream sample to reproduce this issue?

I used Deepstream-app https://github.com/NVIDIA-AI-IOT/yolo_deepstream/tree/main/deepstream_yolo Model is yolov7. Could you check it again please?. Sorry I can not share my model.

did you modify the code? can the issue be reproduced without code and model modification?

It is almost same. Config for yolov7 int8 engine. I run on Jetson Xavier board. [property] gpu-id=0 net-scale-factor=0.0039215697906911373 #0=RGB, 1=BGR model-color-format=0 onnx-file=yolov7.onnx model-engine-file=yolov7.onnx_b1_gpu0_int8.engine labelfile-path=labels.txt ## 0=FP32, 1=INT8, 2=FP16 mode network-mode=1 num-detected-classes=80 gie-unique-id=1 network-type=0 is-classifier=0 ## 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering) cluster-mode=2 maintain-aspect-ratio=0 symmetric-padding=1 ## Bilinear Interpolation scaling-filter=1 #parse-bbox-func-name=NvDsInferParseCustomYoloV7 parse-bbox-func-name=NvDsInferParseCustomYoloV7_cuda #disable-output-host-copy=0 disable-output-host-copy=1 custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so #scaling-compute-hw=0 ## start from DS6.2 #crop-objects-to-roi-boundary=1 [class-attrs-all] #nms-iou-threshold=0.3 #threshold=0.7 nms-iou-threshold=0.65 pre-cluster-threshold=0.001 topk=300 When running inference with JP5.1.2 + DS6.2 or JP5.1.2 + DS6.3, it outputs addional info compared JP5.1.1 + DS6.2 NvMMLiteBlockCreate : Block : BlockType = 256 NvMMLiteBlockCreate : Block : BlockType = 256 NvMMLiteBlockCreate : Block : BlockType = 256 NvMMLiteBlockCreate : Block : BlockType = 256 [JPEG Decode] BeginSequence Display WidthxHeight 640x480 NvMMLiteBlockCreate : Block : BlockType = 256 [JPEG Decode] BeginSequence Display WidthxHeight 640x425 NvMMLiteBlockCreate : Block : BlockType = 256 NvMMLiteBlockCreate : Block : BlockType = 256 NvMMLiteBlockCreate : Block : BlockType = 256 [JPEG Decode] BeginSequence Display WidthxHeight 427x640 [JPEG Decode] BeginSequence Display WidthxHeight 500x332 [JPEG Decode] BeginSequence Display WidthxHeight 640x480 [JPEG Decode] BeginSequence Display WidthxHeight 480x640 [JPEG Decode] BeginSequence Display WidthxHeight 640x427 [JPEG Decode] BeginSequence Display WidthxHeight 480x640 [JPEG Decode] BeginSequence Display WidthxHeight 640x425 NvMMLiteBlockCreate : Block : BlockType = 256 [JPEG Decode] BeginSequence Display WidthxHeight 640x424 I used COCO images with YUV420 format to be supported in DS. Do you think image decoding can be the root cause?

mAP of engine model in Deepstream with JP5.1.1 is much better than the one in Deepstream with JP5.1.2?

Accelerated Computing Intelligent Video Analytics DeepStream SDK

fanzh September 28, 2023, 8:09am 13

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

can you try network-mode=0? maybe in8 calibration file is bonding to specific low-level.
you can use this method to dump the Inference Input, then you will know if the inference inputs are the same.

Topic		Replies	Views
Agx xavier/jp5.1 GPU performance Fall short of expectationsa DeepStream SDK deepstream , chinese	3	168	September 23, 2024
Different performance of differnet DS sdk version on same model DeepStream SDK	3	394	May 16, 2022
DeepStream sample doesn't work with JetPack 5.1? DeepStream SDK	2	1112	February 3, 2023
The DeepStream lasted version can not accpet jetson5(in Orin ) DeepStream SDK	5	682	May 24, 2022
How to get my app to work with Deepstream 5.1? DeepStream SDK	11	611	March 23, 2021
Deepstream 6.1.1 and jetpack 5.1? DeepStream SDK	6	1281	February 5, 2023
Jeson Xavier NX, Triton 2.10, Deepstream 5.1 DeepStream SDK	1	962	June 2, 2021
Can't run deepstream-app-5.1 example using deepstream docker 5.1, but can do it with deepstream docker 5.0.1 DeepStream SDK	3	2114	March 4, 2021
Deepstream 5.0 documentation is out of date DeepStream SDK	8	678	October 18, 2020
Can I use the previous inference or tracking model on DeepStream 7.1 in this way? DeepStream SDK jetson , deepstream	24	536	February 24, 2025

mAP of engine model in Deepstream with JP5.1.1 is much better than the one in Deepstream with JP5.1.2?

Related topics