Unexpected Detection Behavior After Migrating YOLOv4-Tiny Model from DeepStream 6.4 to 7.0

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform : Jetson Orin Nx
• DeepStream Version 7.0
• JetPack Version 6.0
• TensorRT Version 8.6.2
• Issue Type-question
• How to reproduce the issue: using a custom yolov4tiny.onnx model on DS7.0 that worked on DS6.4

I have recently migrated my application from DeepStream 6.4 to DeepStream 7.0 on JetPack 6.0, utilizing a Jetson Orin NX platform. While the migration process was successful, I’ve encountered an issue with my YOLOv4-Tiny model.

Issue Details:
• Model Behavior: After converting my YOLOv4-Tiny ONNX model to a TensorRT engine using DeepStream 7.0, the inference results display an unusually high number of detected objects, all with a confidence score of 1.0. This behavior was not present when using the same model in DeepStream 6.4.
• Configuration Consistency: The configuration file used for inference remains unchanged from the previous setup:

[property]
gpu-id=0
net-scale-factor=1
model-color-format=0 # 0: RGB, 1: BGR
labelfile-path=filter_classes.txt
model-engine-file=filter_recognition_epoch_80.onnx_b1_gpu0_fp32.engine
onnx-file=filter_recognition_epoch_80.onnx
infer-dims=3;416;416
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1
network-mode=0 # 0: FP32, 1: INT8, 2: FP16
num-detected-classes=2
interval=0
gie-unique-id=1
is-classifier=0
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=libnvds_infercustomparser_tao.so
[class-attrs-all]
pre-cluster-threshold=0.5
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0

Custom Parser: The post-processing library (libnvds_infercustomparser_tao.so) was recompiled in the DeepStream 7.0 environment to ensure compatibility.

Observations:
• The same ONNX model, when converted to a TensorRT engine in DeepStream 6.4, produced accurate detections.
• In DeepStream 7.0, the model detects an excessive number of objects, all with a confidence score of 1.0, on the same sample video.

I am seeking insights into the following:
1. ONNX Model Compatibility: Are there known compatibility issues when using ONNX models between DeepStream versions 6.4 and 7.0? Should an ONNX model function consistently across these versions?
2. Engine Conversion Process: Given that I am not utilizing INT8 precision (thus avoiding the need for a calibration file), should the ONNX to TensorRT engine conversion process yield consistent and accurate results across DeepStream versions?
3. Potential Causes and Solutions: What factors could contribute to the observed behavior of excessive detections with maximum confidence scores? Are there specific configuration parameters or steps I should revisit to ensure accurate detection results in DeepStream 7.0?

Any guidance or recommendations to address this issue would be greatly appreciated.

Thank you in advance for your assistance.

3

Could you try the FP16 mode with network-mode=2?

4

I have tried it already without success..

5

What version of JetPack did you install, 6.0 DP or 6.0 GA?

6

I have installed Jetpack 6.0 (rev.2) through sdkmanager.

“cat /etc/nv_tegra_release” gives

R36 (release), REVISION: 3.0, GCID: 36191598, BOARD: generic, EABI: aarch64, DATE: Mon May 6 17:34:21 UTC 2024
KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

and

“dpkg -l | grep nvidia-l4t-core”

ii nvidia-l4t-core 36.3.0-20240506102626 arm64 NVIDIA Core Package

So it should be GA.

7

So you didn’t update any other software and just updated the DeepStream version, then this issue came up. Is that right?
Did you confirm that the engine file was regenerated when you upgraded DeepStream? You can delete the engine file after the upgrade.

8

To update deepstream, I had to flash Jetpack 6.0, build everything new.
But at the end, I took the onnx file that was working in Deepstream 6.4 on jetson nano, nx and orin nx.

Our software uses face and facelandmark detection and another yolov4tiny.onnx model. Interestingly the other yolo4tiny.onnx model works perfectly after migration.

Yet the onnx model that was working fine on DS 6.4 now finds multiple items with confidence=1.0 after deleting old engine files and recreating on DS 7.0 new ones. I suppose the postprocessing part of yolo is not working, but even without nms or any suppression the result is the same.

Any suggestions?

9

Since you are updating from 6.4 to 7.0, there are not much sofrwares on upgrades. Jetson model Platform and OS Compatibility
You don’t have to upgrade Jetpack or other softwares and build everything. Could you try to use the engine file generated on the DeepStream 6.4 to run that on the DeepStream 7.0 and check the result?

10

I appreciate your help.

4889 0xaaaaf82272f0 WARN nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<filter_recognition_probe> NvDsInferContext[UID 5]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2083> [UID = 5]: deserialize engine from file :/home/xxxx.engine failed

No, it does not accept the previous engine. It builds the new one, which does not work

11

Since DeepStream 7.0 and DeepStream6.4 both use the same version of Jetpack, this is odd.
Let’s reconfirm the following things.

  1. What version of Jetapck does your DeepStream 6.4 run on?
  2. Can you try to upgrade to DeepStream 7.0 without flashing the Jetpack?
12

Is DS7.0 compatible with jetpack 5.x ? The last system that everything worked well was ubuntu 20.04 and therefore as I check again, it should be 6.3 (as 6.4 is not compatible on ubuntu20.04) Does this explain where the problem is?

13

We cannot guarantee the compatibility.

Yes. The TensorRT are different versions. How were your YOLOv4 model trained? You may need to retrain it with the appropriate TensorRT version.

14

Sorry,it has been duplicated, my fault, I have opened two topics in 24 hours, than I forgot one, but as it was replied, I continued that too: can you please add any comments (if needed) on

15

Sure. You can choose one of these to track, we will help analyze the problem too.