Yolov7 onnx model convert to trt model error in deepstream 6.2, but use trtexec ok

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only)
• TensorRT Version TRT 8.5.2.2
• NVIDIA GPU Driver Version (valid for GPU only) 470.82.01
• Issue Type( questions, new requirements, bugs) questions

I use deepstream to load yolo model, I want to use deepstrem auto convert onnx model to trt model, it occur error.

this is the config file

[property]
gpu-id=0

net-scale-factor=0.0039215697906911373

onnx-file= ./models/yolov7darknet.onnx
#onnx-file= ./models/yolov7-tiny_all.onnx

#model-engine-file=./models/yolov7darknet.cache
#model-engine-file=./models/yolov7-tiny_all.cache

labelfile-path=./models/labels_city.txt

#int8-calib-file=./models/yolov7darknet.table.int8

process-mode=1

model-color-format=0

## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2

num-detected-classes=1

interval=0

gie-unique-id=1

output-blob-names=ouputs

#force-implicit-batch-dim=1

parse-bbox-func-name=NvDsInferParseCustomYoloV7Darknet
custom-lib-path=/lib/x86_64-linux-gnu/libnvds_infercustomparser.so
## 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=2
#scaling-filter=0
#scaling-compute-hw=0

#Use the config params below for dbscan clustering mode
#[class-attrs-all]
#detected-min-w=4
#detected-min-h=4
#minBoxes=3


#Use the config params below for NMS clustering mode
[class-attrs-all]
#topk=20
nms-iou-threshold=0.3
pre-cluster-threshold=0.5


## Per class configurations
#[class-attrs-0]
#topk=20
#nms-iou-threshold=0.45
#pre-cluster-threshold=0.25

#[class-attrs-1]
#pre-cluster-threshold=0.05
#eps=0.7
#dbscan-min-score=0.5

#[class-attrs-2]
#pre-cluster-threshold=0.1
#eps=0.6
#dbscan-min-score=0.95

#[class-attrs-3]
#pre-cluster-threshold=0.05
#eps=0.7
#dbscan-min-score=0.5

error logs:

 <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: onnx2trt_utils.cpp:377: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
WARNING: [TRT]: Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
WARNING: [TRT]: Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
ERROR: [TRT]: 4: [shapeCompiler.cpp::evaluateShapeChecks::1180] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: IShuffleLayer node_of_090_convolutional_logistic_reshape_1: reshaping failed for tensor: 090_convolutional_logistic Reshape would change volume.)
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:1124 Build engine failed from config file
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:816 failed to build trt engine.
0:00:10.248574860   370      0x32bd100 ERROR                nvinfer gstnvinfer.cpp:674:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1943> [UID = 1]: build engine file failed
0:00:10.339732664   370      0x32bd100 ERROR                nvinfer gstnvinfer.cpp:674:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2029> [UID = 1]: build backend context failed
0:00:10.339765603   370      0x32bd100 ERROR                nvinfer gstnvinfer.cpp:674:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1266> [UID = 1]: generate backend failed, check config file settings
0:00:10.340368804   370      0x32bd100 WARN                 nvinfer gstnvinfer.cpp:888:gst_nvinfer_start:<primary-nvinference-engine> error: Failed to create NvDsInferContext instance
0:00:10.340381723   370      0x32bd100 WARN                 nvinfer gstnvinfer.cpp:888:gst_nvinfer_start:<primary-nvinference-engine> error: Config file path: ../config/config_infer_primary_darknet_tiny.amd64.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

but when use trtexec, it work well

/usr/src/tensorrt/bin/trtexec --onnx=yolov7-tiny_all.onnx --fp16 --saveEngine=yolov7-tiny_all_sm75.cache

and in config file set “model-engine-file=yolov7-tiny_all_sm75.cache”, the program can run.

(service_capture:389): GLib-GObject-WARNING **: 13:52:38.254: g_object_get_is_valid_property: object class 'GstUDPSrc' has no property named 'pt'
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
0:00:04.274279549   389      0x306d700 INFO                 nvinfer gstnvinfer.cpp:680:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1909> [UID = 1]: deserialized trt engine from :/opt/captureService/config/models/yolov7-city-sm75-b10-ds62-0817.cache
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT 000_net         3x416x416       
1   OUTPUT kFLOAT ouputs          10647x6         

0:00:04.345208231   389      0x306d700 INFO                 nvinfer gstnvinfer.cpp:680:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2012> [UID = 1]: Use deserialized engine model: /opt/captureService/config/models/yolov7-city-sm75-b10-ds62-0817.cache
0:00:04.349855052   389      0x306d700 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-nvinference-engine> [UID 1]: Load new model:../config/config_infer_primary_darknet_tiny.amd64.txt sucessfully

what’s different with deepstream and trtexec

Could you attach the model to us?

yolov7-tiny_all.zip (21.4 MB)

Since the batch of your model is 2. Could you try to add batch-size=2 in your config file?

I have tried, but not help

I have tried that with our latest version 7.0. It can generate the engine file normally if I set the batch-size=2. You can try to upgrade your DeepStream.

OK, I will try with version 7.0, but I want to know If there is a problem with version 6.2?

I have tried that with our DeepStream 6.2 docker, it also can generate the engine file. Could you update your driver version first? DeepStream 6.2 needs R525.85.12.