Inference Builder

AI & Data Science Computer Vision & Image Processing
1

DeepStream nvdsinfer engine deserialization failed — NvDsInfer Error: NVDINFER_CONFIG_FAILED

Hi NVIDIA team,

Hi NVIDIA team,

I’m facing an issue while running my DeepStream pipeline that uses a custom YOLOv5 face detection model.
The pipeline fails to start with the following error:

nvdsinfer gstnvinfer.cpp:682:gst_nvinfer_logger:
NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend()
<nvdsinfer_context_impl.cpp:2097> deserialize backend context from engine file failed
…
Error in NvDsInferContextImpl::buildModel(): deserialize backend context from engine file failed
…
NvDsInfer Error: NVDINFER_CONFIG_FAILED
Unable to set the pipeline to the playing state.

MODEL CONFIG

class-attrs-all:
pre-cluster-threshold: 0.65
topk: 300
nms-iou-threshold: 0.7
eps: 0.2
group-threshold: 1
detected-min-w: 65
detected-min-h: 65

property:
gpu-id: 0
net-scale-factor: 0.0039215697906911373   # 1/255 normalization
infer-dims: 3;384;640
onnx-file: yolov5s-face_32x3x384x640_landmarks.onnx
model-engine-file: yolov5s-face_32x3x384x640_landmarks.onnx_b4_gpu0_int8.engine
labelfile-path: fd_labels.txt
custom-lib-path: libnvdsinfer_custom_impl_Yolo_face.so
force-implicit-batch-dim: 1
batch-size: 32
network-mode: 1       # FP16
num-detected-classes: 1
interval: 0
gie-unique-id: 1
output-tensor-meta: 1
model-color-format: 0
process-mode: 1
network-type: 0
cluster-mode: 4
maintain-aspect-ratio: 1
symmetric-padding: 1
parse-bbox-instance-mask-func-name: NvDsInferParseYoloFace
output-instance-mask: 0

Pipeline YAML Configuration

name: "faceintel-pipeline"
model_repo: "/workspace/models"

models:
- name: face_detection
  backend: deepstream/nvinfer
  max_batch_size: 4
  input:
  - name: media_url
    data_type: TYPE_CUSTOM_BINARY_URLS
    dims: [ -1 ]
  - name: mime
    data_type: TYPE_CUSTOM_DS_MIME
    dims: [ -1 ]
  - name: source_config
    data_type: TYPE_CUSTOM_DS_SOURCE_CONFIG
    dims: [ 1 ]
  output:
  - name: output
    data_type: TYPE_CUSTOM_DS_METADATA
    dims: [ -1 ]
  parameters:
    batch_timeout: 33333
    infer_config_path:
      - nvdsinfer_config.yaml
    tracker_config:
      ll_lib_file: /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
      ll_config_file: /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
      width: 384
      height: 640
      display_tracking_id: true
    render_config:
      enable_display: true
      enable_osd: true
    perf_config:
      enable_fps_logs: true
      enable_latency_logs: true
    inference_timeout: 5

CHECKED THE SIZE OF THE MODEL

dim {
dim_value: 32
}
dim {
dim_value: 3
}
dim {
dim_value: 384
}
dim {
dim_value: 640
}

Error Summary

  • The .engine file was successfully generated once, but on subsequent runs it fails to load with:

    backend can not support dims: 3x384x640
deserialize backend context from engine file failed
failed to match config params

  • After this, the pipeline exits with NvDsInfer Error: NVDINFER_CONFIG_FAILED.

What I’ve Tried

  • Verified ONNX input dimensions: [1x3x384x640]

  • Checked that DeepStream and TensorRT versions match

  • Tried deleting the .engine file to force a rebuild

  • Ensured network-mode: 1 (FP16) and force-implicit-batch-dim: 1 are set correctly

Is there a mismatch between infer-dims and model input shape handling in DeepStream 8.0 in nvidia inference builder?

can i use directly the trt model to pass with the inference builder instead og ONNX?