Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
A2 GPU
• DeepStream Version
6.2
• NVIDIA GPU Driver Version (valid for GPU only)
535.161.08
• Issue Type( questions, new requirements, bugs)
Documentation/configuration
I am wanting to host the OCDNet model (downloaded from NGC Catalog as ocdnet_fan_tiny_2x_icdar_pruned.onnx) in Triton Inference Server, and inference from a DeepStream application through the NvDsInferServer element. I’m already doing this with other models (PeopleNet, TrafficCamNet, etc).
I need the Triton config.pbtxt for this model, if it includes any model-specific configuration, but more importantly, I need a valid nvinferserver-config file that will work with this model. Again, I have nvinferserver-config files for the other models, but not for OCDNet.
It’s extremely difficult to build these files from scratch because you must know the input/output tensor information, etc. For reference, I’ve tried the config below, but DeepStream fails with “nvinferserver gstnvinferserver.cpp:407:gst_nvinfer_server_logger: nvinferserver[UID 10]: Error in specifyBackendDims() <infer_grpc_context.cpp:164> [UID = 10]: failed to specify input dims triton backend for model:ocdnet, nvinfer error:NVDSINFER_TRITON_ERROR”
infer_config {
unique_id: 10
gpu_ids: [0]
max_batch_size: 3
backend {
inputs: [ {
name: “input”
}]
outputs: [{
name: “pred”
}
]
triton {
model_name: “ocdnet”
version: -1
grpc {
url: “triton:8001”
enable_cuda_buffer_sharing: true
}
}
}
preprocess {
network_format: MEDIA_FORMAT_NONE
tensor_order: TENSOR_ORDER_LINEAR
tensor_name: “input_1:0”
maintain_aspect_ratio: 0
frame_scaling_hw: FRAME_SCALING_HW_DEFAULT
frame_scaling_filter: 1
normalize {
scale_factor: 0.0039215697906911373
channel_offsets: [0, 0, 0]
}
}
postprocess {
detection {
num_detected_classes: 1
per_class_params {
key: 0
value { pre_threshold: 0.4 }
}
nms {
confidence_threshold:0.2
topk:20
iou_threshold:0.5
}
}
}
extra {
copy_input_to_host_buffers: false
output_buffer_pool_size: 2
}
}
input_control {
process_mode: PROCESS_MODE_FULL_FRAME
interval: 0
}
output_control {
detect_control {
default_filter {
bbox_filter {
min_width: 10
min_height: 10
}
}
}
}