Example nvinferserver not working

I am creating a pgie using one of your example files. I am getting the error:

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
[NvMultiObjectTracker] Initialized
0:00:00.298985586  4129      0x3a502a0 WARN           nvinferserver gstnvinferserver_impl.cpp:284:validatePluginConfig:<primary-inference> warning: Configuration file batch-size reset to: 16
0:00:00.299038736  4129      0x3a502a0 WARN           nvinferserver gstnvinferserver_impl.cpp:290:validatePluginConfig:<primary-inference> warning: Configuration file unique-id reset to: 1
WARNING: infer_proto_utils.cpp:201 backend.trt_is is deprecated. updated it to backend.triton
I1229 16:40:40.690139 4129 metrics.cc:290] Collecting metrics for GPU 0: Tesla T4
I1229 16:40:40.940519 4129 libtorch.cc:1029] TRITONBACKEND_Initialize: pytorch
I1229 16:40:40.940553 4129 libtorch.cc:1039] Triton TRITONBACKEND API version: 1.4
I1229 16:40:40.940560 4129 libtorch.cc:1045] 'pytorch' TRITONBACKEND API version: 1.4
2021-12-29 16:40:41.047776: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
I1229 16:40:41.090509 4129 tensorflow.cc:2169] TRITONBACKEND_Initialize: tensorflow
I1229 16:40:41.090538 4129 tensorflow.cc:2179] Triton TRITONBACKEND API version: 1.4
I1229 16:40:41.090547 4129 tensorflow.cc:2185] 'tensorflow' TRITONBACKEND API version: 1.4
I1229 16:40:41.090554 4129 tensorflow.cc:2209] backend configuration:
{"cmdline":{"allow-soft-placement":"true","gpu-memory-fraction":"0.400000"}}
I1229 16:40:41.092294 4129 onnxruntime.cc:1970] TRITONBACKEND_Initialize: onnxruntime
I1229 16:40:41.092315 4129 onnxruntime.cc:1980] Triton TRITONBACKEND API version: 1.4
I1229 16:40:41.092325 4129 onnxruntime.cc:1986] 'onnxruntime' TRITONBACKEND API version: 1.4
I1229 16:40:41.111152 4129 openvino.cc:1193] TRITONBACKEND_Initialize: openvino
I1229 16:40:41.111171 4129 openvino.cc:1203] Triton TRITONBACKEND API version: 1.4
I1229 16:40:41.111178 4129 openvino.cc:1209] 'openvino' TRITONBACKEND API version: 1.4
I1229 16:40:41.220996 4129 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7fde14000000' with size 268435456
I1229 16:40:41.221340 4129 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
I1229 16:40:41.221992 4129 server.cc:504] 
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I1229 16:40:41.222053 4129 server.cc:543] 
+-------------+-------------------------------+-------------------------------+
| Backend     | Path                          | Config                        |
+-------------+-------------------------------+-------------------------------+
| tensorrt    | <built-in>                    | {}                            |
| pytorch     | /opt/tritonserver/backends/py | {}                            |
|             | torch/libtriton_pytorch.so    |                               |
| tensorflow  | /opt/tritonserver/backends/te | {"cmdline":{"allow-soft-place |
|             | nsorflow1/libtriton_tensorflo | ment":"true","gpu-memory-frac |
|             | w1.so                         | tion":"0.400000"}}            |
| onnxruntime | /opt/tritonserver/backends/on | {}                            |
|             | nxruntime/libtriton_onnxrunti |                               |
|             | me.so                         |                               |
| openvino    | /opt/tritonserver/backends/op | {}                            |
|             | envino/libtriton_openvino.so  |                               |
+-------------+-------------------------------+-------------------------------+
I1229 16:40:41.222083 4129 server.cc:586] 
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+
I1229 16:40:41.222157 4129 tritonserver.cc:1718] 
+----------------------------------+------------------------------------------+
| Option                           | Value                                    |
+----------------------------------+------------------------------------------+
| server_id                        | triton                                   |
| server_version                   | 2.13.0                                   |
| server_extensions                | classification sequence model_repository |
|                                  |  model_repository(unload_dependents) sch |
|                                  | edule_policy model_configuration system_ |
|                                  | shared_memory cuda_shared_memory binary_ |
|                                  | tensor_data statistics                   |
| model_repository_path[0]         | /opt/nvidia/deepstream/deepstream-6.0/sa |
|                                  | mples/triton_model_repo                  |
| model_control_mode               | MODE_EXPLICIT                            |
| strict_model_config              | 0                                        |
| pinned_memory_pool_byte_size     | 268435456                                |
| cuda_memory_pool_byte_size{0}    | 67108864                                 |
| min_supported_compute_capability | 6.0                                      |
| strict_readiness                 | 1                                        |
| exit_timeout                     | 30                                       |
+----------------------------------+------------------------------------------+
W1229 16:40:41.223100 4129 autofill.cc:237] Autofiller failed to retrieve model. Error Details: Internal: unable to autofill for 'ssd_inception_v2_coco_2018_01_28' due to no version directories
W1229 16:40:41.223121 4129 autofill.cc:243] Proceeding with simple config for now
E1229 16:40:41.223469 4129 model_repository_manager.cc:1424] failed to load model 'ssd_inception_v2_coco_2018_01_28': at least one version must be available under the version policy of model 'ssd_inception_v2_coco_2018_01_28'
ERROR: infer_trtis_server.cpp:1053 Triton: failed to load model ssd_inception_v2_coco_2018_01_28, triton_err_str:Invalid argument, err_msg:load failed for model 'ssd_inception_v2_coco_2018_01_28': at least one version must be available under the version policy of model 'ssd_inception_v2_coco_2018_01_28'
ERROR: infer_trtis_backend.cpp:45 failed to load model: ssd_inception_v2_coco_2018_01_28, nvinfer error:NVDSINFER_TRITON_ERROR
ERROR: infer_trtis_backend.cpp:184 failed to initialize backend while ensuring model:ssd_inception_v2_coco_2018_01_28 ready, nvinfer error:NVDSINFER_TRITON_ERROR
0:00:00.875159940  4129      0x3a502a0 ERROR          nvinferserver gstnvinferserver.cpp:362:gst_nvinfer_server_logger:<primary-inference> nvinferserver[UID 1]: Error in createNNBackend() <infer_trtis_context.cpp:248> [UID = 1]: failed to initialize triton backend for model:ssd_inception_v2_coco_2018_01_28, nvinfer error:NVDSINFER_TRITON_ERROR
I1229 16:40:41.223593 4129 server.cc:234] Waiting for in-flight requests to complete.
I1229 16:40:41.223602 4129 server.cc:249] Timeout 30: Found 0 live models and 0 in-flight non-inference requests
0:00:00.875259827  4129      0x3a502a0 ERROR          nvinferserver gstnvinferserver.cpp:362:gst_nvinfer_server_logger:<primary-inference> nvinferserver[UID 1]: Error in initialize() <infer_base_context.cpp:81> [UID = 1]: create nn-backend failed, check config file settings, nvinfer error:NVDSINFER_TRITON_ERROR
0:00:00.875276063  4129      0x3a502a0 WARN           nvinferserver gstnvinferserver_impl.cpp:507:start:<primary-inference> error: Failed to initialize InferTrtIsContext
0:00:00.875285488  4129      0x3a502a0 WARN           nvinferserver gstnvinferserver_impl.cpp:507:start:<primary-inference> error: Config file path: /home/ubuntu/pycharm/projects/components/dstest1_pgie_inferserver_config.txt
0:00:00.875745113  4129      0x3a502a0 WARN           nvinferserver gstnvinferserver.cpp:460:gst_nvinfer_server_start:<primary-inference> error: gstnvinferserver_impl start failed
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
[NvMultiObjectTracker] De-initialized
Warning: gst-library-error-quark: Configuration file batch-size reset to: 16 (5): gstnvinferserver_impl.cpp(284): validatePluginConfig (): /GstPipeline:pipeline0/GstNvInferServer:primary-inference
Warning: gst-library-error-quark: Configuration file unique-id reset to: 1 (5): gstnvinferserver_impl.cpp(290): validatePluginConfig (): /GstPipeline:pipeline0/GstNvInferServer:primary-inference
Error: gst-resource-error-quark: Failed to initialize InferTrtIsContext (1): gstnvinferserver_impl.cpp(507): start (): /GstPipeline:pipeline0/GstNvInferServer:primary-inference:
Config file path: /home/ubuntu/pycharm/projects//dstest1_pgie_inferserver_config.txt
Process finished with exit code 0

Same code works if I use a custom model that I created.

To initialize the pgie:

pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
pgie.set_property("config-file-path",
                              "/home/ubuntu/pycharm/projects/dstest1_pgie_inferserver_config.txt")
            

The config file for the pgie is a copy-paste from this file deepstream_python_apps/dstest1_pgie_inferserver_config.txt at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub . I just replaced the relative paths with absolute paths.
For completeness:

################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

infer_config {
  unique_id: 5
  gpu_ids: [0]
  max_batch_size: 4
  backend {
    trt_is {
      model_name: "ssd_inception_v2_coco_2018_01_28"
      version: -1
      model_repo {
        root: "/opt/nvidia/deepstream/deepstream-6.0/samples/trtis_model_repo"
        log_level: 2
        tf_gpu_memory_fraction: 0.4
        tf_disable_soft_placement: 0
      }
    }
  }

  preprocess {
    network_format: IMAGE_FORMAT_RGB
    tensor_order: TENSOR_ORDER_NONE
    maintain_aspect_ratio: 0
    normalize {
      scale_factor: 1.0
      channel_offsets: [0, 0, 0]
    }
  }

  postprocess {
    labelfile_path: "/opt/nvidia/deepstream/deepstream-6.0/samples/trtis_model_repo/ssd_inception_v2_coco_2018_01_28/labels.txt"
    detection {
      num_detected_classes: 91
      custom_parse_bbox_func: "NvDsInferParseCustomTfSSD"
      nms {
        confidence_threshold: 0.5
        iou_threshold: 0.3
        topk : 20
      }
    }
  }

  extra {
    copy_input_to_host_buffers: false
  }

  custom_lib {
    path: "/opt/nvidia/deepstream/deepstream/lib/libnvds_infercustomparser.so"
  }
}
input_control {
  process_mode: PROCESS_MODE_FULL_FRAME
  interval: 0
}
output_control {
  output_tensor_meta: true
}

Dockerfile:

FROM nvcr.io/nvidia/deepstream:6.0-triton

ENV GIT_SSL_NO_VERIFY=1

RUN sh docker_python_setup.sh
RUN update-alternatives --set python3 /usr/bin/python3.8
RUN apt install --fix-broken -y
RUN apt -y install python3-gi python3-gst-1.0 python-gi-dev git python3 python3-pip cmake g++ build-essential \
  libglib2.0-dev python3-dev python3.8-dev libglib2.0-dev-bin python-gi-dev libtool m4 autoconf automake

RUN cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps && \
    git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps.git
RUN cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/deepstream_python_apps && \
    git submodule update --init
RUN cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/deepstream_python_apps/3rdparty/gst-python/ && \
   ./autogen.sh && \
   make && \
   make install

RUN pip3 install --upgrade pip
RUN cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/deepstream_python_apps/bindings && \
    mkdir build && \
    cd build && \
    cmake -DPYTHON_MAJOR_VERSION=3 -DPYTHON_MINOR_VERSION=8 -DPIP_PLATFORM=linux_x86_64 -DDS_PATH=/opt/nvidia/deepstream/deepstream-6.0 .. && \
    make && \
    pip3 install pyds-1.1.0-py3-none-linux_x86_64.whl
RUN cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/deepstream_python_apps && \
    mv apps/* ./
RUN pip3 install --upgrade pip
RUN pip3 install numpy opencv-python

# RTSP
RUN apt update && \
    apt install -y python3-gi python3-dev python3-gst-1.0
RUN apt update && \
    apt install -y libgstrtspserver-1.0-0 gstreamer1.0-rtsp && \
    apt install -y libgirepository1.0-dev && \
    apt-get install -y gobject-introspection gir1.2-gst-rtsp-server-1.0

# DEVELOPMENT TOOLS
RUN apt install -y ipython3 graphviz

Sory for the late response, is this still an issue to support? Thanks

Please do below:
echo “Downloading TensorFlow SSD Coco…”
mkdir -p ${MODEL_REPO_DIR}/ssd_inception_v2_coco_2018_01_28/1
wget -O /tmp/ssd_inception_v2_coco_2018_01_28.tar.gz
http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2018_01_28.tar.gz
(cd /tmp && tar xzf ssd_inception_v2_coco_2018_01_28.tar.gz)
mv /tmp/ssd_inception_v2_coco_2018_01_28/frozen_inference_graph.pb
${MODEL_REPO_DIR}/ssd_inception_v2_coco_2018_01_28/1/model.graphdef
rm -fr /tmp/ssd_inception_v2_coco_2018_01_28.tar.gz /tmp/ssd_inception_v2_coco_2018_01_28