Triton ensemble model version

Hello!

I’m trying to setup a Deepstream pipeline using NvDsInferServer plugin to use a Triton ensemble.
I get this error:

E0702 16:56:03.198728 1673 model_repository_manager.cc:1705] output 'OUTPUT1' for ensemble 'ensemble_name' is not written
I0702 16:56:03.198738 1170 model_repository_manager.cc:475] VersionStates() 'ensemble_name'
ERROR: infer_trtis_server.cpp:1044 Triton: failed to load model ensemble_name, triton_err_str:Internal, err_msg:failed to load 'ensemble_name', no version is available
ERROR: infer_trtis_backend.cpp:45 failed to load model: ensemble_name, nvinfer error:NVDSINFER_TRTIS_ERROR

It’s the first time I’m trying ensemble. Never seen that error with regular models.

I’ve been reading the documentation at:
DS-nvinferserver#triton-ensemble-models and Triton-architecture.md#ensemble-models and I can’t find what I’m doing wrong.
I’ve created an empty version folder with different version policies and version configurations with no success.

Thanks!

Details:
Triton version: 2.5
Deepstream version: 5.1
Container: deepstream:5.1-21.02-triton
Application: deepstream python bindings.

My ensemble folder:

β”œβ”€ path/to/models
  β”œβ”€β”€ ensemble_name
  |  β”œβ”€β”€ config.pbtxt
  |  β”œβ”€β”€ 1             (empty)
  β”œβ”€β”€ MODEL1
  |  β”œβ”€β”€ config.pbtxt
  |  β”œβ”€β”€ 1
  β”œβ”€β”€ MODEL2
  |  β”œβ”€β”€ config.pbtxt
  |  β”œβ”€β”€ 1

config.pbtxt:
(Ignore tensor dimensions, I set them here to 0)

name: "ensemble_name"
platform: "ensemble"
max_batch_size: 1
input [
  {
    name: "INPUT"
    data_type: TYPE_FP32
    dims: [ 0, 0, 0 ]
  }
]
output [
  {
    name: "OUTPUT"
    data_type: TYPE_FP32
    dims: [ 0, 0 ]
  }
]

ensemble_scheduling {
  step [
    {
      model_name: "MODEL1"
      model_version: -1
      input_map {
        key: "INPUT"
        value: "INPUT"
      }
      output_map {
        key: "OUTPUT"
        value: "OUTPUT"
      }
    },
    {
      model_name: "MODEL2"
      model_version: -1
      input_map {
        key: "INPUT"
        value: "INPUT"
      }
      output_map {
        key: "OUTPUT"
        value: "OUTPUT"
      }
    }
  ]
}

inferserver.txt:

infer_config {
  unique_id: 10
  gpu_ids: [0]
  max_batch_size: 1
  backend {
    inputs: [ {
      name: "INPUT1"
    }]
    outputs: [{
      name: "OUTPUT1"
    }]

    trt_is {
      model_name: "ensemble_name"
      version: -1
      model_repo {
        root: "path/to/models"
        log_level: 3
        tf_gpu_memory_fraction: 0.5
        tf_disable_soft_placement: 1
        strict_model_config: false
      }
    }
  }
  
  preprocess {
    network_format: IMAGE_FORMAT_RGB
    tensor_order: TENSOR_ORDER_NHWC
    frame_scaling_hw: FRAME_SCALING_HW_DEFAULT
    maintain_aspect_ratio: 0
    frame_scaling_filter: 0
    normalize {
      scale_factor: 1.0
      channel_offsets: [0, 0, 0]
    }
  }
}

I solved the problem by editing and assigning properly the input and output tensors in the ensemble pbtxt file.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.