error:expecting model output to be a vector when using trt serving

hello, my output node is a feature map, so when I use trt serving, my config.pbtxt is as follow:

name: "xxx_model"
platform: "tensorrt_plan"
input [
   {
      name: "input_images"
      data_type: TYPE_FP32
      format: FORMAT_NCHW
      dims: [ 3, 1376, 800 ]
   }
]
output [
   {
      name: "yyy"
      data_type: TYPE_FP32
      dims: [ 344, 200, 1 ]
   }
]
instance_group [
  {
    kind: KIND_GPU,
    count: 4
  }
]
cc_model_filenames [
   {
      key: "1.0"
      value: "model.plan"
   }

but when I do a request using image_cilent.py, it shows error:

...
Exception: expecting model output to be a vector

so how should I get it fixed?

need help…

Hello,

Per https://github.com/NVIDIA/tensorrt-inference-server/blob/master/src/clients/python/grpc_image_client.py#L93

Allow any number of dimensions as long as all but 1 is size 1 (e.g. { 10 }, { 1, 10 }, { 10, 1, 1 } are all ok.

You output dim is dims: [ 344, 200, 1 ]

@NVES thanks for your help, but I need this output of dims: [ 344, 200, 1 ]

Hello,

Looks like you are using cc_model_filenames incorrectly. The “key” is a cuda compute capability that you want a particular model used for. I don’t think you mean to say cuda compute capability 1.0’

You don’t need to use cc_model_filenames at all if you are making sure that you generate the model with a GPU that matches what is on the TRTIS node.

Specifically, image_client is an example application that only works with image classification networks that return a single probability vector. If you have a different kind of network you may need to modify it to accept your model’s type of output(s) and do something appropriate with them.

@NVES, thanks for your advises for the parameter cc_model_filenames.

I comment the code about checking the output dim, now it works

Hi,

Can you let us know what was the change that you made in the code to make use of the existing client? Did you just comment those lines about dim? What did you do? Is your output a segmented image?