object detection saved model serving

AI & Data Science Deep Learning (Training & Inference) Triton Inference Server - archived
1

I have deployed object detection saved model to TRTIS, but unfortunately I can’t serve it.
There is status api response:

id: "inference:0"
version: "1.7.0"
uptime_ns: 54413585814048
model_status {
  key: "package_model"
  value {
    config {
      name: "package_model"
      platform: "tensorflow_savedmodel"
      version_policy {
        latest {
          num_versions: 1
        }
      }
      max_batch_size: 1
      input {
        name: "inputs"
        data_type: TYPE_STRING
        dims: 1
        reshape {
        }
      }
      output {
        name: "num_detections"
        data_type: TYPE_FP32
        dims: 1
        reshape {
        }
      }
      output {
        name: "detection_classes"
        data_type: TYPE_FP32
        dims: 300
      }
      output {
        name: "detection_scores"
        data_type: TYPE_FP32
        dims: 300
      }
      output {
        name: "detection_boxes"
        data_type: TYPE_FP32
        dims: 300
        dims: 4
      }
      instance_group {
        name: "package_model"
        count: 1
        gpus: 0
        kind: KIND_GPU
      }
      default_model_filename: "model.savedmodel"
    }
    version_status {
      key: 1
      value {
        ready_state: MODEL_READY
      }
    }
  }
}
status_stats {
  success {
    count: 8
    total_time_ns: 270003
  }
}
ready_state: SERVER_READY

python client code:

import numpy as np
from tensorrtserver.api import *

url = "localhost:8001"
protocol = ProtocolType.from_str("gRPC")
model_name = "package_model"
batch_size = 1
image_path = "/images/20191106_152725_00040891.jpg"
image_bytes = open(image_path, "rb").read()

ctx = ServerStatusContext(url, protocol, model_name)
server_status = ctx.get_server_status()
status = server_status.model_status[model_name]
config = status.config

input_name = config.input[0].name
outputs = [output.name for output in config.output]

input_batch = []
for b in range(batch_size):
    image_data = np.expand_dims(image_bytes, axis=0)
    input_batch.append(image_data)

ctx = InferContext(url, protocol, model_name)
results = ctx.run(
    {input_name: input_batch},
    {output: InferContext.ResultFormat.RAW for output in outputs},
    batch_size)

image_index = 0
print('detection_boxes: ', results['detection_boxes'][image_index].shape)
print('detection_scores: ', results['detection_scores'][image_index].shape)
print('detection_classes: ', results['detection_classes'][image_index].shape)

I get exception:

tensorrtserver.api.InferenceServerException: [inference:0 87] 2 root error(s) found.
  (0) Invalid argument: Invalid JPEG data or crop window, data size 4
	 [[{{node map/while/decode_image/cond_jpeg/DecodeJpeg}}]]
	 [[map/while/decode_image/cond_jpeg/cond_png/cond_gif/check_channels/_155]]
  (1) Invalid argument: Invalid JPEG data or crop window, data size 4
	 [[{{node map/while/decode_image/cond_jpeg/DecodeJpeg}}]]

on the other hand I can serve it using tensorflow api:

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
import numpy as np
import tensorflow as tf

image_path = "/images/20191106_152725_00040891.jpg"
image_bytes = open(image_path, "rb").read()

predict_fn = tf.contrib.predictor.from_saved_model("/model_repository/package_model/1/model.savedmodel")
output_data = predict_fn({"inputs": np.expand_dims(image_bytes, axis=0)})