Error converting tf saved_model to trt

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

Following Blog

Environment

TensorRT Version: 7.2.2
Nvidia Driver Version: 460
CUDA Version: 11.2
Operating System + Version: Ubuntu 18.04
Python Version: 3.6
TensorFlow Version: 2.5

My .pb saved model info

$ saved_model_cli show --dir ../saved_model/ --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
  inputs['input_tensor'] tensor_info:
      dtype: DT_UINT8
      shape: (1, -1, -1, 3)
      name: serving_default_input_tensor:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['detection_anchor_indices'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 100)
      name: StatefulPartitionedCall:0
  outputs['detection_boxes'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 100, 4)
      name: StatefulPartitionedCall:1
  outputs['detection_classes'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 100)
      name: StatefulPartitionedCall:2
  outputs['detection_multiclass_scores'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 100, 3)
      name: StatefulPartitionedCall:3
  outputs['detection_scores'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 100)
      name: StatefulPartitionedCall:4
  outputs['num_detections'] tensor_info:
      dtype: DT_FLOAT
      shape: (1)
      name: StatefulPartitionedCall:5
  outputs['raw_detection_boxes'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 49104, 4)
      name: StatefulPartitionedCall:6
  outputs['raw_detection_scores'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, 49104, 3)
      name: StatefulPartitionedCall:7
Method name is: tensorflow/serving/predict

(Considering above SavedModel SignatureDef of saved_model.pb)
So what should be the tf2onnx.convert command?
somewhat like this…?

python3 -m tf2onnx.convert  --input ../saved_model.pb --inputs serving_default_input_tensor:0 --outputs StatefulPartitionedCall:7 --output model.onnx
2

Hi @skim1,

Please refer following,

Thank you.

5 
The given SavedModel SignatureDef contains the following input(s):
inputs['input_tensor'] tensor_info:
      dtype: DT_UINT8
      shape: (1, -1, -1, 3)
      name: serving_default_input_tensor:0

@spolisetty is it look like invalid shape?

6

Hi @skim1,

Looks like you’re using UINT8 data type. UINT8 is not supported by TensorRT. Please fix this.
Please share us ONNX model if you face any further issues.

Thank you.