Fallback to TF when Input tensor has an unknown non-batch dimension converting to INT8

I’ve tried to convert my graph to INT8 in TF-TRT. Here is my stack trace:

TensorRT node TRTEngineOp_0 added for segment 0 consisting of 3 nodes failed: Invalid argument: Validation failed for TensorRTInputPH_0 and input slot 0: Input tensor with shape [?,?,?,3] has an unknown non-batch dimension at dim 1. Fallback to TF...

However it converts fine to FP16. Do I have to set the dimensions in order for the graph to convert? Do you have any documentation for this?

Hi,

Can you provide a repro package with the code and scripts you’re trying to run for further debugging?

Please also provide your OS, GPU, and TensorRT + CUDA versions

Thanks,
NVIDIA Enterprise Support

I meet this error too…FP16 is ok, while INT8 catch the error.the repo is https://github.com/YunYang1994/tensorflow-yolov3, I use the tensorflow:19.03 images to run it. the convert code is:

with tf.gfile.GFile(‘./yolov3_coco.pb’, ‘rb’) as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
return_elements = [“input/input_data:0”, “pred_sbbox/concat_2:0”, “pred_mbbox/concat_2:0”, “pred_lbbox/concat_2:0”]
trt_graph = trt.create_inference_graph(
input_graph_def=graph_def,
outputs=return_elements,
max_batch_size=32,
max_workspace_size_bytes=2 << 20,
is_dynamic_op=True,
use_calibration=True,
precision_mode=‘INT8’)

    self.return_tensors = tf.import_graph_def(
        trt_graph,
        return_elements=return_elements)