ValueError: Node... Axis is not unique while converting tensorflow segmentation model to tensorrt

Description

A clear and concise description of the bug or issue.

Environment

TensorRT Version: 7.2.3
GPU Type: GeForce GTX 1050 Ti
Nvidia Driver Version: 465
CUDA Version: 11.2
CUDNN Version: 8…*
Operating System + Version: ubuntu 18.04
Python Version (if applicable): 3.6
**TensorFlow Version (if applicable)**2.6:
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

I have tensorflow segmentation model with deeplabv3 (axial_swidernet_1_1_3_os16_axial_deeplab_cityscapes_trainfine_saved_mode) I am trying to convert this tensorlow model to tensorrt .
But I am getting following errors while converting this model

tyTraceback (most recent call last):
  File "convert.py", line 10, in <module>
    converter.convert()
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/compiler/tensorrt/trt_convert.py", line 1099, in convert
    frozen_func = convert_to_constants.convert_variables_to_constants_v2(func)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1154, in convert_variables_to_constants_v2
    converted_input_indices)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1080, in _construct_concrete_function
    new_output_names)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/eager/wrap_function.py", line 650, in function_from_graph_def
    wrapped_import = wrap_function(_imports_graph_def, [])
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/eager/wrap_function.py", line 628, in wrap_function
    collections={}),
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 1007, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/eager/wrap_function.py", line 87, in __call__
    return self.call_with_variable_creator_scope(self._fn)(*args, **kwargs)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/eager/wrap_function.py", line 93, in wrapped
    return fn(*args, **kwargs)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/eager/wrap_function.py", line 648, in _imports_graph_def
    importer.import_graph_def(graph_def, name="")
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 549, in new_func
    return func(*args, **kwargs)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 405, in import_graph_def
    producer_op_list=producer_op_list)
  File "/home/admin1/.virtualenvs/list/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 501, in _import_graph_def_internal
    raise ValueError(str(e))
ValueError: Node 'StatefulPartitionedCall/DeepLab/axial_swidernet/stage4/block2/attention/height_axis/query_rpe/Gather/axis' is not unique
pe or paste code here

Relevant Files

Steps To Reproduce

import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt

conversion_params = trt.DEFAULT_TRT_CONVERSION_PARAMS
conversion_params = conversion_params._replace(precision_mode="FP16")
converter = trt.TrtGraphConverterV2(
    input_saved_model_dir=input_saved_model_dir,
    conversion_params=conversion_params)
converter.convert()
def my_input_fn():
# Input for a single inference call, for a network that has two input tensors:
  Inp1 = np.random.normal(size=(8, 16, 16, 3)).astype(np.float32)
  inp2 = np.random.normal(size=(8, 16, 16, 3)).astype(np.float32)
  yield (inp1, inp2)
converter.build(input_fn=my_input_fn)
converter.save(output_saved_model_dir)

saved_model_loaded = tf.saved_model.load(
    output_saved_model_dir, tags=[tag_constants.SERVING])
graph_func = saved_model_loaded.signatures[
    signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
output = graph_func(input_data)[0].numpy()

Hi,
Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation

Also, request you to share your model and script if not shared already so that we can help you better.

Meanwhile, for some common errors and queries please refer to below link:
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/#error-messaging
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/#faq

Thanks!

Hi @NVES ,
I am using saved_model from tensorflow-2. deeplabv3 segmentation model name is → “axial_swidernet_1_1_3_os16_axial_deeplab_cityscapes_trainfine_saved_mode” .

How to use trtexec command for saved model ? provided link is not opening. Do I need to first convert saved_model to ONNX ?

Hi,

We recommend you to check the below samples links in case of tf-trt integration issues.

If issue persist, We recommend you to please reach out to Tensorflow forum.

Yes, we need to convert the model to ONNX to try trtexec.

Thank you.