Hi,
I don’t have a ton of experience with Tensorflow, so please correct this if it looks wrong.
I tried to make a simple example to verify if tf2onnx can handle ConvTranspose2D.
- Create simple model with conv2d_transpose op:
pip install tensorflow==1.14
# repro.py
import numpy as np
import tensorflow as tf
# https://datascience.stackexchange.com/questions/26451/how-to-calculate-the-output-shape-of-conv2d-transpose
x = tf.placeholder(dtype=tf.float32, shape=(None, 7, 7, 32))
dcout = tf.layers.conv2d_transpose(x, 64, 4, 3, padding="valid")
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
xin = np.random.rand(1,7,7,32)
out = sess.run(dcout, feed_dict={x:xin})
print(out.shape)
# freeze the graph so that it can be converted to onnx
output_graph_def = tf.graph_util.convert_variables_to_constants(
sess,
sess.graph.as_graph_def(),
[n.name for n in tf.get_default_graph().as_graph_def().node])
# https://github.com/onnx/tensorflow-onnx/issues/77#issuecomment-445066091
for node in output_graph_def.node:
print(node.name, node.op)
if node.op == "Assign":
node.op = "Identity"
if 'use_locking' in node.attr: del node.attr['use_locking']
if 'validate_shape' in node.attr: del node.attr['validate_shape']
if len(node.input) == 2:
# input0: ref: Should be from a Variable node. May be uninitialized.
# input1: value: The value to be assigned to the variable.
node.input[0] = node.input[1]
del node.input[1]
output_graph = "model_tf_test.pb"
with tf.gfile.GFile(output_graph, "wb") as f:
f.write(output_graph_def.SerializeToString())
python3 repro.py
- Convert to ONNX with tf2onnx:
pip install onnx==1.6.0 tf2onnx
$ python3 -m tf2onnx.convert --input model_tf_test.pb --inputs Placeholder:0 --outputs conv2d_transpose/BiasAdd:0 --opset=11 --output conv2dtranspose.opset11.onnx
...
2020-02-11 18:51:27,976 - INFO - Using tensorflow=1.14.0, onnx=1.6.0, tf2onnx=1.5.4/ee756a
2020-02-11 18:51:27,976 - INFO - Using opset <onnx, 11>
2020-02-11 18:51:28,014 - INFO - Optimizing ONNX model
2020-02-11 18:51:28,078 - INFO - After optimization: Const -28 (39->11), Gather +1 (0->1), Identity -1 (1->0), Squeeze -1 (3->2), Unsqueeze -2 (4->2)
2020-02-11 18:51:28,081 - INFO -
2020-02-11 18:51:28,081 - INFO - Successfully converted TensorFlow model model_tf_test.pb to ONNX
2020-02-11 18:51:28,082 - INFO - ONNX model is saved at conv2dtranspose.opset11.onnx
- Convert ONNX to TensorRT
$ dpkg -l | grep -i tensorrt
...
ii tensorrt 7.0.0.11-1+cuda10.2 amd64 Meta package of TensorRT
$ trtexec --explicitBatch --onnx=conv2dtranspose.onnx
...
&&&& PASSED TensorRT.trtexec # trtexec --explicitBatch --onnx=conv2dtranspose.opset11.onnx
The above was all using TensorRT 7.0 and ONNX opset 11 on an x86 machine.
I noticed you mentioned Xavier, so I tried to reproduce using TensorRT 6.0 and ONNX opset 9/10.
Seems like TensorRT 6 doesn’t support some of these ops, even with building the OSS ONNX parser release 19.12:
$ dpkg -l | grep -i tensorrt
ii tensorrt 6.0.1.8-1+cuda10.2 amd64 Meta package of TensorRT
$ python3 -m tf2onnx.convert --input model_tf_test.pb --inputs Placeholder:0 --outputs conv2d_transpose/BiasAdd:0 --opset=10 --output=conv2dtranspose.opset10.onnx
$ trtexec --onnx=conv2dtranspose.opset10.onnx
...
WARNING: ONNX model has a newer ir_version (0.0.6) than this parser was built against (0.0.3).
While parsing node number 1 [ConvTranspose]:
ERROR: ModelImporter.cpp:296 In function importModel:
[5] Assertion failed: tensors.count(input_name)
$ bash /opt/tensorrt/install_opensource.sh
...
Done!
$ trtexec --explicitBatch --onnx=conv2dtranspose.opset10.onnx
...
ERROR: /opt/tensorrt/TensorRT/parsers/onnx/builtin_op_importers.cpp:286 In function importCast:
[8] Assertion failed: trt_dtype == nvinfer1::DataType::kHALF && cast_dtype == ::ONNX_NAMESPACE::TensorProto::FLOAT
[01/11/2020-18:58:30] [E] Failed to parse onnx file
[01/11/2020-18:58:30] [E] Parsing model failed
[01/11/2020-18:58:30] [E] Engine could not be created
&&&& FAILED TensorRT.trtexec # trtexec --explicitBatch --onnx=conv2dtranspose.opset9.onnx
Which seems to have been since changed/fixed in TensorRT 7.
TRT6 fails here: onnx-tensorrt/builtin_op_importers.cpp at 397cdbafd898153f69a2b1a87dcc1c4dc5add18b · onnx/onnx-tensorrt · GitHub
TRT7 fixed here: https://github.com/onnx/onnx-tensorrt/blob/84b5be1d6fc03564f2c0dba85a2ee75bad242c2e/builtin_op_importers.cpp#L313
So you might just have to wait for TensorRT >= 7 on Xavier for this, but I can look into if there is a workaround or not.
If you have a model that is not working, please share it so we can investigate what’s unsupported and perhaps make a feature request to tf2onnx if necessary.