Description
I was working with TensorRT and met some probelms about parsing Tensorflow network to TensorRT model.
I use expand_dims
but get error shape in TensorRT.
The details is as below.
Environment
1. GPU 2070 super
2. Driver 440.31
3. CUDA 10.0.130
4. CUDNN 7.6.5
5. Python 3.6
6. Tensorflow 1.13.1
7. TensorRT 6.0.1.5
8. OS Ubuntu18.04
9. TensorRT 6.0
Flow code
#
# Model Code
#
sess.run(tf.initialize_all_variables())
frozen_graph_def = tf.graph_util.convert_variables_to_constants(
sess,
tf.get_default_graph().as_graph_def(),
output_node_names = ["output"]
)
dyn_graph = gs.DynamicGraph(frozen_graph_def)
uff_model = uff.from_tensorflow(
dyn_graph.as_graph_def(),
output_nodes=["output"],
output_filename="./checkpoint/model_test.uff",
txt=True
)
input_name = "input"
input_shape = (1, 32, 32)
output_name = "output"
model_file = "./checkpoint/model_test.uff"
logger = trt.Logger(trt.Logger.VERBOSE)
builder = trt.Builder(logger)
network = builder.create_network()
parser = trt.UffParser()
builder.max_workspace_size = (1 << 30)
# Parse the Uff Network
parser.register_input(input_name, input_shape)
parser.register_output(output_name)
parser.parse(model_file, network)
 NCHW

model
input_chw = tf.placeholder(tf.float32, [None, 1, 32, 32], name="input") b = tf.constant(np.zeros([32, 32], np.float32), name="b") b = tf.expand_dims(b, dim=0, name="expand_dim") input_chw = input_chw + b input_hwc = tf.transpose(input_chw, [0, 2, 3, 1]) conv1_weight = tf.get_variable("conv1_weight", shape=[3, 3, 1, 16]) conv1_bias = tf.get_variable("conv1_bias", shape=[16]) conv1 = tf.nn.conv2d(input_hwc, conv1_weight, strides=[1, 1, 1, 1], padding="SAME") conv1 = tf.add(conv1, conv1_bias, name="output")

result
[TensorRT] VERBOSE: UFFParser: Parsing input[Op: Input]. [TensorRT] VERBOSE: UFFParser: input > [1,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: input [TensorRT] VERBOSE: UFFParser: Parsing b[Op: Const]. [TensorRT] VERBOSE: UFFParser: b > [32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: b [TensorRT] VERBOSE: UFFParser: Parsing expand_dim[Op: ExpandDims]. Inputs: b [TensorRT] VERBOSE: UFFParser: expand_dim > [1,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: expand_dim [TensorRT] VERBOSE: UFFParser: Parsing add[Op: Binary]. Inputs: input, expand_dim [TensorRT] VERBOSE: UFFParser: add > [1,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: add [TensorRT] VERBOSE: UFFParser: Parsing transpose[Op: Transpose]. Inputs: add [TensorRT] VERBOSE: UFFParser: transpose > [1,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: transpose [TensorRT] VERBOSE: UFFParser: Parsing conv1_weight[Op: Const]. [TensorRT] VERBOSE: UFFParser: conv1_weight > [3,3,1,16] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: conv1_weight [TensorRT] VERBOSE: UFFParser: Parsing Conv2D[Op: Conv]. Inputs: transpose, conv1_weight [TensorRT] VERBOSE: UFFParser: Conv2D > [16,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: Conv2D [TensorRT] VERBOSE: UFFParser: Parsing conv1_bias[Op: Const]. [TensorRT] VERBOSE: UFFParser: conv1_bias > [16] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: conv1_bias [TensorRT] VERBOSE: UFFParser: Parsing output[Op: Binary]. Inputs: Conv2D, conv1_bias [TensorRT] VERBOSE: UFFParser: output > [16,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: output [TensorRT] VERBOSE: UFFParser: Parsing MarkOutput_0[Op: MarkOutput]. Inputs: output [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: MarkOutput_0
 HWC Format

model
input_chw = tf.placeholder(tf.float32, [None, 1, 32, 32], name="input") input_hwc = tf.transpose(input_chw, [0, 2, 3, 1]) b = tf.constant(np.zeros([32, 32], np.float32), name="b") b = tf.expand_dims(b, dim=2, name="expand_dim") input_hwc = input_hwc + b # input_hwc = tf.transpose(input_chw, [0, 2, 3, 1]) conv1_weight = tf.get_variable("conv1_weight", shape=[3, 3, 1, 16]) conv1_bias = tf.get_variable("conv1_bias", shape=[16]) conv1 = tf.nn.conv2d(input_hwc, conv1_weight, strides=[1, 1, 1, 1], padding="SAME") conv1 = tf.add(conv1, conv1_bias, name="output")

result
[TensorRT] VERBOSE: UFFParser: Parsing input[Op: Input]. [TensorRT] VERBOSE: UFFParser: input > [1,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: input [TensorRT] VERBOSE: UFFParser: Parsing transpose[Op: Transpose]. Inputs: input [TensorRT] VERBOSE: UFFParser: transpose > [1,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: transpose [TensorRT] VERBOSE: UFFParser: Parsing b[Op: Const]. [TensorRT] VERBOSE: UFFParser: b > [32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: b [TensorRT] VERBOSE: UFFParser: Parsing expand_dim[Op: ExpandDims]. Inputs: b [TensorRT] VERBOSE: UFFParser: expand_dim > [32,1,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: expand_dim [TensorRT] VERBOSE: UFFParser: Parsing add[Op: Binary]. Inputs: transpose, expand_dim [TensorRT] VERBOSE: UFFParser: add > [32,32,32] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: add [TensorRT] VERBOSE: UFFParser: Parsing conv1_weight[Op: Const]. [TensorRT] VERBOSE: UFFParser: conv1_weight > [3,3,1,16] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: conv1_weight [TensorRT] VERBOSE: UFFParser: Parsing Conv2D[Op: Conv]. Inputs: add, conv1_weight [TensorRT] ERROR: Conv2D: kernel weights has count 144 but 4608 was expected [TensorRT] ERROR: Conv2D: count of 144 weights in kernel, but kernel dimensions (3,3) with 32 input channels, 16 output channels and 1 groups were specified. Expected Weights count is 32 * 3*3 * 16 / 1 = 4608 [TensorRT] VERBOSE: UFFParser: Conv2D > [] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: Conv2D [TensorRT] VERBOSE: UFFParser: Parsing conv1_bias[Op: Const]. [TensorRT] VERBOSE: UFFParser: conv1_bias > [16] [TensorRT] VERBOSE: UFFParser: Applying order forwarding to: conv1_bias [TensorRT] VERBOSE: UFFParser: Parsing output[Op: Binary]. Inputs: Conv2D, conv1_bias [TensorRT] ERROR: UffParser: Parser error: output: The input to the Scale Layer is required to have a minimum of 3 dimensions.
 Question

Why
input_hwc = tf.transpose(input_chw, [0, 2, 3, 1])
seems no influence as below,[TensorRT] VERBOSE: UFFParser: Parsing input[Op: Input].
[TensorRT] VERBOSE: UFFParser: input > [1,32,32]
[TensorRT] VERBOSE: UFFParser: Applying order forwarding to: input
[TensorRT] VERBOSE: UFFParser: Parsing transpose[Op: Transpose]. Inputs: input
[TensorRT] VERBOSE: UFFParser: transpose > [1,32,32] 
Why with
nhwc
format, theb = tf.expand_dims(b, dim=2, name="expand_dim")
will lead to error output as below.[TensorRT] VERBOSE: UFFParser: Parsing b[Op: Const].
[TensorRT] VERBOSE: UFFParser: b > [32,32]
[TensorRT] VERBOSE: UFFParser: Applying order forwarding to: b
[TensorRT] VERBOSE: UFFParser: Parsing expand_dim[Op: ExpandDims]. Inputs: b
[TensorRT] VERBOSE: UFFParser: expand_dim > [32,1,32]
