Hi, I am attempting to realize ‘Split’ operation by tensorrt plugin in C++, but there is something wrong with uffparser.
My tensorflow model split.pb is generated as bellow:
#!/usr/bin/env python3
from tensorflow.python.framework import graph_util
import tensorflow as tf
image = tf.placeholder(tf.float32, [1, 5, 5, 3], name="input_image")
red, green,blue = tf.split(image, num_or_size_splits=3, axis=3)
pred = tf.concat([red, green, blue],3)
pred = tf.identity(pred, name = "pred")
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
output_graph_def = graph_util.convert_variables_to_constants( sess, sess.graph_def, output_node_names=['pred'])
with tf.gfile.GFile("./split.pb", "wb") as f:
f.write(output_graph_def.SerializeToString())
There are 3 ouput tensors of split layer, the dimension of each tensor is HWC(5,5,1).
int Split::getNbOutputs() const
{
return 3;
}
Dims Split::getOutputDimensions(int index, const Dims* inputs, int nbInputDims)
{
return Dims3(5,5,1);
}
Then, tf.concat([red, green, blue],3), the dimension should be (5,5,3). But the output dimension of tf.concat is (5,5,9).
Begin parsing model...
UNKNOWN: UFFParser: Parsing split/split_dim[Op: Const].
UNKNOWN: UFFParser: split/split_dim -> []
UNKNOWN: UFFParser: Applying order forwarding to: split/split_dim
UNKNOWN: UFFParser: Parsing input_image[Op: Input].
UNKNOWN: UFFParser: input_image -> [3,5,5]
UNKNOWN: UFFParser: Applying order forwarding to: input_image
UNKNOWN: UFFParser: Parsing split[Op: _Split]. Inputs: split/split_dim, input_image
UNKNOWN: UFFParser: Parse Plugin node split
UNKNOWN: UFFParser: PluginNode Input Descriptor input_image
UNKNOWN: UFFParser: Operation of node Input
UNKNOWN: UFFParser: Parsing concat[Op: Concat]. Inputs: split, split, split
UNKNOWN: UFFParser: concat -> [5,5,9]
UNKNOWN: UFFParser: Applying order forwarding to: concat
UNKNOWN: UFFParser: Parsing pred[Op: Identity]. Inputs: concat
UNKNOWN: UFFParser: pred -> [5,5,9]
UNKNOWN: UFFParser: Applying order forwarding to: pred
UNKNOWN: UFFParser: Parsing MarkOutput_0[Op: MarkOutput]. Inputs: pred
UNKNOWN: UFFParser: Applying order forwarding to: MarkOutput_0
End parsing model...
Begin building engine...
UNKNOWN: Applying generic optimizations to the graph for inference.
UNKNOWN: Original: 4 layers
UNKNOWN: After dead-layer removal: 4 layers
UNKNOWN: After scale fusion: 4 layers
UNKNOWN: After vertical fusions: 4 layers
UNKNOWN: After final dead-layer removal: 4 layers
UNKNOWN: After tensor merging: 4 layers
UNKNOWN: Eliminating concatenation concat
UNKNOWN: Generating copy for split to concat
UNKNOWN: Generating copy for split_1 to concat
UNKNOWN: Generating copy for split_2 to concat
UNKNOWN: Generating copy for split to concat
UNKNOWN: Generating copy for split_1 to concat
UNKNOWN: Generating copy for split_2 to concat
UNKNOWN: Generating copy for split to concat
UNKNOWN: Generating copy for split_1 to concat
UNKNOWN: Generating copy for split_2 to concat
UNKNOWN: After concat removal: 12 layers
UNKNOWN: Graph construction and optimization completed in 0.000289446 seconds.
UNKNOWN: Constructing optimization profile number 0 out of 1
Using kFLOAT for region surrounded by copy operations: concat
UNKNOWN: *************** Autotuning format combination: -> Float(1) ***************
UNKNOWN: *************** Autotuning format combination: Float(1), Float(1,5,25,75) -> Float(1,1,5,25), Float(1,1,5,25), Float(1,1,5,25) ***************
UNKNOWN: --------------- Timing Runner: split copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.006112
UNKNOWN: Tactic: 0 time 0.005856
UNKNOWN: Fastest Tactic: 0 Time: 0.005856
UNKNOWN: --------------- Timing Runner: split_1 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00592
UNKNOWN: Tactic: 0 time 0.005856
UNKNOWN: Fastest Tactic: 0 Time: 0.005856
UNKNOWN: --------------- Timing Runner: split_2 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.005984
UNKNOWN: Tactic: 0 time 0.005792
UNKNOWN: Fastest Tactic: 0 Time: 0.005792
UNKNOWN: --------------- Timing Runner: split copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00608
UNKNOWN: Tactic: 0 time 0.005056
UNKNOWN: Fastest Tactic: 0 Time: 0.005056
UNKNOWN: --------------- Timing Runner: split_1 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00592
UNKNOWN: Tactic: 0 time 0.005952
UNKNOWN: Fastest Tactic: 1002 Time: 0.00592
UNKNOWN: --------------- Timing Runner: split_2 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.005792
UNKNOWN: Tactic: 0 time 0.00592
UNKNOWN: Fastest Tactic: 1002 Time: 0.005792
UNKNOWN: --------------- Timing Runner: split copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00608
UNKNOWN: Tactic: 0 time 0.005824
UNKNOWN: Fastest Tactic: 0 Time: 0.005824
UNKNOWN: --------------- Timing Runner: split_1 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00608
UNKNOWN: Tactic: 0 time 0.00512
UNKNOWN: Fastest Tactic: 0 Time: 0.00512
UNKNOWN: --------------- Timing Runner: split_2 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.005952
UNKNOWN: Tactic: 0 time 0.005888
UNKNOWN: Fastest Tactic: 0 Time: 0.005888
UNKNOWN: --------------- Timing Runner: pred (Reformat)
UNKNOWN: Tactic: 1002 time 0.006144
UNKNOWN: Tactic: 0 time 0.004288
UNKNOWN: Fastest Tactic: 0 Time: 0.004288
UNKNOWN: Formats and tactics selection completed in 0.0496778 seconds.
UNKNOWN: After reformat layers: 12 layers
UNKNOWN: Block size 16777216
UNKNOWN: Block size 1024
UNKNOWN: Block size 512
UNKNOWN: Block size 512
UNKNOWN: Block size 512
UNKNOWN: Total Activation Memory: 16779776
configure
initialize
UNKNOWN: Engine generation completed in 1.03445 seconds.
UNKNOWN: Engine Layer Information:
UNKNOWN: Layer: split/split_dim (Constant), Tactic: 0, -> split/split_dim[Float()]
UNKNOWN: Layer: _Split_split (Plugin), Tactic: 0, split/split_dim[Float()], input_image[Float(3,5,5)] -> split[Float(5,5,1)], split_1[Float(5,5,1)], split_2[Float(5,5,1)]
UNKNOWN: Layer: split copy (Reformat), Tactic: 0, split[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_1 copy (Reformat), Tactic: 0, split_1[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_2 copy (Reformat), Tactic: 0, split_2[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split copy (Reformat), Tactic: 0, split[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_1 copy (Reformat), Tactic: 1002, split_1[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_2 copy (Reformat), Tactic: 1002, split_2[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split copy (Reformat), Tactic: 0, split[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_1 copy (Reformat), Tactic: 0, split_1[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_2 copy (Reformat), Tactic: 0, split_2[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: pred (Reformat), Tactic: 0, concat[Float(5,5,9)] -> pred[Float(5,5,9)]
End building engine...
The concat layer fetch three times data of split’s all 3 output tensors. It seems that the concat layer after split can’t get right tensor by output’s index.
Could someone figure out where is the problem and do me a favor? Thanks everyone for your time and assistance.
ps. let getOutputDimensions() as bellow, the dimension of concat is (1,5,45)
Dims Split::getOutputDimensions(int index, const Dims* inputs, int nbInputDims)
{
return DimsCHW(1, 5, 5);
}