Tensorrt has wrong uffparser in split plugin

Hi, I am attempting to realize ‘Split’ operation by tensorrt plugin in C++, but there is something wrong with uffparser.

My tensorflow model split.pb is generated as bellow:

#!/usr/bin/env python3
from tensorflow.python.framework import graph_util
import tensorflow as tf

image = tf.placeholder(tf.float32, [1, 5, 5, 3], name="input_image")
red, green,blue = tf.split(image, num_or_size_splits=3, axis=3)
pred = tf.concat([red, green, blue],3)
pred = tf.identity(pred, name = "pred")

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

output_graph_def = graph_util.convert_variables_to_constants( sess, sess.graph_def, output_node_names=['pred'])

with tf.gfile.GFile("./split.pb", "wb") as f:
 f.write(output_graph_def.SerializeToString())

There are 3 ouput tensors of split layer, the dimension of each tensor is HWC(5,5,1).

int Split::getNbOutputs() const
{ 
    return 3;
}
Dims Split::getOutputDimensions(int index, const Dims* inputs, int nbInputDims)
{
    return Dims3(5,5,1);
}

Then, tf.concat([red, green, blue],3), the dimension should be (5,5,3). But the output dimension of tf.concat is (5,5,9).

Begin parsing model...
UNKNOWN: UFFParser: Parsing split/split_dim[Op: Const].
UNKNOWN: UFFParser: split/split_dim -> []
UNKNOWN: UFFParser: Applying order forwarding to: split/split_dim
UNKNOWN: UFFParser: Parsing input_image[Op: Input].
UNKNOWN: UFFParser: input_image -> [3,5,5]
UNKNOWN: UFFParser: Applying order forwarding to: input_image
UNKNOWN: UFFParser: Parsing split[Op: _Split]. Inputs: split/split_dim, input_image
UNKNOWN: UFFParser: Parse Plugin node split
UNKNOWN: UFFParser: PluginNode Input Descriptor input_image
UNKNOWN: UFFParser: Operation of node Input
UNKNOWN: UFFParser: Parsing concat[Op: Concat]. Inputs: split, split, split
UNKNOWN: UFFParser: concat -> [5,5,9]
UNKNOWN: UFFParser: Applying order forwarding to: concat
UNKNOWN: UFFParser: Parsing pred[Op: Identity]. Inputs: concat
UNKNOWN: UFFParser: pred -> [5,5,9]
UNKNOWN: UFFParser: Applying order forwarding to: pred
UNKNOWN: UFFParser: Parsing MarkOutput_0[Op: MarkOutput]. Inputs: pred
UNKNOWN: UFFParser: Applying order forwarding to: MarkOutput_0
End parsing model...
Begin building engine...
UNKNOWN: Applying generic optimizations to the graph for inference.
UNKNOWN: Original: 4 layers
UNKNOWN: After dead-layer removal: 4 layers
UNKNOWN: After scale fusion: 4 layers
UNKNOWN: After vertical fusions: 4 layers
UNKNOWN: After final dead-layer removal: 4 layers
UNKNOWN: After tensor merging: 4 layers
UNKNOWN: Eliminating concatenation concat
UNKNOWN: Generating copy for split to concat
UNKNOWN: Generating copy for split_1 to concat
UNKNOWN: Generating copy for split_2 to concat
UNKNOWN: Generating copy for split to concat
UNKNOWN: Generating copy for split_1 to concat
UNKNOWN: Generating copy for split_2 to concat
UNKNOWN: Generating copy for split to concat
UNKNOWN: Generating copy for split_1 to concat
UNKNOWN: Generating copy for split_2 to concat
UNKNOWN: After concat removal: 12 layers
UNKNOWN: Graph construction and optimization completed in 0.000289446 seconds.
UNKNOWN: Constructing optimization profile number 0 out of 1
Using kFLOAT for region surrounded by copy operations: concat
UNKNOWN: *************** Autotuning format combination:  -> Float(1) ***************
UNKNOWN: *************** Autotuning format combination: Float(1), Float(1,5,25,75) -> Float(1,1,5,25), Float(1,1,5,25), Float(1,1,5,25) ***************
UNKNOWN: --------------- Timing Runner: split copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.006112
UNKNOWN: Tactic: 0 time 0.005856
UNKNOWN: Fastest Tactic: 0 Time: 0.005856
UNKNOWN: --------------- Timing Runner: split_1 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00592
UNKNOWN: Tactic: 0 time 0.005856
UNKNOWN: Fastest Tactic: 0 Time: 0.005856
UNKNOWN: --------------- Timing Runner: split_2 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.005984
UNKNOWN: Tactic: 0 time 0.005792
UNKNOWN: Fastest Tactic: 0 Time: 0.005792
UNKNOWN: --------------- Timing Runner: split copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00608
UNKNOWN: Tactic: 0 time 0.005056
UNKNOWN: Fastest Tactic: 0 Time: 0.005056
UNKNOWN: --------------- Timing Runner: split_1 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00592
UNKNOWN: Tactic: 0 time 0.005952
UNKNOWN: Fastest Tactic: 1002 Time: 0.00592
UNKNOWN: --------------- Timing Runner: split_2 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.005792
UNKNOWN: Tactic: 0 time 0.00592
UNKNOWN: Fastest Tactic: 1002 Time: 0.005792
UNKNOWN: --------------- Timing Runner: split copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00608
UNKNOWN: Tactic: 0 time 0.005824
UNKNOWN: Fastest Tactic: 0 Time: 0.005824
UNKNOWN: --------------- Timing Runner: split_1 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.00608
UNKNOWN: Tactic: 0 time 0.00512
UNKNOWN: Fastest Tactic: 0 Time: 0.00512
UNKNOWN: --------------- Timing Runner: split_2 copy (Reformat)
UNKNOWN: Tactic: 1002 time 0.005952
UNKNOWN: Tactic: 0 time 0.005888
UNKNOWN: Fastest Tactic: 0 Time: 0.005888
UNKNOWN: --------------- Timing Runner: pred (Reformat)
UNKNOWN: Tactic: 1002 time 0.006144
UNKNOWN: Tactic: 0 time 0.004288
UNKNOWN: Fastest Tactic: 0 Time: 0.004288
UNKNOWN: Formats and tactics selection completed in 0.0496778 seconds.
UNKNOWN: After reformat layers: 12 layers
UNKNOWN: Block size 16777216
UNKNOWN: Block size 1024
UNKNOWN: Block size 512
UNKNOWN: Block size 512
UNKNOWN: Block size 512
UNKNOWN: Total Activation Memory: 16779776
configure
initialize
UNKNOWN: Engine generation completed in 1.03445 seconds.
UNKNOWN: Engine Layer Information:
UNKNOWN: Layer: split/split_dim (Constant), Tactic: 0,  -> split/split_dim[Float()]
UNKNOWN: Layer: _Split_split (Plugin), Tactic: 0, split/split_dim[Float()], input_image[Float(3,5,5)] -> split[Float(5,5,1)], split_1[Float(5,5,1)], split_2[Float(5,5,1)]
UNKNOWN: Layer: split copy (Reformat), Tactic: 0, split[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_1 copy (Reformat), Tactic: 0, split_1[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_2 copy (Reformat), Tactic: 0, split_2[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split copy (Reformat), Tactic: 0, split[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_1 copy (Reformat), Tactic: 1002, split_1[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_2 copy (Reformat), Tactic: 1002, split_2[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split copy (Reformat), Tactic: 0, split[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_1 copy (Reformat), Tactic: 0, split_1[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: split_2 copy (Reformat), Tactic: 0, split_2[Float(5,5,1)] -> concat[Float(5,5,1)]
UNKNOWN: Layer: pred (Reformat), Tactic: 0, concat[Float(5,5,9)] -> pred[Float(5,5,9)]
End building engine...

The concat layer fetch three times data of split’s all 3 output tensors. It seems that the concat layer after split can’t get right tensor by output’s index.

Could someone figure out where is the problem and do me a favor? Thanks everyone for your time and assistance.

ps. let getOutputDimensions() as bellow, the dimension of concat is (1,5,45)
Dims Split::getOutputDimensions(int index, const Dims* inputs, int nbInputDims)
{
return DimsCHW(1, 5, 5);
}

Hi,
Can you provide the following information so we can better help?
Provide details on the platforms you are using:
o Linux distro and version
o GPU type
o Nvidia driver version
o CUDA version
o CUDNN version
o Python version [if using python]
o Tensorflow version
o TensorRT version
o If Jetson, OS, hw versions

Also, if possible please share the script and model file to reproduce the issue.

Thanks

Hi, I am so happy that you noticed my problem. I’m having trouble with it.
My environment is as bellow:
o Linux distro and version : ubuntu 16.0 LTS
o GPU type :GeForce GTX 1660
o Nvidia driver version :440.36
o CUDA version :10.2
o CUDNN version: libcudnn7-dev_7.6.5.32-1
o Python version [if using python]: python2.7
o Tensorflow version tensorflow1.15
o TensorRT version: TensorRT-7.0.0.11

The Python script, model file and split plugin source code in the baiduyun(the link is available in 7 days):
link: https://pan.baidu.com/s/1d0HgaTgNmgZA_XTvbyIj_g
password: bfe2

I am looking forward to your early reply.

Hi,

I am not able to access the provided link, could you please upload it as zip file in the forum itself?

Thanks

Hi, Thanks for your reply.

The python script to generate model and split.pb model are as follows:
splitPlugin.zip (10.6 KB)

Hi,

I would recommend you to update your implementation to IPluginV2Ext or IPluginV2DynamicExt and see if you still notice the issue with the new implementation.

Also, we are deprecating Caffe Parser and UFF Parser in TensorRT 7. So recommend you use ONNX parser, it already have “Split” op support.

Thanks