TensorRT 4.X support Tensorflow ConcatV2 vs. TensorRT 5.X support Tensorflow ConcatV2

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello,
I have TX2 and Xavier Jetsons boards.

These are the configurations:

  • TX2: Linux distro and version – Ubuntu 16.04.5 LTS (Xenial Xersus) L4T - #R28 (release), REVISION 2.1, GCID: 11272647, BOARD: t186ref, EABI: aarch64, DATE: Thu May 17 07:29:06 UTC 2018 GPU type - As part of the Jetson TX2 developer kit board JetPack – 3.2.1 (But TensorRT and CUDNN were updated according to JetPack 3.3 versions) nvidia driver version - As part of the JetPack CUDA version - Release 9.0, V9.0.252 CUDNN version - 7.1.5 TensorRT version – 4.0.1.6

    The TensorRT uff was generated under the following platform:
    Linux distro and version - Linux-x86_64, Ubuntu, 16.04
    GPU type - GeForce GTX 1080
    nvidia driver version - 396.26
    CUDA version - Release 9.0, V9.0.252
    CUDNN version - 7.1.4
    Python version – 3.5.2
    Tensorflow version – 1.9
    TensorRT version – 4.0.1.6

  • Xavier: Jetpack 4.2 (TensorRT 5.0.6 & CuDNN 7.3.1)

    The TensorRT uff file was created in the Xavier by the Jetpack 4.2 tolls versions

I have a small model which include 4 layers only:

x = tf.reshape(x, [-1, c, h, w, 1])

x = tf.concat([x, x], axis=3)

x = tf.concat([x, x], axis=4)

x = tf.reshape(x, (-1, c, h * factor, w * factor))

In the following Topic:
https://devtalk.nvidia.com/default/topic/1042300/tensorrt/tensorrt-doesn-t-perform-properly-the-tensorflow-concat-and-or-reshape-commands/

In the last comment I described that ConcatV2 of version 4.0.1.6 properly works only when the concat axis is the channels axis which means 1, otherwise the results will be wrong despite the fact that no error raised. Only usage of PlugIn layer can solve the problem.

But when I’m running the same model (uff file) in my new Xavier board without any TensorRT PlugIn layer (as described in the link) it seems that the TensorRT concat is working properly.

I tried to verify in the following link if there was any update with the ConcatV2 support:
https://developer.download.nvidia.com/compute/machine-learning/tensorrt/docs/5.1/rc/TensorRT-Developer-Guide.pdf

And I found this section: A.1.2. IConcatenationLayer (Which isn’t include in the 4.0.1.6 developer guide document)

But after reading it I couldn’t be sure that I’m understand if there was any update related to this Tensorflow layer.

Please clarify

Thanks