Inconsistent output between TensorRT network and Tensorflow output, Kernel order

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

If you open a TensorRT issue, here is our policy.

TensorRT developers respond to issues. We want to focus on fixing bugs and adding features.

Provide details on the platforms you are using:
Linux distro and version: 4.15.0-39-generic #42~16.04.1-Ubuntu SMP Wed Oct 24 17:09:54 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
GPU type: GTX1080Ti
nvidia driver version: 410.73
CUDA version: 9.0
CUDNN version: 7.0
Python version [if using python]:3.6
Tensorflow version: 1.10.1
TensorRT version:5.0
If Jetson, OS, hw versions

Describe the problem
Inconsistent output from TensorRT output. I have to manually added the convolution layer as I have some special operation which is not supported by the Tensorflow parser. What is the order of the weights does the TensorRT expect to read from a .wts file? For example, a conv2d weight tensor of Tensorflow typically has an order of[H,W,IN_CHANNEL,OUT_CHANNEL]. I know that TensorRT expects the input data to be in the NCHW order, but is the order of weights has to be changed too when it’s being writed to .wts file? If so, what is the order TensorRT expected to get? [IN_CHANNEL, OUT_CHANNEL,H,W]?

Files

Include any logs, source, models (uff, pd) that would be helpful to diagnose the problem.

If relevant, please include the full traceback.

Try to provide a minimal reproducible test case.

2

Hello,

how are you writing the weights? What’s the use-case for this? If writing weights from TRT, then there would be no modifications needed for reading them back in.

regards,
NVIDIA Enterprise Support

3

Per engineering, .wts file is defined on a per sample basis and is not an ‘official’ format that has any definition. TensorRT defines in NvInfer.h how the weights are expected to be laid out in memory for a specific layer.