3D conv result different in PyTorch and TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I am trying to convert a torch model to trt engine file. My torch model contains lots of 3d conv layers.

  • My torch model works well. i convert it to onnx model which also works well in onnxruntime.
  • i convert the .onnx to .trt by trtexec(provided by TensorRT SDK), the engine can work, but the output is wrong.
  • i convert the .onnx to .trt by onnx2trt(provided by GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX),
    the engine can work, but the output is wrong.

why 3d conv result is wrong in trt engine? How can i solved this problem? Thanks.

In fact, my problem is the same as the issue: Pytorch and TensorRT 3D Conv different result. · Issue #486 · NVIDIA/TensorRT · GitHub, but it is still not solved.

Environment

TensorRT Version: 7.1.3.4
GPU Type: RTX2060
Nvidia Driver Version: 440.100
CUDA Version: 10.2
CUDNN Version: 8.0.1
Operating System + Version: Ubuntu 18.04.3 LTS
Python Version (if applicable): Python 3.6.9
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.4.0
Baremetal or Container (if container which image + tag):

Relevant Files

The onnx model is below:

This model works well in onnxruntime.

Steps To Reproduce

i use the following code to convert onnx to trt:

./trtexec --onnx=dump.simple.onnx --saveEngine=dump.simple.trt

log:

----------------------------------------------------------------
Input filename:   dump.simple.onnx
ONNX IR version:  0.0.4
Opset version:    11
Producer name:    pytorch
Producer version: 1.3
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[07/23/2020-21:46:48] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/23/2020-21:46:48] [W] [TRT] Setting layouts of network and plugin input/output tensors to linear, as 3D operators are found and 3D non-linear IO formats are not supported, yet.
[07/23/2020-21:46:49] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.

i noticed there is a warning:

Setting layouts of network and plugin input/output tensors to linear, as 3D operators are found and 3D non-linear IO formats are not supported, yet.

maybe this is the reason?

2

Hi @616914658,
Can you help us with the output files, and verbose logs?
Thanks!

3

Thanks for the quick reply. I solved the problem by using tensorrt plugins.

1 Like