Two build results of the same model are different

Description

The network including deformable convolution plugin was builded to trt-engine many times, but only one time the build result is right and not NAN. The different build process logs are given below. It seems like the layer fusion approaches are different, and the right approach happens by chance.

Environment

TensorRT Version: 7.1.3.4
GPU Type: 2080
Nvidia Driver Version: 440.33.01
CUDA Version: 10.2
CUDNN Version: 8.0.2
Operating System + Version: ubuntu 16.04
PyTorch Version (if applicable): 1.3

Relevant Files

log_failed.txt (1.9 MB) log_successful.txt (1.9 MB)

Hi,
Could you please share the model and script files to reproduce the issue so we can help better.

Thanks

The onnx model file is given below:
website:百度网盘-链接不存在
password:w73d
The plugin just like tensorRTIntegrate/DCNv2.cu at master · dlunion/tensorRTIntegrate · GitHub.
Thank you.

Hi @654818923,
Can you please share the onnx file in attachment.
Looks like the link has expired.
Thanks!

The onnx file is too large to be attached.
Can you give me an email to send the file?
Or you can download the file from the website 百度网盘 请输入提取码, the password is X761.
Thanks.

Hi @654818923,
Can you please DM the file,
Thanks!

You can download the model from the above website.
Thanks!

Are you implementing custom plugin?
Can you please share your code and steps to reproduce?

Thanks!

The plugin just like tensorRTIntegrate/DCNv2.cu at master · dlunion/tensorRTIntegrate · GitHub.

Hi @654818923 ,
Apologies for delayed response.
Please try using trtexec command with your onnx model and share the verbose results with us.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

Thanks!

The verbose log just like:
log_failed.txt (1.9 MB) log_successful.txt (1.9 MB)

Hi @654818923 ,
Apologies for the miss. Are you still facing the issue?

Hi, the model is not in use, so I haven’t try the transformation in the newest tensorrt enviroment.
Thanks.