Shufflenet_v2_x1_0 on TesorRT7.0, F32 and FP16 Inference results are quite different

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Environment

TensorRT Version: 7.0
GPU Type: Tesla T4
Nvidia Driver Version: 440.33.01
CUDA Version: 10.2
CUDNN Version: 7.6.5.32
Operating System + Version: ubuntu18.04 + 4.15.0-101-generic
Python Version (if applicable): 3.7.3
PyTorch Version (if applicable): 1.2.0

1、 Generate ONNX
import torch
import torchvision.models as models
model = models.shufflenet_v2_x1_0(pretrained=True).cuda()
model.eval()
dummy_input = torch.randn(1,3,224,224).cuda()
input_names = ["input"]
output_names = ["output"]
torch.onnx.export(model, dummy_input, "shufflenet_v2_x1_0.onnx", verbose=False, opset_version=9,input_names=input_names, output_names=output_names)
2、pytorch onnx to tensorrt
    fp32
    trtexec --onnx=shufflenet_v2_x1_0.onnx --saveEngine=32.trt
    fp16
   trtexec --onnx=shufflenet_v2_x1_0.onnx --fp16 --saveEngine=16.trt
3、interfence
   trtexec  --loadEngine=32.trt  --exportOutput=result_32.trt
   trtexec  --loadEngine=16.trt  --exportOutput=result_16.trt
4、compare resutl
    It's not a difference of decimal places
1 Like
2

Issue is fixed and should be available in next TRT release.
Request you to please stay tuned for TRT release announcement.

Thanks

3

Thanks for your reply. When will the next version be released?

4

I don’t have info regarding the exact release dates.
Please stay tuned for TRT announcement.

Thanks

5

thank you