Softmax layer in Tensorrt7.0 has wrong inference results

NVES_R · February 27, 2020, 2:14am

Hi,

Can you share a small script (preferably python, but C++ is fine) to do inference on your sample TRT engine, and compare the outputs of your TRT6 and TRT7 engines?

I’m seeing the same outputs from TF, TRT6, and TRT7 given the same input with an internal tool, for both TF → UFF → TRT and TF → ONNX → TRT.

As a side note, the UFF parser will be deprecated in the future per the TRT7 release notes, so ONNX parser gets more support / new features / etc. at the moment. You could convert your sample model above to ONNX with tf2onnx like so:

python3 -m pip install tf2onnx
python3 -m tf2onnx.convert --input model.pb --inputs "input_image:0" --outputs "pred:0" --output "model.onnx"

Topic		Replies	Views
TensorRT 8 : C++ inference gives different results compared to tensorflow python inference TensorRT	7	1356	October 5, 2021
Incorrect inference results after converting from ONNX to TRT with trtexec TensorRT tensorrt , python , onnx	4	1573	December 9, 2022
TensorRT Engine gives incorrect inference output for segmentation model TensorRT	6	1318	October 12, 2021
TensorRT python API inference is inconsistent with trtexec inference TensorRT tensorrt	1	995	February 28, 2023
Problem converting TensorFlow 2-> ONNX model to TensorRT Engine (efficientdet_d0) TensorRT	8	1394	November 17, 2022
UFF deprecation - Are there good alternatives for the TensorFlow workflow? TensorRT	5	1465	February 13, 2020
Uff to trt8 TensorRT tensorrt	6	867	January 19, 2022
TF-TRT graph conversion failed for Tensorflow version 1 TensorRT tensorrt , tensorflow , ubuntu , python , tf-trt	1	823	May 17, 2022
Efficientdet-d0 to tensorrt problem TensorRT tensorflow , cudnn	8	519	April 5, 2024
TensorRT model inference result is not correctly TensorRT tensorrt , tensorflow , onnx	1	651	July 1, 2022

Softmax layer in Tensorrt7.0 has wrong inference results

Related topics