Hi, I have a pretrained torchvision segmentation model that i exported to onnx and then to tensorrt (using trtexec) to use on Deepstream. The model ran successfully and as expected. This was with a FP32 precision. I tried doing the same for FP16, I exported the model from pytorch (after converting…

Inference on FP16 segmentation model

AastaLLL November 3, 2021, 2:53am 3

Hi,

This will apply FP32 → FP16 conversion twice.
Please use the original FP32 ONNX model and quantize it with trtexec (--fp16) directly.

More, do you run Deepstream with the serialized TensorRT engine or the ONNX file?

Thanks.

Topic		Replies	Views
Tensorrt FP16 conversion issue TensorRT tensorrt , cuda , gstreamer , onnx , deep-learning , deepstream	8	2616	March 6, 2023
The fp16 engine cannot be generated normally, a Segmentation fault (core dumped) occurs, and the fp32 is ok DeepStream SDK	2	368	February 26, 2023
Convert the TRT model with FP16 Jetson TX2 jetpack , tensorrt , jetson-inference	7	2527	October 18, 2021
Error during conversion of ONNX panoptic model to TensorRT fp16 TensorRT cudnn	3	276	June 5, 2024
Trtexec --layerPrecision and --precisionConstraints not respected when converting onnx model TensorRT	3	2054	February 20, 2023
Warning While DS App Converts ONNX Model To Engine DeepStream SDK tensorrt	5	902	July 11, 2022
Object Detection Inference Optimisation Jetson Xavier NX jetson-inference	4	650	April 17, 2023
Pose_estimation.onnx_b1_gpu0_fp16.engine open error DeepStream SDK	3	658	October 12, 2021
TensorRT 8.6 conversion of RTDETR custom model accuracy dropped at fp16 TensorRT cudnn	0	68	July 26, 2024
Segmentation fault while converting the model with trt8.0.1.6 TensorRT tensorrt	5	1444	November 21, 2021