Hello, I would like the ask a question on TensorCores of Nvidia Jetson AGX Developer Kit. I made an int8 quantization with the pytorch_quantization library and convert the calibrated .pt model to an onnx format and after that to an .engine. I hoped that this would fuse the fake-quantization layer I…

Hi, This depends on the layer used in the model. It’s recommended to enable the log with --verbose to see the layer placement in detail. Based on our doc here: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#reduced-precision There are three precision flags: FP16, INT8, …

Hello, I tried to run the trtexec command on the onnx model. To be more precise. I ran the trtexec --onnx --int8 command on a int8 calibrated onnx model and the trtexec --onnx --fp16 on a fp16 trained onnx model. TensorRT failed to run the int8 version and passed the fp16 test. Do you have any idea…

Hi, Could you try the command on our latest TensorRT 8.5 release (JetPack 5.1)? Thanks.

Hi, I updated the jetson Xavier Developer Kit to the newest available version 5.1-b147. This JetPack Version uses TensorRT 8.5.2.2. I tried the command above and they worked out just fine. I so, I got an other problem. In the same area. I used the following command to create an engines from pytor…

@Patrick_1234 You may also be interested in looking into the deep learning accelerator DLA

Int8 TensorCores for Jetson

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier

AastaLLL February 13, 2023, 2:33am 3

Hi,

Xavier has Tensor Core.
Could you try the model with trtexec first?

$ /usr/src/tensorrt/bin/trtexec --onnx=[model] --fp16
$ /usr/src/tensorrt/bin/trtexec --onnx=[model] --int8

Thanks.

Topic		Replies	Views
How to verify if QAT TRT engine is indeed INT8 on Xavier Jetson AGX Xavier tensorrt	16	779	October 5, 2022
TensorRT generated QAT engine, why the engine is bigger than pretrained fp16 engine? TensorRT	3	1407	January 4, 2022
Jetson Thor AGX - Poor INT8 performance Jetson Thor tensorrt , jetson-inference	6	222	April 1, 2026
Post-Training Quantization (PTQ) for semantic segmentation model running on Jetson Orin NX Jetson Orin NX tensorrt	24	692	March 26, 2025
Trtexec conversion to int8 Jetson AGX Orin tensorrt	2	1373	October 11, 2023
Inference Speed Jetson Xavier NX pytorch	6	1069	April 12, 2023
[Hugging Face transformer models + pytorch_quantization] PTQ quantization int8 is slower than fp16 TensorRT tensorrt , python , onnx , natural-language-processing-nlp	4	3152	January 6, 2022
Int8 is not faster than fp16 on xavier Jetson AGX Xavier tensorrt	5	867	October 18, 2021
NX & TRT & Jetson-inference - Not setting precision to INT8 Jetson Xavier NX tensorrt , jetson-inference	4	967	October 18, 2021
Calibration failed: INTERNAL: Failed to build TensorRT engine (INT8 precision mode) in Jetson Xavier NX (16GB) Jetson Xavier NX tensorrt	9	903	April 12, 2023

Int8 TensorCores for Jetson

Related topics