Precision flag issues in Jetpack 6.0 DP & TensorRT 8.6.2

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

I have installed Jetpack 6.0 DP version which comes along with the Tensorrt 8.6.2 in my Jetson AGX Orin.

Then I have converted my transformer based .pth model to .onnx and then to .trt using trtexec and I used all the three flags seperately (INT8, FP16, BEST, FP8) and we are using torch version 2.1.0 throughout of our process.

  • While using “–int8”, we got 53 QPS, and we are also able to evaluate the model.

  • Next, “–fp16”, we got something around 64 QPS. but we are getting error while evaluating the model accuracy.

  • and lastly “best”, we got 82 QPS but facing the same issues while evaluating the model accuracy.

But these same model and same flags were running good in TensorRT 8.5.2 and JP 5.1.2.

Is there anything wrong with “–fp16” and “best” flag in TensorRT 8.6.2 and JP 6.0 DP as only “–int8” is running good?

And also mention the necessary packages along with their versions to install in above scenario.

2

Hi there @csayakcr7 and welcome to the NVIDIA developer forums.

I think the Jetson forums will be better suited for your inquiry, I hope you don’t mind me moving this post over.

Thanks!

4

Hi,

Could you share the error you encountered with us first?

Thanks.

5

While evaluating the script I am getting the following error:

File “/home/Documents/Vision_Transformer/03_eval_trt.py”, line 270, in **
** main()**
** File “/home/Documents/Vision_Transformer/03_eval_trt.py”, line 246, in main**
** pred_rgb, pred_anno, save_name, t = model.run(image_path)**
** File “/home/Documents/Vision_Transformer/03_eval_trt.py”, line 134, in run**
** _pred_labels, _pred_scores, _mask_labels, _mask_probs = compute_segments(**
** File “/home/Documents/Vision_Transformer/lib/models/post_processing.py”, line 44, in compute_segments**
** mask_probs = nn.functional.interpolate(**
** File “/home/.local/lib/python3.10/site-packages/torch/nn/functional.py”, line 4020, in interpolate**
** return torch._C._nn.upsample_bilinear2d(input, output_size, align_corners, scale_factors)**
RuntimeError: Non-empty 4D data tensor expected but got a tensor with sizes [1, 0, 384, 384]

Can we install python 3.8.10 in jetpack 6.0 DP version and Is python 3.8.10 compatible with TensorRT 8.6.2 ? As these same script is running in JP 5.1.2 and TensorRT 8.5.2.2 with python 3.8.10

6

Hi,

You can build TensorRT with Python 3.8 but it will still be v8.6.2.
It looks like the issue comes from TensorRT 8.6 vs 8.5 rather than Python 3.10 vs 3.8.

We want to reproduce this issue internally.
Could you share the detailed steps and source with us?

Thank.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.