TRT engine successful built on JetPack 5.0.1(trt 8.4.1) but not on JetPack 5.1.2(TensorRT 8.5.2)

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
,
1

Description

I have a model that I’m trying to convert to an Int8, DLA TensorRT engine. I was able to successful convert the model on JetPack 5.0.1 but on JetPack 5.1.2, I’m facing few issues.

  1. I’m able to generate FP16, FP16 with DLA support and INT8 engine when NOT providing the calibration cache file (–cache option in trtexec)
  2. When trying to make an INT8 engine with DLA support and NOT giving the calibration cache file, the process got stuck(it remained stuck even after 24hrs). I tried couple of different workspace sizes ranging from 150MB to 3500MB but the issue remained.
  3. When trying to make an INT8 engine with DLA support and giving the calibration cache file, I get Error[2]: [weightConvertors.cpp::quantizeBiasCommon::337] Error Code 2: Internal Error (Assertion getter(i) != 0 failed. ).

Could you please help me in converting this model. I’m not sure how the calibration cache file works. Would replacing the 0s with something like 837: 00000001 or 01010101 work, or would that have too much impact on accuracy?

Thanks in advance!

NOTE: The code to generate the calibration file has has been used since JetPack 4.6.0.

Environment

TensorRT Version: 8.5.2
Device: Nvidia Jetson Xavier NX
JetPack: 5.1.2
CUDA Version: 11.4
Operating System + Version: 20.04 Ubuntu

Relevant Files

Model onnx:

Log file for issue (2):
trt_int8_dla.txt (531.8 KB)
Calibration cache file generated on JetPack 5.1.2:
engine_cache_jp512.cache (1.7 KB)
Calibration cache file generated on JetPack 5.0.1(and working):
engine_cache_jp501.cache (1.7 KB)

Steps To Reproduce

/usr/src/tensorrt/bin/trtexec --onnx=model.onnx --fp16 --saveEngine=engine_int8_dla.trt --workspace=150/1500/3500 --verbose [–int8] [–cache=engine_cache.cache] [–useDLACore=0 --allowGPUFallback]

2

Hi,

This looks like a Jetson issue. Please refer to the below samples in case useful.

For any further assistance, we will move this post to to Jetson related forum.

Thanks!

4

Thanks for the reply!
I’ve been through the links you have mentioned. My main issue is that the same model and code is working on JetPack 5.0.1 but not on JetPack 5.1.2 so I was hoping for some assistance.
EDIT: The post has been moved to the correct forum.

7

Hi,

About 2, could you add the --verbose flag to see if any error shows?

About 3, since there are some API changes between JetPack 5 and JetPack 4.
You will need to regenerate the calibration file on JetPack 5 directly.

Thanks.

8

Hi,

Regarding 2, in the post I’ve attacked the log file which was the output with the --verbose flag.
Regarding 3, I had generated the calibration cache file on the nx with jetpack 5.1.2 itself. As mentioned, I followed the same steps I followed on jetpack 5.0.1 but it didn’t work. (also attached in the post).
Thanks.

9

Hi,

Sorry that I misunderstood you are using the calibration file of JetPack 4.6.
Not notice that you have regenerated it on JetPack 5.

The trt_int8_dla.txt log is not complete and contains no error message.
Could you regenerate the file and attach it again?

Thanks.

10

Hi,
Apologies for any confusion. I encountered an error in the third scenario (INT8 + DLA with calibration cache file). In the second scenario (INT8 + DLA without providing a cache file), the program became unresponsive, and after waiting for over 25 hours, I had to terminate it. The log file captures events up to the termination point.

Do you wish to see the log file for the third scenario as well?

11

Hi,
Please find attached the log file for the third scenario.
int8_dla_calib_log.txt (6.0 MB)

12

Hi,

In general, scenario 2 should work.

We tried to run your model on XavierNX + JetPack 5.1.2 and the app did get stuck somewhere.

We need to check this with our internal team.
Will share more information with you.

For your reference, we did see some DLA-related errors by running$sudo dmesg.

Thanks.

14

Hi,

We have confirmed that the syncpoint timeout issue (TensorRT gets stuck) won’t happen on Orin’s DLA2.
Since our internal team doesn’t support DLA1’s issue anymore, please use GPU for inference instead.

Thanks.

1 Like
15

Could you provide assistance regarding the 3rd scenario as well?

16

Hi,

The model hit some issues with DLA inference.
This might also cause a problem when running DLA inference with a calibration file.

Thanks.

17

Hi,

We give a further test with the calibration file + GPU mode, and the same error also occurs.
Since GPU mode can run successfully without the calibration file, it looks more like the issue comes from the calibration file itself.

Thanks.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.