Evaluation of Tensorrt model not happening in JETPACK 6 with TENSORRT 8.6

Hi Team,

We have installed JETPACK 6 with TensorRT 8.6 in our JETSON ORIN 32 GB DEVELOPER KIT(s) through SDK manager with the compatible Pytorch version i.e. 2.2.0 which we are using converted TensorRT model to evaluate and check the performance of the model and calculate the quality of the predicted outputs. But, evaluation of the TensorRT model is giving the below mentioned error:

Traceback (most recent call last):
** File “/home/Documents/Vision_Transformer/03_eval_trt.py”, line 270, in **
** main()**
** File “/home/Documents/Vision_Transformer/03_eval_trt.py”, line 246, in main**
** pred_rgb, pred_anno, save_name, t = model.run(image_path)**
** File “/home/Documents/Vision_Transformer/03_eval_trt.py”, line 134, in run**
** _pred_labels, _pred_scores, _mask_labels, _mask_probs = compute_segments(**
** File “/home/Documents/Vision_Transformer/lib/models/post_processing.py”, line 44, in compute_segments**
** mask_probs = nn.functional.interpolate(**
** File “/home/.local/lib/python3.10/site-packages/torch/nn/functional.py”, line 4020, in interpolate**
** return torch._C._nn.upsample_bilinear2d(input, output_size, align_corners, scale_factors)**
RuntimeError: Non-empty 4D data tensor expected but got a tensor with sizes [1, 0, 384, 384]

Basic flags used to convert ONNX model to TRT:
os.system(f"/usr/src/tensorrt/bin/trtexec --onnx={onnx_model_path} --saveEngine={trt_model_path} --fp16)
The same combination works well with Jetpack 5.1.2

Note:

  1. Which converting the ONNX model to TRT without adding any precision, it works as usual without any compression in the model size.

Pytorch installation reference: PyTorch for Jetson
and also we have tested the inference with Pytorch 2.1, but same error persist. Kindly check and assist.

Hi,

The intermediate tensor has size [1, 0, 384, 384] and causes the issue.
Could you check if the model can run with trtexec successfully?

If yes, this sounds like a compatible issue.
We will need the model and the reproducible source to debug further.

Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.