Resize incompatibility when generating a full TensorRT engine for DLA

[lab2022]

Hi,

I am encountering an incompatibility on a specific operation when generating an engine with the trtexec command line. I would like to use the DLA as much as possible and not rely on the GPU fallback.

• Here is the actual operation in PyTorch :

target_size = target.size()  # equals to torch.Size([1,1,32,32]) 
torch.nn.Functional.interpolate(x, size=(target_size[2], target_size[3]), mode='nearest')

• Here is the equivalent operation with an ONNX conversion :

• And finally, the incompatibility observed with trtexec :

So far, I have tried the following:

• checkeing the DLA supported layers guide, but every constraint seems to be respected.
• converting the shapes format with ".type( tensor.type(torch.int8)) " before the interpolation operation.

My configuration :

• Pytorch (on host) : 1.13
• Platform : Orin AGX Dev. kit
• JetPack : 5.0.2
• TensorRT : 8.4.0.1
• CUDA : 11.4

Could you please help me to interpret the “Please explicitly use CAST operator in ONNX model or add an identity layer to convert INT32 to other types for DLA” message?

Thank you very much for your help

[spolisetty]

Hi,

Could you please give us more details about environment you’re using.

Environment

TensorRT Version:
GPU Type:
Jetpack (if applicable):
Nvidia Driver Version:
CUDA Version:
CUDNN Version:
Operating System + Version:
Baremetal or Container (if container which image + tag):

Thank you.

[lab2022]

Hi,

Here is my environment :

• TensorRT Version : 8.4.1
• GPU Type : Jetson Orin Development Kit
• Jetpack version : 5.0.2-b231
• CUDA Version : 11.4
• CUDNN Version : 8.4.1.50
• Operating System : Linux (Jetson OS)

Thank you very much

[spolisetty]

Hi,

To get better help, we’re moving this post to the Jetson Orin related forum.

Thank you.

[AastaLLL]

Hi,

DLA support FP16 and INT8 layer.
The message indicates that your model is using some other data type(INT32?).

Do you meet any errors when converting the model into the TensorRT engine?
Is the output a warning message or an error that leads to an app crash?

More, it’s recommended to give our latest JetPack 5.1 a try.

Thanks.

[lab2022]

Hi,

Thank you for your reply.

I have finally understood why the DLA did not layer not support the layer. It was my fault.

In the constraints DLA guide, I mixed up the scale and the output shape terms for the onnx equivalent resize layer. Therefore, the actual underlying scale factor could not be an integer in my case and the DLA constraints were not satisfied.
Replacing the torch.nn.Functional.interpolate(size= …) with the torch.nn.Upsample(scale_factor = …) solves the problem.

Thank you very much for your help!

[AastaLLL]

Hi,

Good to know it works now.
Thanks for the feedback.

[ramc]

Also check out the DLA github page for samples and resources or to report issues: Recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.

We have a FAQ page that addresses some common questions that we see developers run into: Deep-Learning-Accelerator-SW/FAQ