I am encountering an incompatibility on a specific operation when generating an engine with the trtexec command line. I would like to use the DLA as much as possible and not rely on the GPU fallback.
Here is the actual operation in PyTorch :
target_size = target.size() # equals to torch.Size([1,1,32,32])
torch.nn.Functional.interpolate(x, size=(target_size, target_size), mode='nearest')
Here is the equivalent operation with an ONNX conversion :
And finally, the incompatibility observed with trtexec :
So far, I have tried the following:
DLA supported layers guide, but every constraint seems to be respected. converting the shapes format with ".type(
tensor.type(torch.int8)) " before the interpolation operation.
My configuration :
Pytorch (on host) : 1.13
Platform : Orin AGX Dev. kit
JetPack : 5.0.2
TensorRT : 220.127.116.11
CUDA : 11.4
Could you please help me to interpret the “Please explicitly use CAST operator in ONNX model or add an identity layer to convert INT32 to other types for DLA” message?
Thank you very much for your help
Could you please give us more details about environment you’re using.
Jetpack (if applicable):
Nvidia Driver Version:
Operating System + Version:
Baremetal or Container (if container which image + tag):
To get better help, we’re moving this post to the Jetson Orin related forum.
DLA support FP16 and INT8 layer.
The message indicates that your model is using some other data type(INT32?).
Do you meet any errors when converting the model into the TensorRT engine?
Is the output a warning message or an error that leads to an app crash?
More, it’s recommended to give our latest JetPack 5.1 a try.
Thank you for your reply.
I have finally understood why the DLA did not layer not support the layer. It was my fault.
In the constraints DLA guide, I mixed up the
scale and the output shape terms for the onnx equivalent resize layer. Therefore, the actual underlying scale factor could not be an integer in my case and the DLA constraints were not satisfied.
Replacing the torch.nn.Functional.interpolate(size= …) with the torch.nn.Upsample(scale_factor = …) solves the problem.
Thank you very much for your help!
Good to know it works now.
Thanks for the feedback.
February 9, 2023, 4:13pm
Also check out the DLA github page for samples and resources or to report issues:
Recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.
We have a FAQ page that addresses some common questions that we see developers run into:
February 23, 2023, 4:13pm
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.