Trtexec problem

[pruizb]

Hello,

I have tried to create a TensorRT model using DLA and GPU on Xavier with JetPack 4.5.1, but I have obtained an error. The layer in which I have the problem is a Conv2D that is connected with a Reshape Layer (Lambda). Is there any solution for this problem?

I include two images, the first one with the error and the second shows the summary of the model.

Thanks, Paula

[AastaLLL]

Hi,

Does your model work well in TensorRT GPU mode?

If yes, could you share the detailed input/output dimension of the reshape layer?
There is a similar issue related to the DLA INT8 mode.
And the corresponding fix will be available in the upcoming release:

Can't build engines with the DLA + INT8 in Jetson Xavier NX

Hi. My English isn’t so good so feel free to ask me if there is anything unclear. I can’t build an engine with trtexec in Jetson Xavier NX with a combination of DLA and INT8. The following error message is printed and the program exits. …/builder/tacticOptimizer.cpp (2626) - Assertion Error in exemplar: 0 (pitch == 1 && “requirements impossible to satisfy”) ・Device : Jetson Xavier NX (Dev kit) ・It works well in FP16 + DLA. ・It works well in INT8 + GPU. Thank you in advance. Regards,

Thanks.

[pruizb]

Hi,

Yes, my model works well in TensorRT GPU mode. The goal of the reshape layer is to store temporary information in four dimensions. The input dimension is (None, 25, 60, 60, 1) and the output dimension is (None, 60, 60, 1). This way, we move the temporal information to the batch dimension.

A solution could be to choose to map some layers in DLA and others in GPU, but I don’t find any information. Is there any example or way to do it?

Thanks you!

[AastaLLL]

Hi,

Could you run the model with --dumpProfile in GPU mode to get the exact layer used for the reshaping?
Since based on the document below, we don’t support cross batch resize operation:

Support Matrix :: NVIDIA Deep Learning TensorRT Documentation

Thanks.

[pruizb]

Hello,

Yes, I attach an image with the result of running the model with --dumpProfile in GPU. The last layer is the reshape layer.

Do you mean that GPU mode supports cross batch resize operation but DLA doesn’t support it?

Just in case you forgot my previous question. Is there any example or way to map some layers in DLA and others in GPU? I don’t find any information about it.

Thanks you!

[AastaLLL]

Hi,

Have you verified the output to see if it is correct?

Based on our document, we don’t support cross batch reshaping in both GPU and DLA.
It might work if the reshaping behavior just meets the cross-batch version.
But it’s recommended to double-check the correctness first.

For layer-level placement, you can find some information in the below document.
It will require you to define the layer with TensorRT API directly rather than using onnx-parser:

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Thanks.