TensorRT incompatibility with Xavier AGX 64GB

[maximegado]

I’m using a Xavier AGX 64GB module, flashed with jetpack 4.6.1.

I am having trouble converting a ONNX model using:
trtexec --fp16 --onnx=/home/maxime/model.onnx --saveEngine=out.engine

Which output the following error:

[05/24/2022-03:19:03] [E] Error[2]: [utils.cpp::checkMemLimit::380] Error Code 2: Internal Error (Assertion upperBound != 0 failed. Unknown embedded device detected. Please update the table with the entry: {{1794, 8, 64}, 51309},)
[05/24/2022-03:19:03] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[05/24/2022-03:19:03] [E] Engine could not be created from network
[05/24/2022-03:19:03] [E] Building engine failed
[05/24/2022-03:19:04] [E] Failed to create engine from model.
[05/24/2022-03:19:04] [E] Engine set up failed

This commands used to work well under an AGX 32GB.
Any idea ?

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version

• Jetpack 4.6.1
• TensoRT v8201
• cuda_10.2_r440.TC440_70.29663091_0

Hardware Platform

• Xavier AGX 64GB

[maximegado]

this recent github dissussion suggest incompatibility between AGX64 and tensoRT but I doubt since this nvidia annoucement claims the opposite

[kayccc]

Your topic was posted in the wrong category. I am moving this to the Jetson AGX category for visibility.

[SivaRamaKrishnaNV]

Dear @maximegado,
We already noticed this issue with Xavier AGX 64GB boards and working on fix. We will update you soon on this issue.

[leo.chang]

Hello @SivaRamaKrishnaNV ,
We have the same issue, any updates?

[martin.herren]

In the meantime unfortunately our 64GB AGXs have been upgraded to very expensive door stoppers.

[AastaLLL]

Hi,

Thanks for your patience.
This issue is specified in TensorRT 8.2.

Could you use JetPack 5 (TensorRT 8.4) or JetPack 4.6 (TensorRT 8.0) instead?
To flash Xaiver 64GiB with JetPack 4.6, please find the overlay patch below:

Search “Overlay to support Jetson AGX Xavier 64GB” in this page.

Thanks.

[martin.herren]

Thanks for the info, didn’t know it would work on 4.6 with the overlay.

Unfortunately we depend on a third party carrier board provider and the Nvidia overlay is in conflict with his BSP and he didn’t provide an updated BSP or his own overlay.

At least we know we can use the 64GB boards on a DevKit for some testing.

Update to JetPack 5(.0.1) is an option for some preliminary testing of TensorRT 8.4). Support for A/B mechanism an OTA upgrade seems not supported yet (no ota-tools released for JetPack 5 yet it seems) so the big work of testing and migrating to JetPack 5 cannot start now.

Thanks and best regards.