Using other frameworks on Nvidia Drive AGX

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.7.1.8928
other

Host Machine Version
native Ubuntu 18.04
other

I have a PyTorch model with an lstm layer that I would like to convert and run on Nvidia Drive AGX. However, the latest version of TensorRT officially supported on Drive AGX is TRT 6.5.0 , which does not have lstm layer support. I just wanted to know, what are my options here.

  1. Is there a port of Onnx-runtime/Pytorch/Torch-TensorRT that works with Drive AGX that I can use?
  2. I understand that converting a model to TRT format enables Drive AGX to execute the model at higher speeds. However, is there a way to run the model on Drive AGX without TRT conversion?

Additional Details:

  1. Please note that I was able to convert the PyTorch model to ONNX format. However conversion from ONNX to TRT fails with below error:
No importer registered for op: LSTM​
  1. I was also able to generate a TRT file for the model using Torch-TensorRT library. However, since the library targets TensorRT 8, I can’t use the model file on Drive AGX.

Any insight to tackle the issue is appreciated.

3

Dear @himanshu.dongre,
We officially do not support other frameworks on on DRIVE AGX. You may check installating and get the community support to set up other frameworks on DRIVE AGX. However, we recommend converting your model to ONNX and import into TensorRT for furthur optimzation on DRIVE AGX.

TRT 8 supports LSTM operation (https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-803/support-matrix/index.html#supported-ops) but not on TRT 6.5. The next DRIVE Releases are targetted for DRIVE Orin platform which has TRT 8.x.

To continue development on DRIVE AGX, we recommend creating a plugin layer for unsupported layers. Please check " Extending TensorRT With Custom Layers" section in TensorRT developer guide.