Using other frameworks on Nvidia Drive AGX

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version

Target Operating System

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)

SDK Manager Version

Host Machine Version
native Ubuntu 18.04

I have a PyTorch model with an lstm layer that I would like to convert and run on Nvidia Drive AGX. However, the latest version of TensorRT officially supported on Drive AGX is TRT 6.5.0 , which does not have lstm layer support. I just wanted to know, what are my options here.

  1. Is there a port of Onnx-runtime/Pytorch/Torch-TensorRT that works with Drive AGX that I can use?
  2. I understand that converting a model to TRT format enables Drive AGX to execute the model at higher speeds. However, is there a way to run the model on Drive AGX without TRT conversion?

Additional Details:

  1. Please note that I was able to convert the PyTorch model to ONNX format. However conversion from ONNX to TRT fails with below error:
No importer registered for op: LSTM​
  1. I was also able to generate a TRT file for the model using Torch-TensorRT library. However, since the library targets TensorRT 8, I can’t use the model file on Drive AGX.

Any insight to tackle the issue is appreciated.

Dear @himanshu.dongre,
We officially do not support other frameworks on on DRIVE AGX. You may check installating and get the community support to set up other frameworks on DRIVE AGX. However, we recommend converting your model to ONNX and import into TensorRT for furthur optimzation on DRIVE AGX.

TRT 8 supports LSTM operation (Support Matrix :: NVIDIA Deep Learning TensorRT Documentation) but not on TRT 6.5. The next DRIVE Releases are targetted for DRIVE Orin platform which has TRT 8.x.

To continue development on DRIVE AGX, we recommend creating a plugin layer for unsupported layers. Please check " Extending TensorRT With Custom Layers" section in TensorRT developer guide.