Can Drive Orin support TensorRT-LLM?

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Description

Saw that Nvidia migrated TensorRT-LLM on Drive Orin from below talk " Advancing Automotive AI With Large Language Models and Vision Language Models on NVIDIA DRIVE".

Is there any guidance to get and setup TensorRT-LLM on Drive Orin platform?

Environment

Drive OS: 6.0.6 or 6.0.8
CUDA: 11.4 in Drive OS 6.0.6 or 6.0.8 (TensorRT-LLM requires CUDA 12+)

2

Dear @chun-wei.chen ,
We don’t have TensorRT LLM release available for devzone.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.