LLMs on DLA

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Could you tell me if LLMs/SLMs are supported on DLA on AGX Orin. If yes, could you share some resources for the same?

3

Hi,

Unfortunately, we don’t have a tutorial for running LLMs on DLA.
On Jetson, you can try TensorRT-LLM or MLC for running a LLM model.

Thanks.

4

Do TensorRT-LLM or MLC have support for DLA?

5

Hi,

They use GPU for inference.

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.