Nvidia jetson orin nano has tensorrt support?

[grvtpathak]

Does Nvidia jetson orin nano 8GB developer kit support tensorrt_llm of VLM (qwen 2 vl 2b) for Jetpack 6.2. ? If yes how can i convert the model to tensorrt as jetson was not able to build the library tensorrt_llm.?

[AastaLLL]

Hi,

You can find TensorRT-LLM info in the below link:

TensorRT-LLM for Jetson

TensorRT-LLM is a high-performance LLM inference library with advanced quantization, attention kernels, and paged KV caching. Initial support for TensorRT-LLM in JetPack 6.1 has been included in the v0.12.0-jetson branch of the TensorRT-LLM repo for Jetson AGX Orin. We’ve made pre-compiled TensorRT-LLM wheels and containers available, along with these guides and additional documentation: > TensorRT-LLM Deployment on Jetson Orin

We also have some examples and tutorials of Qwen2.5 below for your reference:

Models - NVIDIA Jetson AI Lab

Thanks.