Jetson R35.3.1 TensorRT-LLM build fails

[andreyprokhar]

Hi, I encountered some issues while building the TensorRT-LLM package in my environment.

Is there a way to successfully build the TensorRT-LLM package for my system? Unfortunately, I cannot upgrade my Jetson to JetPack 6 due to hardware constraints. I attempted both manual building and using the jetson-containers builder, but both approaches failed similarly to the issue described here.
I noticed the note in the jetson-containers github that mentions the compatibility of container images with other minor versions of JetPack/L4T:
_{• L4T R32.7 containers can run on other versions of L4T R32.7 (JetPack 4.6+)
• L4T R35.x containers can run on other versions of L4T R35.x (JetPack 5.1+)}
However, I couldn’t find specific support information for L4T R35.3.1 in the docs.
Could you provide any recommendations or guidance if it possible to build and how to proceed?

[AastaLLL]

Hi,

TensorRT-LLM doesn’t support r35.
You can find more information below:

TensorRT-LLM for Jetson

TensorRT-LLM is a high-performance LLM inference library with advanced quantization, attention kernels, and paged KV caching. Initial support for TensorRT-LLM in JetPack 6.1 has been included in the v0.12.0-jetson branch of the TensorRT-LLM repo for Jetson AGX Orin. We’ve made pre-compiled TensorRT-LLM wheels and containers available, along with these guides and additional documentation: > TensorRT-LLM Deployment on Jetson Orin

Thanks.