# TensorRT-LLM for Jetson

**URL:** <https://forums.developer.nvidia.com/t/tensorrt-llm-for-jetson/313228>\
**Category:** Announcements\
**Tags:** generative\_ai\
**Created:** [November 13, 2024, 7:01pm UTC](https://forums.developer.nvidia.com/t/tensorrt-llm-for-jetson/313228 "2024-11-13T19:01:42Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![dusty\_nv](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/dusty_nv/32/14043_2.png) [@dusty\_nv](https://forums.developer.nvidia.com/u/dusty_nv)\
**Post date:** [November 13, 2024, 7:01pm UTC](https://forums.developer.nvidia.com/t/tensorrt-llm-for-jetson/313228/1 "2024-11-13T19:01:42Z")

</div>

[**TensorRT-LLM**](https://github.com/NVIDIA/TensorRT-LLM)is a high-performance LLM inference library with advanced quantization, attention kernels, and paged KV caching. Initial support for TensorRT-LLM in JetPack 6.1 has been included in the [`v0.12.0-jetson `](https://github.com/NVIDIA/TensorRT-LLM/tree/v0.12.0-jetson)branch of the [TensorRT-LLM repo](https://github.com/NVIDIA/TensorRT-LLM)for Jetson AGX Orin.

We’ve made pre-compiled TensorRT-LLM [wheels](http://jetson.webredirect.org/jp6/cu126/tensorrt-llm/0.12.0)and [containers](https://hub.docker.com/r/dustynv/tensorrt_llm/tags) available, along with these guides and additional documentation:

> **[`> TensorRT-LLM Deployment on Jetson Orin`](https://github.com/NVIDIA/TensorRT-LLM/blob/v0.12.0-jetson/README4Jetson.md)**

[https://www.jetson-ai-lab.com/tensorrt\_llm.html](https://www.jetson-ai-lab.com/tensorrt_llm.html)