Greetings to all,
I published a post on how to integrate @dusty_nv 's MLC-LLM containers with MicroK8s on the NVIDIA Jetson AGX Orin Devloper Kit.
I hope it helps you reproduce the setup.
Best regards,
Shakhizat
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Running LLMs with TensorRT-LLM on Nvidia Jetson AGX Orin Dev Kit | 1 | 446 | December 8, 2024 | |
| Triton Inference Server + vLLM Backend on the NVIDIA Jetson AGX Orin 64GB Developer Kit | 0 | 320 | November 3, 2024 | |
| How to integrate two AGX Orins GPU resources in a Kubernetes (K8s) cluster for running a LLM inference? | 3 | 96 | November 1, 2024 | |
| Workflow to add newer Multimodal LLMs to evaluate | 3 | 62 | October 3, 2024 | |
| LLMs on DLA | 4 | 44 | April 2, 2025 | |
| NVIDIA NIM - TensorRT TurboXL | 2 | 225 | September 16, 2024 | |
| Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit | 0 | 475 | September 22, 2022 | |
| Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit | 0 | 486 | September 22, 2022 | |
| Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit | 0 | 256 | September 22, 2022 | |
| TensorRT-LLM for Jetson | 10 | 1745 | April 21, 2025 |