Greetings to all,
I published a post on how to integrate @dusty_nv 's MLC-LLM containers with MicroK8s on the NVIDIA Jetson AGX Orin Devloper Kit.
I hope it helps you reproduce the setup.
Best regards,
Shakhizat
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Running LLMs with TensorRT-LLM on Nvidia Jetson AGX Orin Dev Kit | 1 | 383 | December 8, 2024 | |
| Triton Inference Server + vLLM Backend on the NVIDIA Jetson AGX Orin 64GB Developer Kit | 0 | 244 | November 3, 2024 | |
| How to integrate two AGX Orins GPU resources in a Kubernetes (K8s) cluster for running a LLM inference? | 3 | 84 | November 1, 2024 | |
| Workflow to add newer Multimodal LLMs to evaluate | 3 | 54 | October 3, 2024 | |
| NVIDIA NIM - TensorRT TurboXL | 2 | 169 | September 16, 2024 | |
| Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit | 0 | 473 | September 22, 2022 | |
| Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit | 0 | 486 | September 22, 2022 | |
| Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit | 0 | 256 | September 22, 2022 | |
| How to infer LLMs on the DRIVE AGX Orin Developer Kit? | 3 | 934 | February 5, 2024 | |
| Running LMdeploy inference engine on the NVIDIA Jetson AGX Orin Devkit | 2 | 75 | January 14, 2025 |