Originally published at: Getting Started with Edge AI on NVIDIA Jetson: LLMs, VLMs, and Foundation Models for Robotics | NVIDIA Technical Blog
Running advanced AI and computer vision workloads on small, power-efficient devices at the edge is a growing challenge. Robots, smart cameras, and autonomous machines need real-time intelligence to see, understand, and react without depending on the cloud. The NVIDIA Jetson platform meets this need with compact, GPU-accelerated modules and developer kits purpose-built for edge AI…
The vllm docker commands do not work because vLLM’s MXFP4 support requires compute capability 9.0+ (H100/B100), but Jetson Orin AGX has compute capability 8.7. It is based on the NVIDIA Ampere architecture, which includes specialized Tensor Cores and CUDA cores designed for high-performance AI and edge computing.