Originally published at: https://developer.nvidia.com/blog/deploy-large-language-models-at-the-edge-with-nvidia-igx-orin-developer-kit/
As large language models (LLMs) become more powerful and techniques for reducing their computational requirements mature, two compelling questions emerge. First, what is the most advanced LLM that can be run and deployed at the edge? And second, how can real-world applications leverage these advancements? Running a state-of-the-art open-source LLM like Llama 2 70B, even…