llava-llama-2-13b-chat-lightning-gptq through oogabooga: RAM usage went from 14.17GB → 20.39GB (6.22GB), but that seems low so take it with a grain of salt. That was after querying it on an image a few times. Think it should run in 16GB though.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Introducing Ollama Support for Jetson Devices | 29 | 13708 | August 28, 2024 | |
| Ollama and Jetson issue | 12 | 6149 | March 20, 2024 | |
| I want to try LLaVa with Jetson Orin | 5 | 1178 | March 10, 2024 | |
| Available with Small Language Model on tutorial | 3 | 979 | May 3, 2024 | |
| Running llama3.3 or llama4 on Jetson AGX Orin Developer Kit (64 GB) | 8 | 1045 | May 12, 2025 | |
| MiniGPT-4 on Jetson Orin Nano 8Gb Dev kit not working | 9 | 591 | May 28, 2024 | |
| Triton Inference Server + vLLM Backend on the NVIDIA Jetson AGX Orin 64GB Developer Kit | 9 | 1126 | June 16, 2025 | |
| Cannot run LLaVa with Orin NX | 7 | 480 | August 1, 2024 | |
| Gen AI Benchmarking: LLMs and VLMs on Jetson | 7 | 142 | November 5, 2025 | |
| LLMs token/sec | 2 | 1213 | April 8, 2024 |