Jetson Orin Nano 8gb Crashing Running NanoVLM Live Streaming

[Tsukasa42]

I’m trying to run one of the projects listed on the site for NanoVLM and the OS locks up and then crashes.
I see that it runs out of memory, starts swapping and then crashes. The example states it can run on the Jetson Nano 8gb but that doesn’t appear to be the case. Is there something i’m missing?
Its currently running on an SD Card, I do have an SSD on order to try but I’m not sure that is going to help.

NanoVLM - NVIDIA Jetson AI Lab

Live Streaming

jetson-containers run $(autotag nano_llm) \
  python3 -m nano_llm.agents.video_query --api=mlc \
    --model Efficient-Large-Model/VILA1.5-3b \
    --max-context-len 256 \
    --max-new-tokens 32 \
    --video-input /dev/video0 \
    --video-output webrtc://@:8554/output

[Tsukasa42]

I see in the benchmarks the model is listed as OOM on the Jetson Orin Nano, seems a bit misleading that its tagged at the top of the page.

[AastaLLL]

Hi,

Have you applied the below changes to optimize the memory on an 8GB device?

🔖 Memory optimization - NVIDIA Jetson AI Lab

Thanks.

[Tsukasa42]

Perfect, this is what I was missing. Thank you :).