Live LLaVA not work

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

I followed this tutorial:Live LLaVA - NVIDIA Jetson AI Lab to set up VILA-2.7b on Jetson Orin Nano 8G.

First, I used

sudo init 3

to save memory and used ssh to connect to Jetson Orin Nano 8G.

When I entered the following command, it was stucked. After that it rebooted automatically.

jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.agents.video_query --api=mlc
–model Efficient-Large-Model/VILA-2.7b
–max-context-len 256
–max-new-tokens 32
–video-input /dev/video0
–video-output webrtc://@:8554/output

bug3
bug31639×683 186 KB

3

Hi @steven8_wang, can you try disabling ZRAM and mounting additional swap? https://github.com/dusty-nv/jetson-containers/blob/master/docs/setup.md#mounting-swap

Also try --vision-api=hf if it getting stuck after. Also can you try running it on some test images the first time during the quantization process?

jetson-containers run $(autotag nano_llm) \
  python3 -m nano_llm.agents.video_query --api=mlc \
    --model Efficient-Large-Model/VILA-2.7b \
    --max-context-len 256 \
    --max-new-tokens 32 \
    --vision-api hf \
    --prompt /data/prompts/images.json