VILA 1.5-3b Model

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

I am using the Jetson Orin Nano develop kit (8gb) however when im running the quantization of the model it says that process is killed due to insufficient memory space. From the LLaVA live demo, it says that VILA 1.5-3b is able to run on the Orin Nano. What can be done for this case?

3

Hi,

Please try the command on the below page to optimize memory and test the VILA model again:

Thanks.

4

Hello, Thanks for this. I will proceed to try this out and comment back on whether it is successful. Is there a nano_vlm code that I can try instead of this nano llm code below?

jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.agents.video_query --api=mlc
–model Efficient-Large-Model/VILA1.5-3b
–max-context-len 256
–max-new-tokens 32
–video-input /dev/video0
–video-output webrtc://@:8554/output

@AastaLLL thanks alot

5

Hi,

Please find the tutorial below.
The example deploys VILA 1.5 3B with VILA Microservice.

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.