Nvidia Jetson Orin Nano shows 6.3GB of RAM, run VILA-7B OOM!

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

When I am using Nano, I want to run a VILA-7B model on it, but I can only use 6.3GB of RAM, so the process will be killed because of oom. How can I solve this problem. My code is :
python3 -m local_llm.chat --api=mlc --model /data/vila/vila_7b --vision-model /data/clip-vit-large-patch14-336 --max-context-len 1536 --max-new-tokens 128

image
image1976×2004 491 KB

3

Hi,

The app used 6.1G over a total of 6.3G memory, which is already 97%.
Please use the model that requires less memory.

Thanks.

4

But Nano should has 8GB Ram, why I can only see 6.3GB? How can I get more memory?

5

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Could you also share the tegrastats log with us?

$ sudo tegrastats

Thanks.