Cannot use VILA on jetson-containers

[masaki_yamagishi]

I cannot use VILA on jetson-containers, as like NanoVLM, or Live LLaVA.
Are there anything wrong when using VILA on jetson-containers?
At previous time (as yesterday), it worked fine.

For example, I execute NanoVLM following tutorial, an error is showed as below.

NanoVLM - NVIDIA Jetson AI Lab

$ jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.chat --api=mlc
–model Efficient-Large-Model/VILA1.5-3b
–max-context-len 256
–max-new-tokens 32

vila_error_0.txt (3.2 KB)

This error (… /data/models/mlc/dist/VILA1.5-3b-ctx256 ’ returned non-zero exit status 1) is also showed when running Live LLaVA.

Then, I tried other models, VILA1.5-3b, and liuhaotian/llava-v1.5-7b.
When with VILA1.5-3b, an error (Repository Not Found for url: https://huggingface.co/api/models/VILA1.5-3b/revision/main) was occurred.

$ jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.chat --api=mlc
–model VILA1.5-3b
–max-context-len 256
–max-new-tokens 32

vila_error_1.txt (3.1 KB)

And with llava-v1.5-7b, it worked fine.

$ jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.chat --api=mlc
–model liuhaotian/llava-v1.5-7b
–max-context-len 256
–max-new-tokens 32

Please give some information to improve it.

Best regards,

[dusty_nv]

@masaki_yamagishi

When with VILA1.5-3b, an error (Repository Not Found for url: https://huggingface.co/api/models/VILA1.5-3b/revision/main) was occurred.

Hi @masaki_yamagishi, perhaps it was some temporary network issue or corruption - the correct model name/repo should be Efficient-Large-Model/VILA1.5-3b

If you had used previous version of VILA but not updated your NanoLLM container, please do a docker pull $(autotag nano_llm) (it required code updates to support VILA-1.5)

You might also try deleting/re-downloading your VILA-1.5 model under jetson-containers/data/models/huggingface

[masaki_yamagishi]

Hi, @dusty_nv

Thank you for your dedicated reply.
I can use Efficient-Large-Model/VILA1.5-3b today.
So that might be temporary network issue.

And I understood how update VILA network.

Thank you for your support.

Additional:
WebUI of NanoDB is still disappeared when running Live LLaVA as below.
If you can, please help me.

Live LLaVA webUI don't show NanoDB webUI

I tried Live LLaVA Tutorial following below instruction, but it not shows NanoDB UI. How can I integrate NanoDB to Live LLaVA appropriately? My webUI of LLaVA is like below. [live_llava_no_nanodb] In same environment, NanoDB is works fine. [work_nanodb] And “chrome://flags/#enable-webrtc-hide-local-ips-with-mdns” is disabled. [chrome_settings] I use below equipment. Jetson AGX Orin 32GB Devkit / JetPack 6.0DP / 2.0 TB NVMe SSD / Logicool C615 HD Webcam Best regards,

[dusty_nv]

OK great, glad to hear that you got VILA-1.5 working @masaki_yamagishi! We will tackle the NanoDB issue you have next!