LLaVA Live Demo

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

I have ran the code for live llava demo

jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.agents.video_query --api=mlc
–model Efficient-Large-Model/VILA1.5-3b
–max-context-len 256
–max-new-tokens 32
–video-input /dev/video0
–video-output webrtc://@:8554/output

however it seems that its only tracking inputs and showing it on my terminal.

im unable to reach the port today but yesterday when I tried to do it , the web UI does not show my video there.

could I get some help please

3

Hi,

Do you mean it ran well before but failed recently?
If so, do you see any error or warning shown?

Thanks.

4

image
image2288×927 338 KB

When I refresh the page to enter the web UI, the script exits on its own as shown in the left of the image.

I am still unable to view the camera feed as well from the web UI even though i have followed the steps as mentioned in the tutorial page here: Live LLaVA - NVIDIA Jetson AI Lab

The VideoQuery agent applies prompts to the incoming video feed with the VLM. Navigate your browser to https://<IP_ADDRESS>:8050 after launching it with your camera (Chrome is recommended with chrome://flags#enable-webrtc-hide-local-ips-with-mdns disabled)

jetson-containers run $(autotag nano_llm) \
  python3 -m nano_llm.agents.video_query --api=mlc \
    --model Efficient-Large-Model/VILA1.5-3b \
    --max-context-len 256 \
    --max-new-tokens 32 \
    --video-input /dev/video0 \
    --video-output webrtc://@:8554/output

could i get some help? thanks @AastaLLL

5

Hi,

Thanks for sharing the details.
We will give it a try and provide more info to you later.

Thanks.

6

thanks alot.

7

Hi,

Could you share the JetPack version you use with us?

If you are using JetPack 6.1 or 6.2, we recommend to try VILA with microservice instead:

Thanks.

8

Hi @AastaLLL

i have fixed this issue but i have a new issue that i have shared on here VILA 1.5-3B Video Query Model

regarding framerate speed and object detection reliability, could you help me out over here? thanks

9

Sure. Let’s discuss the following question on that topic directly.
Thanks.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.