VLM on video running on NVIDIA Jetson

Hello,

I have been playing around with the Jetson Generative AI Lab on an AGX Orin 32GB device and also saw this blog post:

In this post, there is a video:

In the very beginning of this video, it shows that VLM is running on a video where it is generating text to explain what is hapenning in the video.

Can I know how I can run this demo on my device?

Thank you.

Hi @lakshantha.d, I don’t believe that particular demo has been released yet. However there are tutorials on Llava and Mini-GPT4 here:

I also have optimized Llava-1.5 in updated version of llamaspeak coming out:

Hi @lakshantha.d, just following up - given your interest in VLM video queries, you may want to check out this upcoming webinar:

https://info.nvidia.com/metropolis-gen-ai-webinar.html

Great!

Thank you for the share and the updates @dusty_nv . Will attend the webinar as well!

Hello @dusty_nv,

Is Llava for llamaspeak out yet?

The code is technically available here, I just need to document it (will get to it soon)

Great! Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.