Jetson Orin Nano with LLaVA 1.5 Model

[sng137]

Hi All!

I am trying to use the LLaVA 1.5 7B model from Jetson AI Lab (Models - NVIDIA Jetson AI Lab), and I have a Jetson Orin Nano Jetpack 6.2, Ubuntu 22.04 and Cuda 12.6.

However, I noticed that the docker image for LLaVA requires Cuda 12.8 and Ubuntu 24.04. I have tried searching for docker images which can be used with Ubuntu 22.04 and Cuda 12.6, but I have found none so far.

Any and all help is appreciated thanks!

[AastaLLL]

Hi,

You can find the container info on the same page.

For example, the LLaVA model uses vllm model which you can find in the below link:

https://hub.docker.com/r/dustynv/vllm/tags

Thanks.