Which dustynv/ollama I should use?

[relaxtheo]

I want to use dustynv/ollama for testing, and I can find many different 36.4 versions in https://hub.docker.com/r/dustynv/ollama/tags?name=36.4, like r36.4.0, main-r36.4.0, 0.6.3-r36.4.0, etc. Which one I should use？ Any guide for selection?

[AastaLLL]

Hi,

Which model do you want to try?

For example, you can find the corresponding container that recommended to use in the below page:

Models - NVIDIA Jetson AI Lab

Thanks.

[relaxtheo]

By my understanding, the ollama is a platform to manage and run LLMs, so I can use the ollama to pull and run other models. For such case, which ollama I should install, the one from ollama.com, or I can also the dustynv/ollama?

I thought the dustynv/ollama should support the Jetson hardware better than the one from ollama.com (like can run models on GPU with better performance). Right?

And I found there is docker image specified for each jetson-ai-lab models, so I should use the specified docker image to test the models? And can the dustynv/ollama can be used for other non-jetson-ai-lab models? And If I can get better performance than the ollama from ollama.com?

Thank you.

[AastaLLL]

Hi,

Please try with dustynv/ollama container.
The official ollama might need some modification to solve the compatibility issues on Jetson while dustynv/ollama has all the dependencies pre-installed.

In our tutorial, we also test LLMs with MLC and VLMs with vLLM containers.

Thanks.