Can I use models like Llama-3-8B-Instruct-Coder with NIM?

AI & Data Science NVIDIA NIM Models
,
1

I’m wondering whether or not it’s possible to use NIM with models not listed on Models - NVIDIA Docs
E.g there is a fine tuned version of llama-3-8b-Instruct-bnb-4bit called rombodawg/Llama-3-8B-Instruct-Coder that I would like to use with NIM framework.
Is that somehow achievable?

3

Hey @joe_swanson – at the moment this isn’t possible. You can only use the base models that are included in the NIM itself, as well as any LoRA adapters trained for that base model.

1 Like