I downloaded the installer from Nvidia, but for some reason Llama LLM did not seem to have been installed. There are however some Llama looking files in the installation folder.
Anyone has any ideas on how to make the Llama LLM available to be used?
2 Likes
I have the same problem. There’s no option in the installer to install the Llama 2 model even though it’s included in the download package.
I am also having this same problem. Fresh download and it doesnt show up in the install list.
Same here, I tried to update the config file “\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\config\config.json” to installed = true and copied the llama folder from the install folder to the “\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\model”.
No success yet.
So it seems the engine is not being built for some reason, the “…/llama/llama13_int4_engine/” was not created.
K, so you can modify the minimum requirements for a GPU with less than 16GB in this file llama13b.nvi located in the installation directory
“\NVIDIA_ChatWithRTX_Demo\ChatWithRTX_Offline_2_11_mistral_Llama\RAG”.
9 Likes
This solution really WORKS for me!
Problem solved!
hi, I’m struggling with the same problem and its my first time using AI for anything. does this step fix the problem? so i install it directly or do i have to copy the llama folder from the install folder to the “\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\model”. like someone specified before?
thank you.
1 Like
You need to extract the installation zip. In inside its file structure you will find the nvi file referenced above. Adjust the value from 16 to 7. Obviously there’s a reason it’s supposed to have 16 or more. LOL
where is the solution?
this solution did not work for me. installer version 0.4