While running the VIA Summarization model I am getting the below mentioned error:
ERROR We couldn’t connect to ’ https://huggingface.co’
to load this file, couldn’t find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json.
ERROR Failed to load VIA pipeline - CA-RAG setup failed. Check if NVIDIA_API_KEY set correctly and/or LLM configuration in CA-RAG config is valid.
Please note:
I have downloaded the VITA 2.0 Model from web and kept the files inside folder: VIA/nvidia_tao_vita_2.0.1_vila-llama-3-8b-lita/ . and given the export NGC_MODEL_CACHE as the same.
If I am downloading the Hugging face model offline. Where i shall keep these files?
We have manually downloaded (As our network is restricting them from auto download) both the VITA 2.0 Model and the Hugging Face model and have kept the Hugging face model inside /home/VIA/all-MiniLM-L6-v2 path and my model is inside /home/VIA/nvidia_tao_vita_2.0.1_vila-llama-3-8b-lita/ and my NGC_MODEL_CACHE is /home/VIA/.
What changes I shall do as it is giving below error:
2024-10-21 12:52:51,150 ERROR We couldn’t connect to ‘https://huggingface.co’ to load this file, couldn’t find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json.
2024-10-21 12:52:51,150 ERROR Failed to load VIA pipeline - CA-RAG setup failed. Check if NVIDIA_API_KEY set correctly and/or LLM configuration in CA-RAG config is valid.
Killed process with PID 56
I tried running the VIA Summarization Warehouse Use case using VITA 2.0 model as VLM but our network was not allowing us to download the model, so we downloaded the model and kept it inside folder: /home/VIA/nvidia_tao_vita_2.0.1_vila-llama-3-8b-lita/.
And then we tried running again, it came out with the below error:
2024-10-21 12:52:51,150 ERROR We couldn’t connect to ‘https://huggingface.co’ to load this file, couldn’t find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json.
2024-10-21 12:52:51,150 ERROR Failed to load VIA pipeline - CA-RAG setup failed. Check if NVIDIA_API_KEY set correctly and/or LLM configuration in CA-RAG config is valid.
Killed process with PID 56
From above error we feel that again our network has again blocked hugging face model, So we downloaded the model and kept it inside /home/VIA/all-MiniLM-L6-v2 but again it is giving same error.
We tried with above folder structure but still getting the same error:
2024-10-23 09:55:18,771 INFO Stopping VIA pipeline
2024-10-23 09:55:18,771 ERROR We couldn’t connect to ‘https://huggingface.co’ to load this file, couldn’t find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode’.
2024-10-23 09:55:18,771 ERROR Failed to load VIA pipeline - CA-RAG setup failed. Check if NVIDIA_API_KEY set correctly and/or LLM configuration in CA-RAG config is valid.
Killed process with PID 56
Do we need to download any other models or can you share which Hugging face model it is actually searching for?
Our file structure looks something like this that we downloaded from Hugging Face and we have kept these files inside : /home/VIA/via-hf-cache/hub/models–sentence-transformers–all-MiniLM-L6-v2
We were able to solve the Huggingface error problem. Now we are getting below error, We tried giving --privileged=true in docker run command as well:
2024-10-25 09:48:53,051 INFO Stopping VIA pipeline
2024-10-25 09:48:53,052 ERROR Expecting value: line 1 column 1 (char 0)
2024-10-25 09:48:53,052 ERROR Failed to load VIA pipeline - CA-RAG setup failed. Check if NVIDIA_API_KEY set correctly and/or LLM configuration in CA-RAG config is valid.
As per our understanding this is due to network restriction at our organization because of which it is not able to make the call with NVIDIA NIM API
Is there is any possibility to download the model offline and keep it at desired path, then please share the steps, path and the link to download the model.
You can refer to the link I attached before Using Locally Deployed LLM NIM instead of NVIDIA Hosted LLM NIM. At the end of page 44, we have instructions on how to deploy locally.
All you need to do is deploy the NIM locally. The documentation explains how to configure the model before you start the docker command.