I get the following error when I launch the Rtx with Chat application in the command window:
I am sure this is because of some version clashes in the download but I am not sure how to resolve them, any help will be greatly appreciated.
Edit: Issue resolved.2024-03-24T18:30:00Z
Copy paste the code from the below mentioned link to the NVIDIA’s version of pooling.py
Same error here. It was working fine before, it seems a recent update is causing the problem. I also get the following warning message prior to the error:
[03/25/2024-09:07:39] You try to use a model that was created with version 2.5.1, however, your version is 2.2.2. This might cause unexpected behavior or errors. In that case, try to update to the latest version.
Same issue for me.
Windows 11 23H2
Problem has been resolved from my end. This was the fix I had applied:
Copy paste the code from the below mentioned link to the NVIDIA’s version of pooling.py sentence-transformers/sentence_transformers/models/Pooling.py at master · UKPLab/sentence-transformers · GitHub
Thanks a lot for the fix, it works!
Works for me! Thanks!
Somewhat scary that NVidia updates code that prevents the whole system from launching.
I got same error
Thanks. It works for me.
Got same error
Environment path found: C:\Users\ArtyomBoyko\AppData\Local\NVIDIA\ChatWithRTX\env_nvd_rag
App running with config
{
"models": {
"supported": [
{
"name": "Mistral 7B int4",
"installed": true,
"metadata": {
"model_path": "model\\mistral\\mistral7b_int4_engine",
"engine": "llama_float16_tp1_rank0.engine",
"tokenizer_path": "model\\mistral\\mistral7b_hf",
"max_new_tokens": 1024,
"max_input_token": 7168,
"temperature": 0.1
}
},
{
"name": "Llama 2 13B int4",
"installed": true,
"metadata": {
"model_path": "model\\llama\\llama13_int4_engine",
"engine": "llama_float16_tp1_rank0.engine",
"tokenizer_path": "model\\llama\\llama13_hf",
"max_new_tokens": 1024,
"max_input_token": 3900,
"temperature": 0.1
}
}
],
"selected": "Mistral 7B int4"
},
"sample_questions": [
{
"query": "How does NVIDIA ACE generate emotional responses?"
},
{
"query": "What is Portal prelude RTX?"
},
{
"query": "What is important about Half Life 2 RTX?"
},
{
"query": "When is the launch date for Ratchet & Clank: Rift Apart on PC?"
}
],
"dataset": {
"sources": [
"directory",
"nodataset"
],
"selected": "directory",
"path": "dataset",
"isRelative": true
},
"strings": {
"directory": "Folder Path",
"nodataset": "AI model default"
}
}
[03/25/2024-20:31:30] You try to use a model that was created with version 2.5.1, however, your version is 2.2.2. This might cause unexpected behavior or errors. In that case, try to update to the latest version.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\ArtyomBoyko\AppData\Local\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\app.py:109 in │
│ <module> │
│ │
│ 106 ) │
│ 107 │
│ 108 # create embeddings model object │
│ ❱ 109 embed_model = HuggingFaceEmbeddings(model_name=embedded_model) │
│ 110 service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model, │
│ 111 │ │ │ │ │ │ │ │ │ │ │ context_window=model_config["max_input_to │
│ 112 │ │ │ │ │ │ │ │ │ │ │ chunk_overlap=200) │
│ │
│ C:\Users\ArtyomBoyko\AppData\Local\NVIDIA\ChatWithRTX\env_nvd_rag\lib\site-packages\langchain\em │
│ beddings\huggingface.py:66 in __init__ │
│ │
│ 63 │ │ │ │ "Please install it with `pip install sentence-transformers`." │
│ 64 │ │ │ ) from exc │
│ 65 │ │ │
│ ❱ 66 │ │ self.client = sentence_transformers.SentenceTransformer( │
│ 67 │ │ │ self.model_name, cache_folder=self.cache_folder, **self.model_kwargs │
│ 68 │ │ ) │
│ 69 │
│ │
│ C:\Users\ArtyomBoyko\AppData\Local\NVIDIA\ChatWithRTX\env_nvd_rag\lib\site-packages\sentence_tra │
│ nsformers\SentenceTransformer.py:95 in __init__ │
│ │
│ 92 │ │ │ │ │ │ │ │ │ │ use_auth_token=use_auth_token) │
│ 93 │ │ │ │
│ 94 │ │ │ if os.path.exists(os.path.join(model_path, 'modules.json')): #Load as Sen │
│ ❱ 95 │ │ │ │ modules = self._load_sbert_model(model_path) │
│ 96 │ │ │ else: #Load with AutoModel │
│ 97 │ │ │ │ modules = self._load_auto_model(model_path) │
│ 98 │
│ │
│ C:\Users\ArtyomBoyko\AppData\Local\NVIDIA\ChatWithRTX\env_nvd_rag\lib\site-packages\sentence_tra │
│ nsformers\SentenceTransformer.py:840 in _load_sbert_model │
│ │
│ 837 │ │ modules = OrderedDict() │
│ 838 │ │ for module_config in modules_config: │
│ 839 │ │ │ module_class = import_from_string(module_config['type']) │
│ ❱ 840 │ │ │ module = module_class.load(os.path.join(model_path, module_config['path'])) │
│ 841 │ │ │ modules[module_config['name']] = module │
│ 842 │ │ │
│ 843 │ │ return modules │
│ │
│ C:\Users\ArtyomBoyko\AppData\Local\NVIDIA\ChatWithRTX\env_nvd_rag\lib\site-packages\sentence_tra │
│ nsformers\models\Pooling.py:120 in load │
│ │
│ 117 │ │ with open(os.path.join(input_path, 'config.json')) as fIn: │
│ 118 │ │ │ config = json.load(fIn) │
│ 119 │ │ │
│ ❱ 120 │ │ return Pooling(**config) │
│ 121 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: Pooling.__init__() got an unexpected keyword argument 'pooling_mode_weightedmean_tokens'
Для продолжения нажмите любую клавишу . . .
I replaced the local pooling.py file with the file at sentence-transformers/sentence_transformers/models/Pooling.py at master · UKPLab/sentence-transformers · GitHub. This did not help. Maybe I did something wrong?
exact same here, i also have no idea about this replacing local pooling file so i’ll keep track here if anyone can help :)
EDIT: Open the pooling.py file with notepad then literally copy and paste this code above into it replacing what was there.
Hey @blademoon , maybe you replaced the wrong Pooling.py.
There are several in the Python environment, but only one with a capital P.
C:\Users\<user>\AppData\Local\NVIDIA\ChatWithRTX\env_nvd_rag\Lib\site-packages\sentence_transformers\models.
Too late here to check if there is some -Update to the env to avoid @aakashr1996 's kludge.
Anyone any idea how to keep this up to date anyway?
Thx
G.
This fixed my issue as well
Got the same error and replacing Pooling.py solved it.
Yes This works! I have been updating sentence transfromers and other pip’s and finally a fix! After days of messing around I finally have it working! Thankyou! Just copy and paste the code to the file path listed and the fix will work!
Can somebody make picture guide, or something like that? I can’t find Pool.py. I was trying to navigate by answers I got here, but some folders missing. Thank you
If you are getting the following Error (2.5.1 instead of 2.2.2):
Have a look at the error log, there you can see where the problem occured (also marked red in the picture).
You need to find this directory in your File Explorer.
It should be “[PATH TO MAIN DIRECTORY OF CHATRTX]\env_nvd_rag\lib\site-packages\sentence_transformers\models”.
(as seen and marked in the picture)
Download the “Pooling.py” file from GitHub or copy the content of the file.
Overwrite the old file in the location of the second step or change the content of the existing “Pooling.py”, so that it is matching the online version.
Start the Link on the Desktop or the “app_launch.bat” in “[PATH TO MAIN DIRECTORY OF CHATRTX]\RAG\trt-llm-rag-windows-main”
Profit. ;)
Thank you a lot. It worked now
@nbkdnz0r nbkdnz0r You are amazing. I want you to know I am a non-tech person (about 70%, so more than your average person, but nowhere near sophisticated coders or developers) who loves AI. I bought a new laptop specifically to use Chat with RTX. I’ve been trying to download and run it for 2 days and kept getting this error. Then I found you and this amazing step-by-step “fix.” THANK YOU. It’s running, and I am eternally grateful =)
And as of March 27, 2024 it looks like they have new download version with a pooling patch! Finally.
Do you all have YouTube capability with the newest?
