[SUPPORT] Workbench Example Project: Hybrid RAG

wwb2portals · November 27, 2024, 7:32pm

Its AMD Ryzen 7 3700X
Finally I want to use Cloud Endpoint but the project environment wont build.

edwli · December 2, 2024, 8:14pm

12/2/2024

Fixed an issue with the openai api calls (cloud, NIMs) failing due to the following error

TypeError: Client.__init__() got an unexpected keyword argument 'proxies'

Solution identified here and fixed in the project.

Alex.Y · December 19, 2024, 2:45am

Hi sir @edwli, I am a newer guy of Nvidia AI workbench, I want to have a test the Hybird RAG project on my machine, and now i meet some problem need your help.

After clone the whole project of hybrid-rag using AI Workbench in Linux(Ubuntu 22.04), when i click the “Open Chat” button, it happen some errors , I attached the log here, i see the from the log and from the AI workbench, it also remind that the GPU is not available, but i installed the NVIDIA RTX A5000 already and installed the Linux driver.
So can you help tell me how i should do to fix this issue?, thank you.
workbench-hybird-rag-Open-Chat-Linux22.04-error.log (456.6 KB)

Alex.Y · December 19, 2024, 2:49am

Alex.Y · December 19, 2024, 6:35am

Hi @edwli sir, I also tried the AI workbench in windows11, after clone the project, I can open the Hybird RAG Chat UI after click “Open Chat”, but when i click “Set Up RAG Backend” button, it will occur error, i checked the error, find the curl and libcurl.so.4 mis match, do you have any suggestion how to fix the issues, thank you.

the chat error log as below, i also attached the log.
nvai-workbench-example–chat-error-log.txt (11.1 KB)

"
stat: cannot statx ‘/var/host-run/docker.sock’: No such file or directory
groupadd: invalid group ID ‘docker’
usermod: group ‘docker’ does not exist
Starting Milvus
Starting API
*Polling inference server. Awaiting status 200; trying again in 5s. *
curl: /opt/conda/lib/libcurl.so.4: no version information available (required by curl)
*Polling inference server. Awaiting status 200; trying again in 5s. *
Traceback (most recent call last):

File “/home/workbench/.conda/envs/api-env/lib/python3.10/runpy.py”, line 196, in _run_module_as_main*
return _run_code(code, main_globals, None,*
File “/home/workbench/.conda/envs/api-env/lib/python3.10/runpy.py”, line 86, in _run_code*
exec(code, run_globals)*
File “/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/main.py”, line 4, in *
uvicorn.main()*
File “/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/click/core.py”, line 1157, in call*
return self.main(*args, *kwargs)
File “/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/click/core.py”, line 1078, in main*
rv = self.invoke(ctx)*
File “/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/click/core.py”, line 1434, in invoke*
return ctx.invoke(self.callback, *ctx.params)
"

edwli · January 14, 2025, 8:14pm

Hey Alex,

Unfortunately I am unable to reproduce the issue you are seeing. I just did a clean clone from the GitHub page and it was working for me…

I see an error in the logs in your screenshot

No module named ‘pkg_resources’

It looks like a pip package conflict. Are you working on an older version of the project? Or is this a fresh clone from the Github?

From the logs it looks like it originates from the llama_index package, I’m thinking perhaps it may need a version bump on your end:

from llama_index.download.module import (
File “/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/llama_index/download/module.py”, line 12, in
import pkg_resources
ModuleNotFoundError: No module named ‘pkg_resources’

Alex.Y · January 15, 2025, 2:34am

Hi @edwli , thanks for your reply and help, the issue have been fixed yet, A good engineer of Nvidia told me i need apply the credits of access to the LLM Foundation models with the Corporate(Business) E-mail, and after that i can build this correctly.

dinesh_mehra · February 19, 2025, 6:24pm

I have same problem, started creating new project hybrid-rag yesterday. Reached step 29 of 34. Today still at step 29/34. Tried to stop build, get error ’ there was a problem stopping the build for hybrid-rag’. here is the log

…
…
0 upgraded, 1 newly installed, 0 to remove and 52 not upgraded.

Need to get 8489 kB of archives.

After this operation, 18.1 MB of additional disk space will be used.

Get:1 https://packagecloud.io/github/git-lfs/ubuntu jammy/main amd64 git-lfs amd64 3.6.1 [8489 kB]

debconf: delaying package configuration, since apt-utils is not installed

Fetched 8489 kB in 1s (12.3 MB/s)

Selecting previously unselected package git-lfs.

(Reading database … (Reading database … 5% (Reading database … 10% (Reading database … 15% (Reading database … 20% (Reading database … 25% (Reading database … 30% (Reading database … 35% (Reading database … 40% (Reading database … 45% (Reading database … 50% (Reading database … 55% (Reading database … 60% (Reading database … 65% (Reading database … 70% (Reading database … 75% (Reading database … 80% (Reading database … 85% (Reading database … 90% (Reading database … 95% (Reading database … 100% (Reading database … 19396 files and directories currently installed.)

Preparing to unpack …/git-lfs_3.6.1_amd64.deb …

Unpacking git-lfs (3.6.1) …

Setting up git-lfs (3.6.1) …

Git LFS initialized.

time=“2025-02-18T15:21:21-05:00” level=warning msg=“SHELL is not supported for OCI image format, [/bin/bash -c] will be ignored. Must use docker format”

Build Initiated

twhitehouse · February 20, 2025, 5:30pm

Hi. Thanks for posting.

Sometimes this can be fixed by deleting a file.

Try the following:

Go to the ~/.nvwb/project-runtime-info folder on your system. I’m assuming it’s your laptop and not a remote.
- Note that if you are on Windows this is in the WSL distro called NVIDIA-Workbench.
Find the runtime folder for the hybrid rag project and look for the build-output file. It will have some sort of ending.
Delete it.
Go back to the Desktop App and you should have cleared the problem.
Rebuild the container and see what happens.

edwli · April 7, 2025, 9:16pm

4/7/2025

unstructured version bump to resolve

ImportError: cannot import name ‘PSSyntaxError’ from ‘pdfminer.pdfparser’ (/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/pdfminer/pdfparser.py)

aryamonani · April 7, 2025, 9:51pm

Hi Everyone I am trying to run the example rag on my workbench with docker installed on my macbook pro M3. The build is complete but the environment wont start saying that I dont have enough GPUs. Can someone help out with this?

edwli · April 7, 2025, 9:59pm

Hi, it looks like you are working locally on an ARM-based Mac machine.

If you are working locally on the mac, then please note that you do need a dedicated GPU to run this project. Your currently system does not have a dedicated GPU.

Do you have a remote Ubuntu box you have access to with a GPU? If so you can connect to it from your Mac and use that location for compute for this project. You can read more about how to do so here.

(Also please note that ARM-based macs are generally unsupported by HF TGI (base container image for this project). But it seems like the project built fine for you, so maybe it is ok)

aryamonani · April 7, 2025, 10:05pm

I am using a mac with docker and NGC, to access a remote GPU though I’m not sure if I am able to access it. I used desktop docker’s mac socket and pointed to it in my workbench which allowed for my build to be complete.

edwli · April 7, 2025, 10:14pm

I am using a mac … to access a remote GPU though I’m not sure if I am able to access it

What do you mean by this? Can you access this remote machine? If so, you can add this machine as a new location to work in from your Mac.

Install Workbench for Remote Ubuntu on this remote system
Add your SSH public key to this remote system
Open AI Workbench on your Mac and Add Location with your instance details and SSH private key

(Full instructions)

Then, you should be good to go. Every time you open AI Workbench on your Mac you can click into this remote location and work there as if it were any other GPU-enabled system.

From your screenshot in the other post, it looks like workbench is still pointed to the local context, so it is looking for local resources for a GPU that does not exist. If you have access to that other GPU system, then AI Workbench is designed with this exact use case in mind to help you work wherever you want.

btapiasierra · May 10, 2025, 10:05am

Hi, I recently forked and cloned over the rag project in github. App builds fine, am able to open the chat app at localhost:10000, but when attempting to “Set Up RAG Backend” it doesn’t bring up the backend server. This is the error found in chat logs

stat: cannot statx '/var/host-run/docker.sock': No such file or directory
groupadd: invalid group ID 'docker'
usermod: group 'docker' does not exist
Starting Milvus
Starting API
curl: /opt/conda/lib/libcurl.so.4: no version information available (required by curl)
Polling inference server. Awaiting status 200; trying again in 5s. 
Traceback (most recent call last):
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/__main__.py", line 4, in <module>
    uvicorn.main()
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/click/core.py", line 1443, in invoke

  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/main.py", line 418, in main
    run(
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/main.py", line 587, in run
    server.run()
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/server.py", line 62, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/server.py", line 69, in serve
    config.load()
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/config.py", line 458, in load
    self.loaded_app = import_from_string(self.app)
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/project/code/chain_server/server.py", line 29, in <module>
    from chain_server import chains
  File "/project/code/chain_server/chains.py", line 25, in <module>
    import torch
  File "/home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/torch/__init__.py", line 409, in <module>
    from torch._C import *  # noqa: F403
ImportError: /home/workbench/.conda/envs/api-env/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so: undefined symbol: ncclCommInitRankScalable

I think the issue is suspect to “stat: cannot statx ‘/var/host-run/docker.sock’: No such file or director” . I have docker running and container is started so I know that works, but in WSL terminal I noticed this sock extension file doesn’t exist. Is this a windows issues thing?

Steps to repro:

Install workbench on windows 11, set up WSL and docker
create project and clone workbench hybrid RAG (GitHub - NVIDIA/workbench-example-hybrid-rag: An NVIDIA AI Workbench example project for Retrieval Augmented Generation (RAG))
build project, start container and open chat app.
click on “Set Up RAG Backend”
Next is when the error is observed after scrip times out trying to reach local backend server spun up.

Let me know if anything else here is needed. Not finding much around the internet on this specific issue. Thanks!

Topic		Replies	Views
[SUPPORT] Workbench Example Project: Agentic RAG NVIDIA AI Workbench	30	467	April 7, 2025
NIM embedding model downloads but fails with auth error on startup Access/Accounts nim , nv-embedqa-e5-v5	29	616	April 10, 2025
Batch processing using NVIDIA NIM \| Docker \| Self-hosted Models python , nim , llama3-8b-instruct , llama-31-8b-instruct , llama	11	234	January 29, 2025
RTX 4090 shows as "non-free GPU" when running NIM model in docker AI Foundation Models and Endpoints nim	8	1923	October 21, 2024
Blueprint RAG v2.0.0 NVIDIA Blueprints nim , llama-31-70b-instruct , llama , blueprints	1	32	April 24, 2025
VSS blueprint 2.2.0 - ERROR Failed to load VIA stream handler - Failed to generate TRT-LLM engine Visual AI Agent nim , llama-31-70b-instruct , llama	16	242	April 22, 2025
Quick build guide for visual studio code on the nano Jetson Nano	29	15797	October 14, 2021
Deepstream_pose_estimation repository and provided libnvds_osd.so DeepStream SDK	7	618	April 24, 2023
Cannot load built engine resnet50_market1501_aicity156 DeepStream SDK nvbugs	53	1735	February 14, 2025
Tao toolkit version5 is getting error when comes to training part TAO Toolkit	45	1718	August 22, 2023

[SUPPORT] Workbench Example Project: Hybrid RAG

I have same problem, started creating new project hybrid-rag yesterday. Reached step 29 of 34. Today still at step 29/34. Tried to stop build, get error ’ there was a problem stopping the build for hybrid-rag’. here is the log

Related topics