Internal Server Error. Try Again

AI & Data Science NVIDIA AI Workbench
1

Internal Server Error. Try Again
To double check I switched off Firewall, Virus and other Defender features.

Submission Type
Error

Workbench Version
Desktop App v0.44.8

Host Machine operating system and location
Local Windows 11

LOGS:
*comment: I don’t think it produced proper log on this issues. Down below is a log on another issue that I reported earlier - ERR: Unable to process query.

{“level”:“info”,“time”:“2024/05/21 - 09:25:57”,“status”:200,“latency”:“13.692µs”,“client-ip”:“127.0.0.1”,“method”:“OPTIONS”,“path”:“/v1/query”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“GIN-Request”}
{“level”:“info”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“Processing git status output”}
{“level”:“info”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“Processing git diff output”}
{“level”:“info”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“Processing git diff output”}
{“level”:“info”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“Processing git status output”}
{“level”:“info”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“Processing git diff output”}
{“level”:“info”,“time”:“2024-05-21T09:25:57+10:00”,“message”:“Processing git diff output”}
{“level”:“info”,“time”:“2024/05/21 - 09:25:58”,“status”:200,“latency”:“1.174839448s”,“client-ip”:“127.0.0.1”,“method”:“POST”,“path”:“/v1/query”,“time”:“2024-05-21T09:25:58+10:00”,“message”:“GIN-Request”}

2

Hi Alex,

I’m assuming this is referring to an issue with the Hybrid RAG example project? If so, the support megathread is here.

You are viewing the Service logs. Select the dropdown on the left hand side of the Output widget, scroll down, and select “Chat” under applications. You can then view the proper logs there.

Looks like you are running into an issue with starting the local inference server? Here are some common issues we see users facing:

  1. Authentication. Make sure you are either using an ungated model (nvidia/Llama3-ChatQA-1.5-8B), or have your Hugging Face token configured as a project secret with access to the gated models you are interested in running.

  2. Out of memory. Make sure you are running locally on a GPU enabled system and that your GPU is not running any other processes that may interfere with running the model.

  3. Timeout. The most common issue we see is the application times out on server start. The default timeout is set to 90s but depending on hardware you may need more time. You can edit this in code under code/chatui/chat_client.py, scrolling to the bottom and extending the timeout.