Unable to Install Models(Mistral / Llama2 / Gemma) via ChatRTX 0.4 UI

[elochen]

Hello :)

I have installed ChatRTX 0.4(from Taiwan). However, when trying to download the Mistral, Llama2, or Gemma models via the ChatRTX UI, the system always shows an error message: “There was a problem downloading Mistral 7B int4 / Llama2 13B int4 / Gemma 7B int4. Please restart ChatRTX and and try again later.” After multiple restarts, I am still unable to download the models properly. Could you please provide a solution? Thank you!

Any suggestion would be appreciated!

My PC:
Win 11 24H2
ASUS 4060TI 16GB
NVIDIA Driver 580.97-desktop-win11-64bit

[bpawar]

@elochen

580.97-desktop-win11-64bit

hi @elochen ,
ChatRTX 0.4 is EOL. ChatRTX 0.5 is the latest version that should work on 4060 Ti. Please try ChatRTX 0.5 from here.

[pawelpac]

Hello,

I have an RTX 5080 graphics card and CHATRTX 0.5. I’m very interested in the RTXCHAT project to search my own PDF files on my computer. Unfortunately, I can’t install any of the models except CLIP (which is for photos).

Is there a solution to this problem? Thanks in advance for your reply.

When I try to install it, I get the following message:

There was a problem downloading Meta Llama 3.1 8b NIM. Restart ChatRTX and try again later

[cody.librock]

Hi mate !! This project has now been deprecated on nvidias end and has a full EOL now. Theyre moving towards using the nim and ai work bench flow. Look up the video on ai workbench rag model and that should run for you. Nvidia.build.com has a run on rtx filter for you to see what should work !