Error 504 when calling the model through api

AI & Data Science NVIDIA NIM Access/Accounts
1

Is there anyone also suffering from unable to call the model? Even using the demo code with api key gave the same result error 504 ( as I didn’t find information about where can i check the status of the NIM service, im not sure whether its down for now or what. Tried diffierent account and re-generating the api key. not working. Any update will be appreciated. Thx

2

Hi @aaronfeng.lw14, which model was this? Are you still seeing the 504 error?

3

hi @neal.vaidya , sorry for missing the information, it was yentinglin/llama-3-taiwan-70b-instruct, i having using it after I found its just the specific model keeps failing so im not sure what is the current status. Thank you for looking into it.

4

Hello @aaronfeng.lw14 Could you re-confirm if you are still facing this issue? If not, will have to re-deploy the model.

5

Hi @abodhankar , i tested the model with sample code on the model page as follow, the problem still exist, ty.

openai.InternalServerError: Error code: 504

from openai import OpenAI

client = OpenAI(
base_url = “https://integrate.api.nvidia.com/v1”,
api_key = “”
)

completion = client.chat.completions.create(
model=“yentinglin/llama-3-taiwan-70b-instruct”,
messages=[{“role”:“user”,“content”:“cheese burger”}],
temperature=0.5,
top_p=1,
max_tokens=1024,
stream=True
)

for chunk in completion:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end=“”)