Result of nvidia nims in openai SDK and API inconsistent

dinhtra2000 January 7, 2025, 9:06am 1

Hi all,
i check model llama-31-405b-instruct provide by NVIDIA NIMS : llama-3.1-405b-instruct Model by Meta | NVIDIA NIM
i try use with python SDK and shell script, it quite diffirence result,
Do you have any notes for me regarding preprocessing special characters in strings within the input data?
I think this is reason of inconsistent from python SDK and API
Thanks,
TraDinh

Topic		Replies	Views
NVIDIA NIM Access/Accounts nim , mistral-7b-instruct-v03 , llama-31-8b-instruct , llama , nemotron	0	39	August 10, 2026
OpenAI Compatible API does not work Models llama-31-8b-instruct , llama-31-70b-instruct	6	1157	August 26, 2024
帖子标题 (Title) Request to enable Public API Endpoints for personal organization Access/Accounts nim , codestral-22b-instruct-v01 , mistral-7b-instruct-v03 , mistral-large , nemotron-4-340b-instruct , nemotron-4-340b-reward , llama , deepseek , nemotron	0	63	July 3, 2026
Nvidia / llama-3.1-nemotron-70b-instruct openai api is not working TensorRT llama	1	472	November 10, 2024
Models That Don't Work Models nim , codestral-22b-instruct-v01 , nemotron-4-340b-reward , llama , deepseek , nemotron	0	184	July 25, 2026
NVIDIA NIM API invoked by Langchain returns statuscode 500 Access/Accounts nim , llama-31-70b-instruct , llama	1	517	September 4, 2024
NIM HTTP API Inference (Run Anywhere) Taking Extremely Long! Models nim , llama-31-70b-instruct , llama-31-405b-instruct , llama	1	927	September 11, 2024
Token limit defaults to 4096 for all models Models nim	1	326	May 23, 2026
Open AI Endpoint NVIDIA Nemotron	0	373	April 28, 2024
Request to enable "Public API Endpoints" for my account Access/Accounts nim	0	40	July 25, 2026

Result of nvidia nims in openai SDK and API inconsistent

Related topics