The models on the NVIDIA platform have been pretty unreliable.
I have been debugging repeated model-specific failures on the hosted Integrate OpenAI-compatible endpoint:
https://integrate.api.nvidia.com/v1/chat/completions
This does not appear to be a general credential, connectivity, or account-wide outage.
Related detailed posts:
- z-ai/glm5 on Hosted Integrate appears unhealthy: repeated direct requests stall with zero bytes returned
- moonshotai/kimi-k2.5 on Hosted Integrate returns success-shaped failures: repeated HTTP 200 responses with unusable content
- minimaxai/minimax-m2.5 leaks reasoning into choices[0].message.content on /v1/chat/completions; larger max_tokens only masks it
Control / baseline:
- We repeatedly tested a baseline model: meta/llama-3.1-8b-instruct
- Those requests returned 200 OK with exact “OK” content
- So the hosted platform is reachable and basic authenticated inference is working
The failures are model-route-specific and reproducible across repeated independent requests:
- z-ai/glm5
- Repeated direct requests timed out with zero bytes returned
- We allowed very large timeouts, including 150-second windows
- This looks like route-level outage, extreme queueing, or severe first-token latency rather than a simple client bug
- moonshotai/kimi-k2.5
- Repeated direct requests often returned HTTP 200 but with unusable output
- In different runs we saw:
- empty or null content
- reasoning-only output
- finish_reason=“length”
- In other runs the route also stalled or timed out
- So this looks like both route-health instability and success-shaped semantic failure
- minimaxai/minimax-m2.5
- Repeated direct requests were intermittent
- Some returned exact OK
- Some timed out with zero bytes
- Other runs returned malformed 200 responses where message content began with reasoning text
- So this route appears to have both reliability and response-shape problems
Additional concerns:
- /v1/models is not a reliable indicator that a model is actually healthy or callable via /v1/chat/completions
- Across this and other forum threads, we have seen:
- blank/no-output 200s
- models listed but not reliably usable
What I did on my side:
We built defensive proxy-side mitigations to contain the damage:
- strip unstable or unsupported parameters
- force buffered normalization instead of trusting raw streaming
- reduce or disable reasoning where supported
- floor max_tokens for problematic models
- detect blank/null-content 200s and leaked reasoning
- retry or fail closed instead of passing malformed success bodies downstream
These mitigations reduce downstream breakage, but they are containment only. They do not fix the underlying hosted route problems. Specifically, our proxy can fail these routes closed rather than passing garbage downstream, but the remaining issue is upstream: the routes themselves return unusable or misleading responses.
What we need from NVIDIA:
- permanent fixes for glm5, kimi-k2.5, and minimax-m2.5 on the hosted Integrate chat-completions surface
- do not return 200 for blank/null/unusable completions
- do not list models in /v1/models unless they pass real chat-completions health checks
- fix parser / reasoning / tool-call correctness on the OpenAI-compatible surface
- publish a clearer supported-endpoints and supported-parameters matrix per hosted model
- acknowledge whether these are known incidents or known model-specific limitations
- provide some model-health visibility into /v1/models or a separate health endpoint
If useful, I can provide exact timestamps, request IDs, and minimal curl repros for each model.