- Application: Both NIM Models website and Hermes Agent
- Model: Kimi K2.6 (via NVIDIA NIM)
- Context Limit: 256k tokens
- Notes: Hour-Of-Release on NIM API, expected bugs
- Severity: High. I say this as it is entirely unusable when doing basic tasks.
When querying the Kimi K2.6 via standard API calls, the model’s reasoning trace occasionally breaks and enters an infinite repetition loop. Mid thought, it gets stuck on a single low-probability token (specifically “!”) and repeats it endlessly until the entire 256k context window is completely exhausted. I expected Kimi K2.6 to have bugs on release on NIM, and as I am writing this, Im in the exact hour of release on NIM.
Example:
me: how do i setup mem0 for use with hermes agent locally on my raspberry pi 5 8gb, do research please
Kimi’s Thinking:
… (lots of thinking, of course)
"pip3 install mem0ai chromadb --break-system-packages
Or use a venv (recommended)
!!! (~50,000 characters when unstopped, will not stop)
Follow Up, heres how it looks.
I also have this issue, it often happens
dpesnt happen to me
though in my case i barelt got any response at all with 502 and 504 but output is fine
you can try repetition penalty man
set it to 1.1 to 1.2
also tried on the web ui its fine
I’d recommend using a more complex problem or repeating followups
i see the promblem now
using your exact prompt including case sensitivity Well can you tell me how openAI's chatgpt works do output infinite exclamation mark
but if you use Well, can you tell me how OpenAI's ChatGPT works? instead it works just fine bro
same problem in API when you send exact same prompt
but i have confirmed adding "frequency_penalty”: 1.1 in the API completely fixed it man
before:
“reasoning”:" The!!!
after:
“reasoning”:" The user is asking for an explanation of how OpenAI’s ChatGPT works. This is a general knowledge question about AI/ML technology.
Ah I see. Can this be added on the webUI? Might be a stupid question but I need to know.
Also, I’d like to note that the bug is very hit-or-miss. Sometimes it works and sometimes its ‘!!!’
I thought I fixed it as well when I was tinkering with temperature and some other settings, but it simply just delayed it a bit.
I’d like to have a sure-set solution that patches out the bug rather than delaying it and is something that can be implemented A.S.A.P. by an NVIDIA Dev.
i tried it again and freq pen as low as 0.4 fixed it realiably
0.4: 4/5 coherent
0.3: 3/5
0.2: 2/5
its honestly not that bad as only that specific prompt with specific punctuation and case sensitivity that break it though even if you capitalize a single character to OpenAI’s it will output correctly
Hmmm. I will have to try that.
Update, I’m unable to replicate any success. Its still bugging out and spamming !!!, but now not only does it do it in its thinking, but also if it completes its thinking, the !!!'s are to be found in the main message.
Notes;
Wording is very odd, certain phrasing of the same points can cause it to succeed with or without failure
Adjusting Temperature has an effect, usually positive, however it feels as it is delaying the inevitable**.
Just as adjusting the temperature does, (but at a greater scale), adjusting** frequency_penaltydoes the same, this is thanks to the findings of JohnDeere
If you are able to fix the model, I’d recommend doing your own testing while keeping these observations in mind. Fixing this model A.S.A.P. would be VERY greatly appreciated as Kimi k2.6 is the best model on NVIDIA’s NIM API catalogue at this time.
It has not been fixed at all
Update, I have been using the model for a bit.
It’s not fixed.
Sorry for the confusion
Confirmed to be still a prevalent issue as of this moment, happening ~1/3 times you ask it to think.
still happening, and deepseek pro too has problems with thinking strange letters, so i dont think they will solve this, sad
Yeah, in their current state Kimi 2.6 and Deep Seek v4 pro API and Web UI are unusable from the platform.
