Great! I will try it at once.
Been using mias Tensorfold build for GLM5.3 with opencode v2. Appears to work very well, concurrency is much better for sure.
I’ve been a little busy over the past week but may have some time next week to make this happen. If anyone wants to submit a PR beforehand: I’ll happily consider it!
Edit: feat: support TensorFold discovery and benchmark metrics by SeraphimSerapis · Pull Request #190 · SeraphimSerapis/tool-eval-bench · GitHub - available via the main branch, landing in the next stable release
I am able to run Qwen3.8 27B MLX 4-bit + DFlash2 with TensorFold and it has higher throughput than my previous setup of NVFP4 +DFlash2 with vLLM. Pretty impressed with the performance of running the dense model on the spark.
Metrics dash for TensorFold (and VLLM too)
I just fully tested it on Qwen3.8-Flash-Next:
Tl;dr? Qwen3.8 flash next / glm5.3 flash
I’ve got @MiaAI_Lab’s Qwen 3.8 Next Flash and GLM 5.3 recipes both running on 3 of my 4 sparks. Performance is great but my benchmarks are keyed specific to my throughput and custom harness build. I did run standard quality tests on both models and they were both over 90% and perfect on needles. Both were 40-50% increases in C1 sliding gracefully down the curve to c=8.
Only oddity I noted was TensorFold does not provide a seed on prompts.
I pointed Claude at TensorFold and told it to implement all of it’s speed ups into VLLM and I am hitting the same speeds without all of the TensorFold headaches. I recommend doing the same, I’ll update my GLM 5.3 repo to have these updates soon.
Is it also going to be EXL3? I really like the quant this variant uses.
Well, that score actually looks pretty low for GLM 5.3F though. Might just be an unlucky run though.
That’s on Normal mode, not Hard mode. Try something like this:
tool-eval-bench run --hardmode --seed 42 --max-turns 32 --timeout 300 --parallel 4 --backend-kwargs '{"chat_template_kwargs": {"thinking": true, "reasoning_effort": "max"}, "temperature": 0.1}' --trials 5
I think others have noted that it requires some patched version of the eval bench though, to properly use this engine somehow.
Dev11 is the patched one, only affects performance (prefill) due to incompatibility in llama-bency (tailored to vlln not canonical openai metrics)
Similar to what I came up with as well. Mine was 7 partial and 5 fails. But I do not really trust the overall results do to the incompatibly issues. I ended up trying another benchmark system just so I could give it a fair a/b against the Int4 - vLLM v29 version.
It’s regular openai endpoints, what incompatibility? There is an issue with llama-benchy not properly counting prefill but what is has to do with hardmode test, besides TEB already been patched. Don’t spread the rumors please
Perfect, I must have tested with TEB prior to the 2.7.1 patch :-)
I am running MiaAI’s GLM 5.3 Flash Tensorfold recipe running on two sparks. Not getting nearly the numbers they report in their repo but I’m able to get 83 tok/s single stream on structured output, it feels substantially faster than Deepseek V4 Flash Exp Vision did. Very impressed.
I was very impressed with the speed, and coding was fine… but it just kept looping on my document processing task for some reason. Not 100% of the time, but frequently. GLM itself (when I asked it to investigate) said that it was some prompt templating issue because this looping is an actual bug with the official upstream release of the model. So the quant doesn’t really matter here, there’s something about the engine that causes this to happen with certain payloads at high context.
The looping type was long form, with it arriving at the same paragraph and analysis over and over and over again, ad infinitum. The problem with Tensorfold is that it’s also deterministic. Unless you set a random seed, an input will fail the exact same way every single time.
That’s about my experience with the engine right now.