You’re right on that:
tool-eval-bench
New local-inference-lab quant: 89/100 — 156/176 points
Previous LibertAIDAI quant: 85/100 — 149/176 points
[ Z.AI FP8 quant: 90/100 ]
Improvement: +4 score, +7 points
Outcomes: 73 pass, 10 partial, 5 fail
Largest category changes: P +3, K +2, O +2; H and I −1 each
By the way, VLLM for this quant needs a slight fix in modelopt.py, because MTP layer are stored under language_model.model.layers.45... but DSv4F fixes that without issues.