Up to 70tok/s Qwen3.8-Flash-Next-Int4-AutoRound

Tool-eval-bench has a knowledge benchmark:

GSM8K benchmark:
–gsm8k Run GSM8K (Grade School Math) benchmark after tool-call scenarios
–gsm8k-only Run ONLY the GSM8K benchmark (skip tool-call scenarios)
–gsm8k-shots N Number of few-shot CoT examples (0–8, default: 8)
–gsm8k-limit N Max questions to evaluate (default: 200, 0 = all 1319)
–gsm8k-shuffle Shuffle question order (uses --seed for reproducibility)

MMLU benchmark:
–mmlu Run MMLU (Massive Multitask Language Understanding) benchmark
–mmlu-only Run ONLY the MMLU benchmark (skip tool-call scenarios)
–mmlu-shots N Number of few-shot examples per subject (0–5, default: 5)
–mmlu-limit N Max questions to evaluate (default: 500, 0 = all 14042)
–mmlu-subjects LIST Comma-separated subjects or categories (e.g. ‘STEM,abstract_algebra’)

IFEval benchmark:
–ifeval Run IFEval (Instruction Following) benchmark
–ifeval-only Run ONLY the IFEval benchmark (skip tool-call scenarios)
–ifeval-limit N Max prompts to evaluate (default: 0 = all 541)

I don’t run it since I don’t care much about the knowledge the model has, but more of what capabilities does it hold. I don’t ask the AI for answers (other than code that most benchmarks can do well) but to do stuff instead. I do care about Mathematics so I generated my own benchmark for it so it can distinguish not only Model A vs Model B, but also Quant 1 vs Quant 2 within the same model.