KVarN: Native vLLM backend for KV-cache quantization by Huawei

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

⚡️ Built for agentic and long-context workloads.

💡 KVarN delivers 3-5x more KV-cache capacity and up to ~1.3x the throughput of FP16, so you fit far longer contexts and serve more concurrent requests, with FP16-level accuracy.

🔌 Calibration-free, plug-and-play with vLLM. A native vLLM attention backend: add one flag, no model changes, no calibration.

🥊 Up to ~2.4× TurboQuant throughput, same capacity, higher accuracy.

2

Have you tried ? It doesn’t work with QWEN 3.6 , so the whole “no model changes” isn’t exactly accurate.

3

It doesn’t surprise me. There are so many moving parts in vLLM, that I do not expect any third-party component to successfully undergo comprehensive regression tests. That being said as Qwen 3.6 is still relevant model, it would be good to bring this to the attention of the KVarN authors.