From the guys who brought us DFlash, this looks kinda interesting. They’ve released a bunch of models on HF, I’m downloading Qwen 3.5 27B one, will check how it compares against FP8. [image] ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM... ParoQuant: Pairwise Rot…

How was results on them? I just found them and was interested too

Sorry, forgot to update - seemed pretty close to FP8 on accuracy, but unfortunately it did not work with TP=2 for me and also seemed to lack DFlash support, so I went back to running FP8 instead.

It’s important to note that paroquant is kind of an orthogonal optimization strategy. Prismaquant optimizes formats based on sensitivity – but we also do things like paroquant – GPTQ, HALO, (QuaRot), outlier sweeps, and a closed-form variant of Intel Autoround. If paroquant ends up being great, we …

Given that its results with Qwen 3.6 27B model werent as good as PrismaQuant, I havent bothered looking at it any further. Performance wise it also seemed to be lagging behind FP8 / BF16 speeds, but that might have been due to lack of TP >1 and Dflash support.

I have actually made a fork, and may try to add some features from various other places I have found, with the aid of AI. hope to get round to starting in the next few days, but unfortunately I can’t guarantee this. My plan is here if you are interested: prismaquant/docs/cross_repo_quantization_idea…

It’s not going to be a great direct port because it’s locked to int4 today. If you do spend some time with it, replace the HALO path

I tried ParoQuant for Qwen 3.6 27B and I somehow get 2x slower prefill speed than on Prismascout or Prismaquant. llama-benchy Averages Metric PrismaSCOUT PARO Delta Runtime 0:04:50 0:07:20 +2:30 slower Est latency 150.1ms 141.9ms slightly better Avg pp t/s 2,163 997 -54% Avg tg t/s 6…

Yep, I had the same result, so havent bothered continuing with PARO. Wonder if EXL3 is worth exploring on our sparks, its meant to offer higher accuracy per byte than PARO: [image]

I did some work here and ultimately came to the conclusion that rotation-based schemes are NOT useful for NVFP4. I found a paper that basically validated what I determined experimentally. The larger the block-size, the more useful rotation methods are because larger blocks capture more outliers. Sma…

Has anyone tried any of ParoQuant quantized models?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

norman.2 May 9, 2026, 6:23am 4

If its not even as good as FP8 you should probably have a look at Introducing PrismaScout -- PrismaQuant v2!

Topic		Replies	Views
Introducing PrismaQuant DGX Spark / GB10	165	6098	May 23, 2026
Introducing PrismaScout -- PrismaQuant v2! DGX Spark / GB10	87	5264	June 9, 2026
Benchmarks for Qwen3.6 FP8 vs PrismQuant DGX Spark / GB10	6	1787	April 30, 2026
What's the best speed we can get with Qwen 3.6 27B without quantizing? DGX Spark / GB10	30	14514	June 7, 2026
NVFP4 quantization of a 100B-class Llama on 2× DGX Spark — lessons + open questions DGX Spark / GB10 llama	5	369	May 15, 2026
Best Q4 / NVFP4 model for quality Qwen3.5-27B or alternatives? DGX Spark / GB10 llama , deepseek , nemotron	16	3505	April 26, 2026
Why Turboquant saves DGX twice DGX Spark / GB10	134	11470	May 31, 2026
PSA: State of FP4/NVFP4 Support for DGX Spark in VLLM DGX Spark / GB10	234	12632	May 15, 2026
FP4 on DGX Spark — Why It Doesn't Scale Like You'd Expect DGX Spark / GB10	213	6344	March 13, 2026
Post-Training Quantization of LLMs with NVIDIA NeMo and NVIDIA TensorRT Model Optimizer Technical Blog	0	92	September 10, 2024

Has anyone tried any of ParoQuant quantized models?

Related topics