I’ve download the original Deepseek 4 Flash 0731 from Hugging Face, but a faceoff against the cloud version consistently show the Hugging Face version performing worst than the cloud version in various test benches. Is deepseek giving us the real deal ?
Good local quant outperforms cloud version every single time. Make sure your inference is configured properly, check the numerous threads here
Not sure where you got the idea. Go do a full bench, not just a tools calls, not off some 3rd party sites (e.g. ollama) that offers the “same model” , feelings don’t count .
Sure. Whatever floats your boat. Peace.
There you go:
HumanEval Benchmark — DeepSeek-V4-Flash-0731
Local vLLM vs OpenRouter
Benchmark: HumanEval (openai/openai_humaneval), 164 problems, pass@1 (official verify scorer)
Harness: Inspect (inspect-ai 0.3.253, inspect-evals 0.16.0)
| Endpoint | Version / Serving | Sampling | Pass@1 |
|---|---|---|---|
Local vLLM 192.168.1.151:8888 |
vLLM 0.1.dev19023+g30038602b.d20260805 (eugr/spark-vllm-b12x image, 2×DGX Spark TP2, 200 Gb/s CX-7) |
temp 1.0, top_p 0.95, reasoning=max | 158/164 = 96.3% |
OpenRouter deepseek/deepseek-v4-flash-0731 |
hosted | temp 1.0, top_p 0.95, reasoning=max | 156/164 = 95.1% |
| OpenRouter (sensitivity) | hosted | temp 0.8 | 149/164 = 90.9% |
Model: deepseek-ai/DeepSeek-V4-Flash-0731 @ rev 7872f01b1d1fe23eabc4c98b48bffcef5a386062
Local vLLM serving params: max_model_len auto → observed 1,048,576;
max_num_batched_tokens=4096; max_num_seqs=6; long_prefill_token_threshold=1024;
prefix-cache retention 4096; partial-prefill 1/1/0; DSpark k5.
Run params (both): temp 1.0, top_p 0.95, reasoning=max, max_tokens=32768, client_timeout=7200; local concurrency=1,
OpenRouter concurrency=10. 0 errors / 0 timeouts on all legs.