GadflyII/GLM-4.7-Flash-NVFP4 was just released on HuggingFace. However, its requirements are vLLM 0.14.0 and Transformers 5.0.0. The GitHub for vLLM release is currently 0.13.x, and Transformers is 4.x. I currently have a working GPT OSS 120B setup through Docker compose, but can’t seem to figure …

FYI. I released the images at New pre-built vLLM Docker Images for NVIDIA DGX Spark ; I don’t think GLM-4.7-Flash works on the latest image, but I was planning to release a new version tomorrow coinciding with pytorch 2.10 release. I don’t think vllm 0.14.0 release will work; I think a key commit ca…

I was getting confused about assigning IP addresses to both enp1s0f1np1 and enP2p1s0f1np1 What is the actual difference between the two, and why should I choose one over the other. That’s what’s missing in the documentation. docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 172.17.0.1 n…

[image] brian322: enP2p1s0f1np1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 inet 192.168.200.16 netmask 255.255.255.0 broadcast 192.168.200.255 inet6 fe80::32c5:99ff:fe3e:b0e9 prefixlen 64 scopeid 0x20 ether 30:c5:99:3e:b0:e9 txqueuelen 1000 (Ethernet) RX packets 6438 bytes 1427740 …

I am also very eager to get 4.7 Flash running for local development in my homelab. Got an ASUS Ascent for a few days now. I started from the same docker and made some minor changes to vllm to allow for MLA usage with Triton and also changed the num_kv_splits. By default it is set to 4 so on long con…

I also tried to make flashinfer work with it for prefill and got even faster speeds but then turned into gibberish. I have to admit this lower level stuff is not my usual work, so I am slowly trying to get into it. I mostly worked with pytorch, but recently feels more like full stack development aro…

ok so ran a longer test with full 200k context, this took quite a while: vllm serve GadflyII/GLM-4.7-Flash-NVFP4 –gpu-memory-utilization 0.85 –tool-call-parser glm47 –reasoning-parser glm45 –enable-auto-tool-choice –served-model-name glm-4.7-flash –max-model-len 202752 –max-num-batched-token…

Oh boy, this testing really takes a long time for 100k+ contexts. For now this is the best I could find, I think this is quite the speedup for long contexts considering I only made some slight alterations in vllm. I’ll create a comprehensive report and way to reproduce and test easily when i find t…

Interesting For general use on single Spark machine, what max-model-len and gpu-memory-utilization would you recommend?

Keen to see more details on how you got these improvements, as I’m struggling with exactly the same setup.

One last test, this one was also interesting. Confirms the bottleneck most likely is kv cache kernels for sm_121 that support MLA. Maybe Ill run the full benchy over night, but you can see the numbers are still way better for long context even with the original weights from z-ai, short context suffe…

Ill just give you the quick and dirty commands, then will compile some info another day, the fixes are quite small actually: I use the standard container from scitrera no magic on my side docker run -d --privileged --gpus all --rm --ipc=host --network host \ --name glm \ -v ~/.cache/huggingfac…

GLM-4.7-Flash-NVFP4 was just released, but for Transformers 5.0 + vLLM 0.14...?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

eugr January 30, 2026, 6:17am 85

BTW, I’ve posted in the other thread, but I incorporated @mjpansa’s improvements (the essential one, at least, the other one is properly solved by using Transformers 5 version) into spark-vllm-docker as a mod. Mods can now be used on a single node too, not just in cluster: Make GLM-4.7-Flash go BRRRRR - #3 by eugr

Topic		Replies	Views
Help: Running NVFP4 model on 2x DGX Spark with vLLM + Ray (multi-node) DGX Spark / GB10 mistral-large	18	2845	December 25, 2025
Two-Spark cluster with vLLM using tensor-parallel-size 2 causes one node to drop while the other's GPU goes 100% forever DGX Spark / GB10	36	2163	February 13, 2026
vLLM on GB10: gpt-oss-120b MXFP4 slower than SGLang/llama.cpp... what’s missing? DGX Spark / GB10	143	8182	February 24, 2026
New bleeding-edge vLLM Docker Image: avarok/vllm-nvfp4-gb10-sm120 DGX Spark / GB10 Projects	32	3412	December 17, 2025
PSA: State of FP4/NVFP4 Support for DGX Spark in VLLM DGX Spark / GB10	234	14176	May 15, 2026
We unlocked NVFP4 on the DGX Spark: 20% faster than AWQ! DGX Spark / GB10	144	9811	March 14, 2026
Make GLM-4.7-Flash go BRRRRR DGX Spark / GB10	18	2815	March 25, 2026
vLLM containers DGX Spark / GB10	45	2746	July 22, 2026
Install and Use vLLM for Inference on two Sparks does not work DGX Spark / GB10	159	5995	December 9, 2025
Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 nim , llama	177	13024	April 16, 2026

GLM-4.7-Flash-NVFP4 was just released, but for Transformers 5.0 + vLLM 0.14...?

Related topics