DeepSeek-V4-Flash, GLM-5.3-Flash, Qwen3.8-Flash-Next

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

as not expert, can someone guide me to the best option to deploy the below models on my single dgx spark on vllm
DeepSeek-V4-Flash,
GLM-5.3-Flash,
Qwen3.8-Flash-Next

any straight forward way to deploy and get it running

2

Big context 1M and good speed 45-60 t/s tg and 2000 t/s pp, general good intelligence - > deepseek
Big context 1M and tolerable speed ~25-35 t/s tg and 900 t/s pp, better intelligence - > glm
Medium context ~500K, semi-decent speed ~35-40 t/s tg and 2500 t/s pp, front end and visual specialist - > Qwen 3.8 Flash Next

3

Can you give me guide how to deploy?

4

Search respective threads, plenty of recipes. Which one you want? I have repos for qwen and ds4f on github 0rand, glm I use MiaAI repo with no changes bar env settings

5

Qwen - > GitHub - 0rand/qwen3.8-flash-next-2x-dgx-sparks: Generalized Qwen3.8-Flash-Next FP8 TP2 serving stack for 2x DGX Spark (RoCE) — one .env, compose head/worker, start/stop/status/download · GitHub
Deepseek - > GitHub - 0rand/DeepSeek-v4-DSpark-Aidendle94-GB10-ServingStack: Docker compose serving stack for DeepSeek v4 Flash DSpark for NVIDIA Spark GB10 system using Aidendle94 image · GitHub
Glm - > GitHub - MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks: GLM-5.3 Flash EXL3 for 2x DGX Sparks · GitHub

6

all of them require 2 sparks and i need to deploy on 1 spark only

7

Well, search one spark threads, do the work - @entrpi has many top notch 1 sparks serving stacks. I would chose 3.8 27b in fp8 for 1 spark

8

Entrpi’s GLM 5.3 Flash stack is also superior to the MiaAi stack which is broken (single stream only, bad PP, excessive drafter pool reservation).
The entrpi stack is both faster at PP as well as TG, does C6 without issues and has comparable to larger KV pool size.

9

can you provide the glm topic link?

10

Sorry it’s 2 x GB10 and it doesn’t have a dedicated topic yet: GitHub - Entrpi/glm-5.3-flash-exl3-2x-spark: GLM-5.3-Flash (EXL3 4bpw + DFlash2) on 2x NVIDIA DGX Spark: one-shot installer, 33-74 tok/s c1, 1.3M+ context, vision · GitHub

11

it requires 2 sparks, i am looking for 1 spark option

12

If you want to do Deepseek I wouldn’t advise those threads anymore, despite being very thankful for their efforts.

The ones in this topic are just plain better in my experience. One of them gives you a decent quant of DSV4F with full 1m context somehow. Another gives you 250-400k context on a EXL3 quant (going to assume it’s some mixture of Q3ish). From how much I’ve used them, I’d say they’re just generally much better than any of the Q2 quants in that topic. They use VLLM to serve and are just plain faster. Incredibly stable, too, had one running for a full week doing tasks.

I’m currently trying this executable variant of the 0xSero quant, which includes a few patches. Startup is insanely simple:

It’s a little slower than the base 0xSero build (which ime reached up to 60 tok/s), but supposedly more stable and you get 400kish context vs 250ish.

13

Yet multiple reports that head node hangs during the weight load. I stopped it right there. I will try this one next - confirmed working by fellow GLM-5.3-Flash NVFP4 (lab quant) — 92/100 tool-eval, reproducible build for 2x DGX Spark (GB10/sm_121), vLLM day-0 + patches, MTP k=3, 512K ctx · GitHub

Its a different checkpoint but I downloaded it overnight

14

ELX3 is a packaging format, can be 4 bits, can be 2 bits, like INT or FP

15

Yeah I am definitely out of my depth when talking about the intricacies of the models… very much layman level.

Regardless, I do think that these models are the way to go for single spark DSV4F. I was wondering how they’d do on your bench. A lot of people report way better results than the Q2 quants with them on actual work.

For OP, there is also Ling 3.0. I haven’t tried it yet, but that’s another single spark model with solid performance supposedly.

16

Ling is actually quite decent daily driver if 27b is too slow. Much better than Laguna S2.1. But I would focus on setting up 3.8 27B to serve fast. It’s really incredible. Yesterday and today I forewent on using any cloud AI to setup my serving stack for GLM and went to use 3.8 27B on my M5 Max - zero problems, it did everything very well. Yes reasons a lot, but switch to low mode for regular tasks - more than capable.

17

Look into the Qwen 3.8 Next Flash Single Node discussions, there is a repository with a single node that is installed with just 1 command. If you got a PC that is able to run Unsloth,etc or has Claude Code or other cloud . The following prompt is generated by my Hermes.

You are an autonomous installer operator. Your job: perform a clean, verified installation of the Qwen3.8 serving stack on a DGX Spark (NVIDIA GB10, 128 GB unified memory) running stock DGX OS. Execute every step yourself against the live host. Do not fabricate output — report only what the tools actually return.

TARGET SETTING

PREREQUISITES (verify first, fail loudly if absent)

  1. Host reachable; you have SSH (or working shell) as a root/privileged user.
  2. docker + NVIDIA Container Toolkit present (stock DGX OS guarantee).
  3. Free disk >= ~230 GB on the flash target: ~195 GB under $HOME (~136 GB checkpoint + ~48 GB PLE mmap backing file written at first boot) + ~35 GB Docker partition (~30 GB image).

INSTALL PROCEDURE

  1. Run the canonical one-liner (install AND update path; clones or updates ~/dgx-spark-qwen38, then runs the pinned installer):
    curl -fsSL https://raw.githubusercontent.com/hasso5703/dgx-spark-qwen38/main/get.sh | MODEL_CHOICE=flash bash
    (Env options ride the BASH side of the pipe, not an env prefix on curl.)
  2. If the systemd service install prompts for sudo, provide it. Expect the installer to:
  • Fetch checkpoints (RadixArk/Qwen3.8-Flash-Next-NVFP4 @ pinned revision)
  • Build the flash serving image (pinned base digest + sha256-verified overlay files)
  • Write the API key + patched chat template
  • Install the qwen38-flash systemd service + opencode provider config
  1. First boot takes ~15 min (kernel compile + writes the 48 GiB PLE backing file + CUDA graph capture; later boots ~10 min). Watch the installer’s “still loading…” progress; do not abort.

VERIFICATION (required — do all)

  1. Container up: docker ps --filter name=qwen38-flash shows image qwen38-flash:v1.5.3, status Up.
  2. Health: curl -s http://127.0.0.1:30000/health returns empty 200.
  3. Models: curl -s http://127.0.0.1:30000/v1/models lists qwen3.8-flash-next.
  4. Scheduler-hang fix flag live: docker inspect qwen38-flash Env contains SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1.
  5. Real generation: POST to /v1/chat/completions with model qwen3.8-flash-next, Bearer key from ~/.config/qwen38/api-key; expect a non-empty completion (reasoning_content included) and response usage.
  6. Report the API key PATH (never print the key value).

FILE-CHANGE DISCIPLINE

  • Do not hand-edit the installer, unit files, or launch template to fix a boot hang — trivial tweaks won’t resolve a known repo issue. Re-run the pinned installer instead.
  • Do not edit files outside $HOME/dgx-spark-qwen38 and the paths the installer writes.

REPORT FORMAT (when done)

  • Installed image + tag, model id, repo commit
  • Verified endpoint URL(s) + health/models/generation results
  • API key path (key value redacted)
  • Any caveats (first-boot duration, serving gaps)
    ═══════ END PROMPT ═══════
18

So overall Deepseek v4 flash still winds on 2x Sparks?

19

For interactive work - absolutely imo. If if have a large task with many steps you can research and build a detailed plan with deepseek, file it, boot glm and leave it overnight or take a hike while it works. It’s really good being autonomous.

20

I havent upgrade from 122B yet , now thinking . I have 3x DGX SParks so i think :
2 x DGX Spark running V4 Flash
1 x DGX Spark running Qwen 3.8-Next-Flash NVFP4 with vllm patch by @ siertum

What do you think of that setup?
Planner : Deepseek , Coderder Qwen 3.8