Qwen-Image-2.1 via vLLM-OMNI

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

Hi!

For those who would like to play around with vLLM OMNI for image generation and the newest Qwen-Image-2.1 like me, I would like to share the following files for creating a vLLM OMNI docker image for our sparks.

For transparency: it’s a side product of testing agent harnesses. It started with Hermes Agent Desktop (using DS4F on a regular x86_64 linux system with RTX 5090) and after a successful build on that platform, I failed to reproduce it on my Spark. So that the fix for the GB10 version was made by Claude Desktop.

But back to the original topic. The official release does not support Qwen/Qwen-Image-2.1 yet.

But there is already a pull request open - [New model] Qwen-image-2.1 support by Isotr0py · Pull Request #7759 · vllm-project/vllm-omni · GitHub - to enable vLLM OMNI.

My first approach was working for the official vLLM image on a RTX 5090, but not for the GB10 chip with my my favorite community image. That is not “standard” - so it needs some patching which is done in the Dockerfile. More details in the comments of the Dockerfile.

It is using the 20260921 build - Package dgx-vllm-eugr-nightly-b12x · GitHub - as base - that version was tested on my spark. Later versions might break, but at some point the Qwen-Image-2.1 will land in the official release.

Just copy the Dockerfile and docker-compose.yml onto your spark and start it with:

docker compose up -d --build

For adjusting the port and/or adding a HF_TOKEN create a .env as in:

PORT=8091
HF_TOKEN=hf_asUcanSEE…

and you will have an endpoint on port 8091 to play with.

For better prompting have a look at: GitHub - kjranyone/qwen-image-2.1-prompt-guide: Agent Skills for Qwen-Image-2.1 · GitHub

Have fun playing with Qwen-Image-2.1!

Best regards, cos

2

I ran the same model on a Spark through plain diffusers (bf16, no container) and posted timings here: Qwen-Image-2.1 on DGX Spark: 54 s per 1024² image (40 steps), 31.6 GB, and a fix for the pink vertical line

1024² / 40 steps: 54 s, ~31.6 GB. Did you get any numbers out of vLLM-OMNI? Curious whether it beats diffusers on the GB10.

3

With 1024x1024, 40 steps - around 40s using ~30 GB.

(DiffusionWorker pid=1) INFO 10-03 08:44:44 [stats.py:837] | e2e_total_ms | 39,968.537 |
(DiffusionWorker pid=1) INFO 10-03 08:50:55 [stats.py:837] | e2e_total_ms | 40,297.785 |
(DiffusionWorker pid=1) INFO 10-03 08:53:54 [stats.py:837] | e2e_total_ms | 39,977.431 |

These are the stats that vllm-omni you get for each run:

[Overall Summary]
+-----------------------------+------------+
| Field                       |      Value |
+-----------------------------+------------+
| e2e_requests                |          1 |
| e2e_wall_time_ms            | 39,977.941 |
| e2e_avg_time_per_request_ms | 39,977.941 |
| e2e_stage_0_wall_time_ms    | 39,977.431 |
+-----------------------------+------------+

[RequestE2EStats [request_id=img_gen-ad003c4cf4678ae4-91216df1]]
+--------------+------------+
| Field        |      Value |
+--------------+------------+
| e2e_total_ms | 39,977.431 |
+--------------+------------+
[OmniTiming] req=img_gen-ad003c4cf4678ae4-91216df1 total=39.98s engine=39.98s stages=[0:39.98s]

[StageRequestStats [request_id=img_gen-ad003c4cf4678ae4-91216df1]]
+---------------------------------+------------+
| Field                           |          0 |
+---------------------------------+------------+
| batch_id                        |          6 |
| batch_size                      |          1 |
| denoise_step_latency_ms         |    999.402 |
| diffusion_engine_exec_time_s    |     39.953 |
| height                          |  1,024.000 |
| image_num                       |      1.000 |
| image_pixels                    |  1,048,576 |
| image_time_to_first_output_ms   | 39,976.059 |
| inter_output_latencies_ms       |            |
| kv_recv_time_s                  |      0.000 |
| num_inference_steps             |     40.000 |
| output_unit_count               |          1 |
| output_unit_type                |      image |
| postprocess_time_s              |      0.023 |
| preprocess_time_s               |      0.000 |
| resolution                      |    640.000 |
| scheduler_queue_wait_s          |      0.000 |
| serving_time_to_first_output_ms | 39,977.503 |
| stage_gen_time_ms               | 39,976.084 |
| vllm_itls_ms                    |            |
| width                           |  1,024.000 |
+---------------------------------+------------+
4

Thanks for running it! ~40 s vs our 54 s with plain diffusers at the same 1024² / 40 steps, with similar memory (~30 GB vs 31.6 GB), so vLLM-OMNI looks like the faster way to serve it on the Spark. I’ll add a pointer to your numbers in my post.

5

27s with TP2 on 0.30.1