Today as usually I’ve launched my DGX FE and using vllm v0.25.0 QWen3.6-35b-a3b.
And while working I’ve noticed that it’s extremely slowly on decoding. It’s not overheating at all, it’s slightly warm while working, like it was downgraded by throughput in 3-4 times.
Can someone tell me what was changed? How is it possible?
Benchmarks before update:
============ Serving Benchmark Result ============
Successful requests: 5
Failed requests: 0
Maximum request concurrency: 1
Benchmark duration (s): 93.43
Total input tokens: 50000
Total generated tokens: 10000
Request throughput (req/s): 0.05
Output token throughput (tok/s): 107.03
Peak output token throughput (tok/s): 36.00
Peak concurrent requests: 2.00
Total token throughput (tok/s): 642.20
---------------Time to First Token----------------
Mean TTFT (ms): 1599.46
Median TTFT (ms): 1592.62
P99 TTFT (ms): 1613.86
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 8.55
Median TPOT (ms): 8.80
P99 TPOT (ms): 9.80
---------------Inter-token Latency----------------
Mean ITL (ms): 28.98
Median ITL (ms): 28.93
P99 ITL (ms): 32.18
---------------Speculative Decoding---------------
Acceptance rate (%): 79.81
Acceptance length: 3.39
Drafts: 2948
Draft tokens: 8844
Accepted tokens: 7058
Per-position acceptance (%):
Position 0: 88.57
Position 1: 80.56
Position 2: 70.28
==================================================
Benchmarks after update:
============ Serving Benchmark Result ============
Successful requests: 3
Failed requests: 0
Maximum request concurrency: 1
Benchmark duration (s): 131.56
Total input tokens: 30000
Total generated tokens: 6000
Request throughput (req/s): 0.02
Output token throughput (tok/s): 45.61
Peak output token throughput (tok/s): 22.00
Peak concurrent requests: 2.00
Total token throughput (tok/s): 273.64
---------------Time to First Token----------------
Mean TTFT (ms): 1558.85
Median TTFT (ms): 1564.11
P99 TTFT (ms): 1566.39
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 21.16
Median TPOT (ms): 14.93
P99 TPOT (ms): 33.76
---------------Inter-token Latency----------------
Mean ITL (ms): 79.15
Median ITL (ms): 50.59
P99 ITL (ms): 238.87
---------------Speculative Decoding---------------
Acceptance rate (%): 91.40
Acceptance length: 3.74
Drafts: 1604
Draft tokens: 4812
Accepted tokens: 4398
Per-position acceptance (%):
Position 0: 94.51
Position 1: 92.39
Position 2: 87.28
==================================================
I’m using the same qwen3.6-35b-a3b more than a month and it’s always was fast. Today it’s not working as usually.
Here is my docker-compose:
qwen-moe:
image: vllm/vllm-openai:v0.25.0
container_name: qwen36-moe
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]
environment:
- HF_TOKEN=<token>
- CUTE_DSL_ARCH=sm_121a
- FLASHINFER_DISABLE_VERSION_CHECK=1
- VLLM_USE_FLASHINFER_MOE_FP4=0
- VLLM_FP8_MOE_BACKEND=flashinfer_cutlass
- CUDA_VISIBLE_DEVICES=0
volumes:
- ~/.cache/huggingface:/root/.cache/huggingface
- ./qwen-chat-tmp.jinja:/root/qwen-chat-tmp.jinja
ipc: host
ports:
- "8000:8000"
command: >
--model nvidia/Qwen3.6-35B-A3B-NVFP4
--port 8000
--reasoning-parser qwen3
--dtype auto
--kv-cache-dtype fp8
--tensor-parallel-size 1
--max-num-seqs 16
--max-model-len 262000
--gpu-memory-utilization 0.7
--trust-remote-code
--enable-prefix-caching
--enable-prompt-tokens-details
--max-num-batched-tokens 65536
--enable-auto-tool-choice
--tool-call-parser qwen3_coder
--quantization modelopt
--attention-backend flashinfer
--enable-chunked-prefill
--async-scheduling
--speculative-config '{"method":"mtp","num_speculative_tokens":3,"moe_backend":"triton"}'
--default-chat-template-kwargs '{"preserve_thinking":false}'
--chat-template /root/qwen-chat-tmp.jinja
Try to shut it down, remove the power from the wall socket (just removing the USB C cable will not work). Wait for a few minutes, then plug in and turn it on and try again.
After you unplug from the wall socket, some user reports that you can press the power button for 10 seconds and it will complete the process without waiting for few minutes.
Yep, thank you a lot, it helped.
============ Serving Benchmark Result ============
Successful requests: 3
Failed requests: 0
Maximum request concurrency: 1
Benchmark duration (s): 71.32
Total input tokens: 30000
Total generated tokens: 6000
Request throughput (req/s): 0.04
Output token throughput (tok/s): 84.13
Peak output token throughput (tok/s): 38.00
Peak concurrent requests: 2.00
Total token throughput (tok/s): 504.80
---------------Time to First Token----------------
Mean TTFT (ms): 1648.05
Median TTFT (ms): 1528.52
P99 TTFT (ms): 1881.15
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 11.07
Median TPOT (ms): 10.85
P99 TPOT (ms): 12.60
---------------Inter-token Latency----------------
Mean ITL (ms): 27.64
Median ITL (ms): 27.73
P99 ITL (ms): 29.91
---------------Speculative Decoding---------------
Acceptance rate (%): 50.02
Acceptance length: 2.50
Drafts: 2401
Draft tokens: 7203
Accepted tokens: 3603
Per-position acceptance (%):
Position 0: 67.85
Position 1: 49.90
Position 2: 32.32
==================================================
I’ve had the problem with power in the previous update when it was roasting like a hell in stale mode even nothing was happened. After the latest update of DGX SPark it's roasting like a hell in stale
it helped but a bit slower as usually, but it’s ok. Seems like NVIDIA something is updating and looks like working with power consumption.
Neill
August 3, 2026, 5:26pm
5
Hello, thank you for this feedback. I see that the topic has been marked closed by the OP. I encourange anyone who has outstanding concerns with recent updates to please open a new topic. Be sure to include as much as possible regarding your system, former version, updated versions, and system journal logs, as this will help us address concerns in an expedited manner.
Thanks!
As I mentioned before, it can be caused by update in this thread After the latest update of DGX SPark it's roasting like a hell in stale . It was 23 of jully update.
2026-07-23 07:14:22 upgrade tar:arm64 1.35+dfsg-3ubuntu0.3 1.35+dfsg-3ubuntu0.4
2026-07-23 07:14:22 upgrade gawk:arm64 1:5.2.1-2build3 1:5.2.1-2ubuntu0.1
2026-07-23 07:14:22 upgrade krb5-locales:all 1.20.1-6ubuntu2.6 1.20.1-6ubuntu2.7
2026-07-23 07:14:22 upgrade libgssapi-krb5-2:arm64 1.20.1-6ubuntu2.6 1.20.1-6ubuntu2.7
2026-07-23 07:14:22 upgrade libkrb5-3:arm64 1.20.1-6ubuntu2.6 1.20.1-6ubuntu2.7
2026-07-23 07:14:23 upgrade libkrb5support0:arm64 1.20.1-6ubuntu2.6 1.20.1-6ubuntu2.7
2026-07-23 07:14:23 upgrade libk5crypto3:arm64 1.20.1-6ubuntu2.6 1.20.1-6ubuntu2.7
2026-07-23 07:14:23 upgrade accountsservice:arm64 23.13.9-2ubuntu6 23.13.9-2ubuntu6.1
2026-07-23 07:14:23 upgrade libaccountsservice0:arm64 23.13.9-2ubuntu6 23.13.9-2ubuntu6.1
2026-07-23 07:14:23 upgrade code:arm64 1.129.1-1784303630 1.130.0-1784734546
2026-07-23 07:14:37 upgrade gir1.2-accountsservice-1.0:arm64 23.13.9-2ubuntu6 23.13.9-2ubuntu6.1
2026-07-23 07:14:37 upgrade gstreamer1.0-gtk3:arm64 1.24.2-1ubuntu1.4 1.24.2-1ubuntu1.5
2026-07-23 07:14:37 upgrade gstreamer1.0-plugins-good:arm64 1.24.2-1ubuntu1.4 1.24.2-1ubuntu1.5
2026-07-23 07:14:37 upgrade libgstreamer-plugins-good1.0-0:arm64 1.24.2-1ubuntu1.4 1.24.2-1ubuntu1.5
2026-07-23 07:14:37 upgrade libarchive13t64:arm64 3.7.2-2ubuntu0.7 3.7.2-2ubuntu0.8
2026-07-23 07:14:37 upgrade libgif7:arm64 5.2.2-1ubuntu1 5.2.2-1ubuntu1.2
2026-07-23 07:14:38 upgrade libgphoto2-port12t64:arm64 2.5.31-2.1ubuntu1 2.5.31-2.1ubuntu1.1
2026-07-23 07:14:38 upgrade libgphoto2-6t64:arm64 2.5.31-2.1ubuntu1 2.5.31-2.1ubuntu1.1
2026-07-23 07:14:38 upgrade libgphoto2-l10n:all 2.5.31-2.1ubuntu1 2.5.31-2.1ubuntu1.1
2026-07-23 07:14:38 upgrade libhtml-parser-perl:arm64 3.81-1build3 3.81-1ubuntu0.1
2026-07-23 07:14:38 upgrade libjbig2dec0:arm64 0.20-1build3 0.20-1ubuntu0.24.04.1
2026-07-23 07:14:38 upgrade remmina-plugin-rdp:arm64 1.4.43+dfsg-0ubuntu0.24.04.1 1.4.43+dfsg-0ubuntu0.24.04.2
2026-07-23 07:14:38 upgrade remmina-plugin-vnc:arm64 1.4.43+dfsg-0ubuntu0.24.04.1 1.4.43+dfsg-0ubuntu0.24.04.2
2026-07-23 07:14:39 upgrade remmina-plugin-secret:arm64 1.4.43+dfsg-0ubuntu0.24.04.1 1.4.43+dfsg-0ubuntu0.24.04.2
2026-07-23 07:14:39 upgrade remmina:arm64 1.4.43+dfsg-0ubuntu0.24.04.1 1.4.43+dfsg-0ubuntu0.24.04.2
2026-07-23 07:14:39 upgrade remmina-common:all 1.4.43+dfsg-0ubuntu0.24.04.1 1.4.43+dfsg-0ubuntu0.24.04.2
2026-07-23 07:14:39 upgrade snapd:arm64 2.75.2+ubuntu24.04 2.76+ubuntu24.04.1
2026-07-23 18:14:47 upgrade libfprint-2-tod1:arm64 1:1.94.7+tod1-0ubuntu5~24.04.7 1:1.94.7+tod1-0ubuntu5~24.04.8
2026-07-23 18:14:47 upgrade libfprint-2-2:arm64 1:1.94.7+tod1-0ubuntu5~24.04.7 1:1.94.7+tod1-0ubuntu5~24.04.8
2026-07-23 18:14:47 upgrade nvidia-driver-580-open:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:47 upgrade linux-modules-nvidia-580-open-nvidia-hwe-24.04:arm64 6.17.0-1026.26 6.17.0-1026.26+3
2026-07-23 18:14:47 upgrade linux-modules-nvidia-580-open-6.17.0-1026-nvidia:arm64 6.17.0-1026.26 6.17.0-1026.26+3
2026-07-23 18:14:47 upgrade libnvidia-extra-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:47 upgrade libnvidia-common-580:all 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:48 upgrade libnvidia-decode-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:48 upgrade libnvidia-compute-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:49 upgrade nvidia-utils-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:49 upgrade libnvidia-fbc1-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:49 upgrade libnvidia-encode-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:50 upgrade libnvidia-gl-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:52 upgrade xserver-xorg-video-nvidia-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:52 upgrade libnvidia-cfg1-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:52 upgrade nvidia-kernel-common-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:52 upgrade nvidia-kernel-source-580-open:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
2026-07-23 18:14:53 upgrade nvidia-compute-utils-580:arm64 580.159.03-0ubuntu0.24.04.1 580.173.02-0ubuntu0.24.04.1
Some fresh logs of updates:
/var/log/dpkg.log.1:2026-07-27 17:35:11 upgrade libc6-dbg:arm64 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:11 upgrade libc-devtools:arm64 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:12 upgrade libc6-dev:arm64 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:12 upgrade libc-dev-bin:arm64 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:12 upgrade libc6:arm64 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:13 upgrade libc-bin:arm64 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:13 upgrade locales:all 2.39-0ubuntu8.7 2.39-0ubuntu8.8
/var/log/dpkg.log.1:2026-07-27 17:35:13 install linux-nvidia-6.17-headers-6.17.0-1029:all <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:19 install linux-headers-6.17.0-1029-nvidia:arm64 <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:22 install linux-modules-6.17.0-1029-nvidia:arm64 <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:25 install linux-image-6.17.0-1029-nvidia:arm64 <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:25 install linux-modules-nvidia-fs-6.17.0-1029-nvidia:arm64 <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:25 upgrade linux-nvidia-hwe-24.04:arm64 6.17.0-1026.26 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:25 upgrade linux-image-nvidia-hwe-24.04:arm64 6.17.0-1026.26 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:25 upgrade linux-headers-nvidia-hwe-24.04:arm64 6.17.0-1026.26 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:25 install linux-modules-nvidia-580-open-6.17.0-1029-nvidia:arm64 <none> 6.17.0-1029.29+1
/var/log/dpkg.log.1:2026-07-27 17:35:26 upgrade linux-modules-nvidia-580-open-nvidia-hwe-24.04:arm64 6.17.0-1026.26+3 6.17.0-1029.29+1
/var/log/dpkg.log.1:2026-07-27 17:35:26 upgrade linux-modules-nvidia-fs-nvidia-hwe-24.04:arm64 6.17.0-1026.26 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:26 install linux-nvidia-6.17-tools-6.17.0-1029:arm64 <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:26 install linux-tools-6.17.0-1029-nvidia:arm64 <none> 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-27 17:35:26 upgrade linux-tools-nvidia-hwe-24.04:arm64 6.17.0-1026.26 6.17.0-1029.29
/var/log/dpkg.log.1:2026-07-28 17:21:58 upgrade libsmbclient0:arm64 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:21:59 upgrade samba:arm64 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:21:59 upgrade samba-common-bin:arm64 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:21:59 upgrade python3-samba:arm64 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:21:59 upgrade libwbclient0:arm64 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:22:00 upgrade samba-libs:arm64 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:22:00 upgrade python3-ldb:arm64 2:2.8.0+samba4.19.5+dfsg-4ubuntu9.6 2:2.8.0+samba4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:22:00 upgrade libldb2:arm64 2:2.8.0+samba4.19.5+dfsg-4ubuntu9.6 2:2.8.0+samba4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:22:00 upgrade samba-common:all 2:4.19.5+dfsg-4ubuntu9.6 2:4.19.5+dfsg-4ubuntu9.7
/var/log/dpkg.log.1:2026-07-28 17:22:00 upgrade freeipmi-common:all 1.6.13-3 1.6.13-3ubuntu0.1
/var/log/dpkg.log.1:2026-07-28 17:22:00 upgrade libfreeipmi17:arm64 1.6.13-3 1.6.13-3ubuntu0.1
/var/log/dpkg.log.1:2026-07-29 17:09:47 upgrade libfreerdp-client3-3:arm64 3.30.0+dfsg-0ubuntu0.24.04.1 3.30.0+dfsg-0ubuntu0.24.04.2
/var/log/dpkg.log.1:2026-07-29 17:09:47 upgrade libfreerdp-server3-3:arm64 3.30.0+dfsg-0ubuntu0.24.04.1 3.30.0+dfsg-0ubuntu0.24.04.2
/var/log/dpkg.log.1:2026-07-29 17:09:47 upgrade libfreerdp3-3:arm64 3.30.0+dfsg-0ubuntu0.24.04.1 3.30.0+dfsg-0ubuntu0.24.04.2
/var/log/dpkg.log.1:2026-07-29 17:09:47 upgrade libwinpr3-3:arm64 3.30.0+dfsg-0ubuntu0.24.04.1 3.30.0+dfsg-0ubuntu0.24.04.2
/var/log/dpkg.log.1:2026-07-30 17:25:34 upgrade distro-info-data:all 0.60ubuntu0.6 0.72-0ubuntu0.24.04.1
/var/log/dpkg.log.1:2026-07-30 17:25:34 upgrade tzdata:all 2026b-0ubuntu0.24.04.1 2026c-0ubuntu0.24.04.1
/var/log/dpkg.log.1:2026-07-30 17:25:34 upgrade code:arm64 1.130.0-1784734546 1.131.0-1785237876