[THOR] Cannot run LLM - System Reboots

[eeeeeeeeeek]

Good day all,

I have an Nvidia Thor that has previously worked with LLMs such as: Qwen3.6-27B-FP8 on container Package vllm · GitHub (35.6 GB) from multiple months ago (using Tegra 38 revision 4).

As of upgrading to Tegra 39 revision 2.1, updating the container ( Package vllm · GitHub ) or using an alternative nvidia container (nvcr.io/nvidia/vllm:26.06-py3). I have also tried alternative LLMs (e.g., Qwen3.6-27B-FP8, Qwen3.6-27B-NVFP4, Qwen3.6-35B-A3B-FP8, Qwen3.8-27B-FP8 and Qwen3.8-27B-NVFP4.) I continue to receive the same issue / outcome, noted below.

• ISSUE: Nvidia Thor reboots either shortly after starting LLM (please see example recipes below), or the LLM does not start.

NOTE: It sounds like the fan restarts just after reboot, and that is my initial indication that it has rebooted. This appears to happen when CPU is the highest (or over .

Please see the following attachments:

• System specifications: Screenshot 2026-08-21 135024.png

  

• Example 1 - CPU Usage just before crash: Screenshot 2026-08-21 140350.png

  

• Example 1 - CPU Usage just before crash: Screenshot 2026-08-21 141658.png

  

Example recipes:

• With previous tegra version:
  • HF_HUB_OFFLINE=0 sudo docker run --restart=unless-stopped -d -e HF_HUB_OFFLINE=“0” --runtime=nvidia --gpus all --ipc=host -v //Qwen3.6-27B-FP8/:/Qwen3.6-27B-FP8 --name qwen --network host <Package vllm · GitHub> vllm serve Qwen3.6-27B-FP8 --gpu-memory-utilization 0.80 --enable-prefix-caching --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
• Attempts with new tegra version include examples like:
  • HF_HUB_OFFLINE=0 sudo docker run --restart=unless-stopped -d -e HF_HUB_OFFLINE=“0” --runtime=nvidia --gpus all --ipc=host -v //Qwen3.6-27B-FP8/:/Qwen3.6-27B-FP8 --name qwen --network host <Package vllm · GitHub> vllm serve Qwen3.6-27B-FP8 --gpu-memory-utilization 0.80 --enable-prefix-caching --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
  • HF_HUB_OFFLINE=0 sudo docker run --restart=unless-stopped -d -e HF_HUB_OFFLINE=“0” --runtime=nvidia --gpus all --ipc=host -v //Qwen3.8-27B-FP8/:/Qwen3.8-27B-FP8 --name qwen --network host <Package vllm · GitHub> vllm serve Qwen3.8-27B-FP8 --gpu-memory-utilization 0.80 --enable-prefix-caching --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml

Please note that in the recipes I have tried with various options / arguments, including (but not limited to):

• docker containers attempted to use:
  • nvcr.io/nvidia/vllm:26.06-py3
  • <Package vllm · GitHub>
    • NOTE: I have also attempted with an older version, which was a different version, however - I can no longer find that version.
  • vllm/vllm-openai:latest
    • NOTE: No ‘vllm serve’ included, since the container already assumes it.
• docker options / arguments:
  • -e VLLM_LOGGING_LEVEL=“DEBUG”
  • -e TORCH_CUDA_ARCH_LIST=“11.0a”
  • -e GPU_MAX_HW_QUEUES=1
• vllm options / arguments:
  • --tool-call-parser qwen3_xml
  • --tool-call-parser qwen3_coder
  • --tensor-parallel-size 1
  • --max-model-len 262144
  • --max-model-len 102400
  • --enforce-eager
  • --disable-custom-all-reduce
  • --max-num-seqs 1
  • --max-num-batched-tokens 32768
  • --max-num-batched-tokens 8192
  • --served-model-name “”
  • --gpu-memory-utilization 0.45 to 0.8
    • Attempted in increments of 0.5

NOTE: I am currently using nvcr.io/nvidia/vllm:26.06-py3 instead of the latest (nvcr.io/nvidia/vllm:26.07), due to a missing python xgrammar library.

NOTE: I have already attempted to run both (Qwen3.6 27B | Jetson AI Lab) and (Qwen3.8 27B | Jetson AI Lab) as is, and they result in the same issue noted above.

Thanks,
E

[whitesscott]

https://docs.vllm.ai/en/latest/configuration/conserving_memory/

These are things I’ve done to avoid oom.

Try a lower percentage than -gpu-memory-utilization 0.80

Or use this in place of -gpu-memory-utilization. The docs above describe the variable.
--kv-cache-memory 4G \

[eeeeeeeeeek]

Thank you for the reply @whitesscott !

I appreciate the support on this.

I have attempted the same memory management techniques with --gpu-memory-utilization, and I have just spent over an hour attempting to use this with a couple variants including a couple variants of --kv-cache-memory options. I am not seeing a change.

Interestingly enough, I used a command you have mentioned on previously mentioned in another thread: echo on | sudo tee /sys/bus/pci/devices/0000:01:00.0/power/control and this still does not fix the issue.

Even just pulling a docker image (vllm/vllm-openai:latest), crashes with the details in the below image:

top

watch free -h

watch nvidia-smi

sudo docker pull vllm/vllm-openai:latest

Thanks,
E

[whitesscott]

What is your sudo nvpmodel -q
If it is ID=0 NAME=MAXN

You could see if changing to
sudo nvpmodel -m 1
might help.

[eeeeeeeeeek]

Good question.

It appears I have been running off of 120W (mode 1) the entire time.

Thanks,
E

[eeeeeeeeeek]

Good day all,

I am suspecting that this is simply a hardware challenge. I am not sure what to make of it. Even when only connecting:

1. Nvidia shipped Power Adapter (Type-C) plugged into one of the vertical Type-C ports (I have never used anything else)
2. USB Type-C Cable connected to USB on another system for running commands via horizontal debug Type-C UART port

NOTE: No Ethernet, USB, HDMI, or other interfaces / devices connected.

I receive the following after attempting to run the following:

HF_HUB_OFFLINE=0 sudo docker run -d -e HF_HUB_OFFLINE=“0” -e GPU_MAX_HW_QUEUES=“1” --runtime=nvidia --network host -v //Qwen3.6-27B-FP8/:/Qwen3.6-27B-FP8 --name qwen --gpus all --ipc=host nvcr.io/nvidia/vllm:26.06-py3 vllm serve /Qwen3.6-27B-FP8 --gpu-memory-utilization 0.7 --served-model-name “Qwen3.6-27B-FP8” --max-model-len 102400 --enable-prefix-caching --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml && sudo docker logs qwen -f

I get simply a restart:

FYI - This is on an Nvidia AGX Thor Developer Kit.

Thanks,
E

[whitesscott]

vllm is exceptionally version versus vllm serve command line sensitive. I tried to use other models on my Thor but have had no luck. So here’s what works with compiled on Thor VLLM-version 0.27.2rc1.dev5. I post it only so that you might see some of the variables that can be used to lower memory use.

Prior to running vllm you may want to run:
sudo sync && echo 3 |sudo tee /proc/sys/vm/drop_caches

VLLM_PLUGINS="" \
VLLM_FLOAT32_MATMUL_PRECISION="high" \
vllm serve Inferact/Qwen3.8-27B-NVFP4 \
    --host 127.0.0.1 \
    --port 8000 \
    --served-model-name qwen3.8-27b \
    --language-model-only \
    --async-scheduling \
    --max-model-len 32768 \
    --max-num-seqs 4 \
    --max-num-batched-tokens 4096 \
    --kv-cache-memory 3G \
    --kv-cache-dtype auto \
    --gpu-memory-utilization 0.85 \
    --reasoning-parser qwen3

With the above running.

 free -h
               total        used        free      shared  buff/cache   available
Mem:           122Gi        40Gi        55Gi       136Mi        26Gi        82Gi

[eeeeeeeeeek]

Thank you again for the update @whitesscott ! So happy you are in the forum with us!

Great recipe. Unfortunately, I cannot run an LLM due to the Thor self restarting. I hear the fans running, and this is my que to know that the server rebooted. I am simply pinging the Thor on my network, and I have evidence that it continues to go offline (at least from the network), and it continues to restart, independently, all by itself. I am presently lost on were to continue diagnosing the issue. UART logs (only using minicom) are not rendering anything I see of note based on some research. I am attaching 1) the ping log (from today, just after you sent the message), 2) an example (from earlier today) of a simple startup of the Thor, and 3) an image of when using the UART debug port, and attempting to run an LLM the server rebooting during the UART session.

1. nvidiaport22.log (85.7 KB)
2. thor-boot.log (130.2 KB)
3. 

@whitesscott - I do, sincerely appreciate the support. One minor update is that it appears the Thor is experiencing automatic reboots. Any chance you (or someone you know) have experience with troubleshooting a automatic reboots / self-rebooting of Nvidia AGX Thor Developer Kit (T5000) on Tegra 39 R2 (Jetpack V7.2.1)?

Nvidia Support - any ideas?

Thanks,
E

[whitesscott]

On Thor is there anything here?

sudo ls -lah /sys/fs/pstore
sudo ls -al /var/crash/

---

On your host pc can you

Run:
ls /dev/ttyACM0 
# if it doesn't exist then
sudo apt install linux-modules-extra-$(uname -r)

Then 
lsmod | grep cdc_acm.
# If that exists you are good to proceed to next codeblock.

# If it doesn't exist then:
sudo modprobe cdc_acm

# And to make it load at boot run:
sudo tee /etc/modules-load.d/cdc_acm.conf >/dev/null <<'EOF'
cdc_acm
EOF

Then still on host:

cd path-to/Linux_for_Tegra/tools/demuxer

sudo rm -rf /tmp/thor-reboot
sudo mkdir -p /tmp/thor-reboot

sudo ./nv_tcu_demuxer \
    -m T264 \
    -d /dev/ttyACM0 \
    -s /tmp/thor-reboot \
    -t

Then run vllm on Thor.

Once Thor reboots you could stop demuxer and post the /tmp/thor-reboot/*.txt files here.

[eeeeeeeeeek]

Good day all,

Thank you @whitesscott for the directions on this. I’ve done the following, and it appears to still be rebooting.

Attached are the logs from two sessions.

Please note, I am still attempting to run any kind of LLM (vllm raw, vllm via docker, or any alternative). I am presently sticking to what has worked before:

I am also sharing screenshots from the interaction:

The host terminal running nv_tcu_demuxer:

The host terminal (when I ran the command):

Enclosed are the files:

• thor-reboot-secondtry-after-a-few-minutes.zip (4.8 KB)
• thor-reboot.zip (3.8 KB)

I hope this helps. If there is anything else for me to try, I am more than happy to.

Thanks,
E

[eeeeeeeeeek]

@whitesscott

sudo ls -lah /sys/fs/pstore
sudo ls -al /var/crash/

From these commands, I receive this output:

[whitesscott]

Try this by itself and after it’s set try vllm.
sudo nvpmodel -m 2

Please copy to text file results of:

cat /etc/docker/daemon.json 
ls /etc/nvidia-container-runtime/host-files-for-container.d/

---

Copy and run this entire code block as one command into a second terminal instance, immediately before starting vllm

sudo stdbuf -oL dmesg -Hw --nopager | tee dmesg.txt &
sudo stdbuf -oL journalctl --follow --no-pager | tee journal.txt &

(
    echo "=== THOR THERMAL MONITOR START ==="
    date --iso-8601=ns
    echo

    echo "=== THERMAL TRIP POINTS ==="
    for file in /sys/class/thermal/thermal_zone*/trip_point_*_temp; do
        [[ -r "$file" ]] || continue
        temp=$(<"$file")
        type=$(<"${file%/*}/type")
        printf "%-24s %-32s %6.1f C\n" \
            "$type" "${file##*/}" "$(awk "BEGIN {print $temp/1000}")"
    done

    echo
    echo "=== LIVE TEMPERATURES ==="

    while :; do
        ts=$(date '+%Y-%m-%d %H:%M:%S.%3N')

        for zone in /sys/class/thermal/thermal_zone*; do
            [[ -r "$zone/type" && -r "$zone/temp" ]] || continue

            type=$(<"$zone/type")
            temp=$(<"$zone/temp")

            printf "%s  %-24s %6.1f C\n" \
                "$ts" "$type" "$(awk "BEGIN {print $temp/1000}")"
        done

        sleep 0.1
    done
) | stdbuf -oL tee thermal.txt &

---

After reboot run:

cat /sys/devices/platform/bus@0/c800000.pmc/reset_reason
cat /sys/devices/platform/bus@0/c800000.pmc/reset_level

[eeeeeeeeeek]

Thank you for the response!

@whitesscott

cat /etc/docker/daemon.json

This is the content of ‘cat /etc/docker/daemon.json’:

{
"runtimes": {
"nvidia": {
"args": [ ] ,
"path": "nvidia-container-runtime"
}
}
}

@whitesscott

ls /etc/nvidia-container-runtime/host-files-for-container.d/

This is the content of ls /etc/nvidia-container-runtime/host-files-for-container.d/:

devices.csv  drivers.csv  l4t.csv

Also seen as (ls -alh /etc/nvidia-container-runtime/host-files-for-container.d/):

total 36K
drwxr-xr-x 2 root root 4.0K Dec 31  1969 .
drwxr-xr-x 3 root root 4.0K Jul  2  2025 ..
-rw-r--r-- 1 root root 1.1K Dec 30  2025 devices.csv
-rw-r--r-- 1 root root  19K Aug  7 01:41 drivers.csv
-rw-r--r-- 1 root root  167 Aug 20 18:50 l4t.csv

@whitesscott

Copy and run this entire code block as one command into a second terminal instance, immediately before starting vllm

I’ve created a script to make it easier:

NOTE: The Thor has rebooted at least 2 times since I have started troubleshooting and typing this so far.

Running command to start vllm:

HF_HUB_OFFLINE=0 sudo docker run --restart=unless-stopped -d -e HF_HUB_OFFLINE="0" -e VLLM_LOGGING_LEVEL="DEBUG" --runtime=nvidia --network host -v /home/t/Qwen3.6-27B-FP8/:/Qwen3.6-27B-FP8 --name qwen --gpus all --ipc=host nvcr.io/nvidia/vllm:26.06-py3 vllm serve qwen/Qwen3.6-27B-FP8 --gpu-memory-utilization 0.6 --served-model-name "Qwen3.6-27B-FP8" --enable-prefix-caching --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml && sudo docker logs qwen -f

The run script (commands you provided me, thank you!) output ended like so:

The last output from attempting to run an LLM:

Additionally, here is the output from top just before restart:

@whitesscott

cat /sys/devices/platform/bus@0/c800000.pmc/reset_reason

Here is my output:

cat: /sys/devices/platform/bus@0/c800000.pmc/reset_reason: No such file or directory

@whitesscott

cat /sys/devices/platform/bus@0/c800000.pmc/reset_level

Here is my output:

cat: /sys/devices/platform/bus@0/c800000.pmc/reset_level: No such file or directory

Screenshot:

Thank you immensely for the support @whitesscott!

NOTE: At time time of writing this, the Thor has restarted at least 5 times.

Thanks,
E

[whitesscott]

Can you post the, I think it is 3 *.txt files that would be in the directory where you ran run.sh?

[eeeeeeeeeek]

I apologize, here is a ZIP of the three files (dmesg.txt, journal.txt, thermal.txt)

08232026thortroubleshooting.zip (102.3 KB)

Thanks,
E

[whitesscott]

Some of the tests did not collect any, or much, data. I think it would be good to see this info:

On host pc.

cd Linux_for_Tegra/tools/demuxer
mkdir -p logs

sudo ./nv_tcu_demuxer \
    -m T264 \
    -d /dev/ttyACM0 \
    -s ./logs \
    -t

Once that is running power on Thor.

Once you have logged on to Thor desktop run
sudo dmesg -H --nopager | tee dmesg.txt

Then start vllm. When you hear Thor fan start; stop the nv_tcu_demuxer and post the *.txt files.

[AastaLLL]

Hi,

Just want to double-confirm your issue:

1. You can run vLLM with r38.4 normally with the Thor devkit + official power.
2. After upgrading to r39.2.1, Thor automatically reboots when running the vLLM.

Could you share how you upgrade the device?
Do you apply the apt upgrade or run the SDK Manager on the device directly?

How long does it take to reproduce the issue?
Could you share a command that can trigger the reboot with us?

We would like to reproduce this issue locally to gather more information.

Thanks.

[eeeeeeeeeek]

Good day @AastaLLL,

Thank you so much for jumping in!

Yes, to 1., and with 2. it appears that automatic rebooting is happening (in general), unrelated to any runtime (e.g., vllm, docker, et cetera.) I am not sure about re flashing to a previous version (I have not flashed the Thor). I did upgrade from 38.4.0 to 39.2.0 (Jetpack 7.2.1) directly.

My update process was connecting an HDMI / and two USB (1 x mouse, 1 x keyboard) to the thor directly (Ethernet was already connected), and preforming the update as documented here (Install Jetson Software with SDK Manager on Jetson Device — SDK Manager). I am using the same USB Type-C power cable that came with the device (and have never used anything else).

I order to reproduce the issue: I simply plug in the power cable to the Nvidia Thor (no other cables / interfaces connected), and the system will reboot all by itself.

Here is a 15 minute example video of this happening (reboot occurs after minute 12.30): https://photos.app.goo.gl/PyszcqPKEZ29bn8s6

Thanks,
E

[AastaLLL]

Hi,

Thanks for the confirmation and the video.

We are going to reproduce this issue and will provide more information to you.
In the meantime, are you able to connect the device with UART and share the logs before/after the reboot?

Thanks

[eeeeeeeeeek]

Good day again @AastaLLL!

I appreciate the support on this.

In the 11th post (quote below), I attempted to connect UART and run the commands (with demuxer), and received the below results:

@eeeeeeeeeek

Enclosed are the files:

• thor-reboot-secondtry-after-a-few-minutes.zip (4.8 KB)
• thor-reboot.zip (3.8 KB)

Are there new things that we are thinking we should run, that we haven’t on this thread?

Thanks,
E