# Setting Up the NVIDIA Jetson Orin Nano Super Dev Kit on JetPack 7.2 — A Practical Guide (June 2026)
A working setup guide based on actually walking the bleeding edge. Covers everything that's not in the official docs: which pieces of the AI ecosystem work, which don't (yet), and what to build from source to fill the gaps.
## Why this guide exists
JetPack 7.2 (L4T 39.2, Ubuntu 24.04, CUDA 13.2, cuDNN 9.20, TensorRT 10.16) is fresh enough that most of the third-party ecosystem hasn't caught up. dustynv's `jetson-containers` images, Ollama's prebuilt binary, and PyTorch's official Jetson wheels were all built against JetPack 6.x and CUDA 12.6. They either don't run, run on CPU, or produce silent NaN output on the new stack.
What does work: **anything you build from source against your local CUDA 13.2 toolchain**. The goal of this guide is to give you a known-good native-build setup for LLM inference and image generation, plus enough context to evaluate other ecosystem tools as they're updated.
Tested hardware: Jetson Orin Nano Super Developer Kit (P3767-0005 module + P3768-0000 carrier), 2 TB NVMe, included DC supply, June 2026.
---
## Part 1: Flashing JetPack 7.2 to NVMe
### What you need
- Ubuntu 24.04 x86_64 host machine (bare metal preferred; VM USB passthrough is finicky)
- USB-C cable, data-capable (not charge-only — many random cables fail here)
- NVIDIA Developer account
- The Super dev kit, with NVMe pre-installed in the M.2 2280 slot before flashing
- An ethernet cable for first boot networking (or be prepared to configure WiFi over serial console)
### Get the dev kit into recovery mode
The reference carrier doesn't ship with populated tactile buttons. You enter recovery mode by jumping pins on the button header (J14, near the DC barrel jack):
1. Power off — unplug the DC barrel completely
2. Wait 10 seconds for caps to drain
3. Short pins 9 and 10 of the button header — jumper cap, tweezers, or bent paperclip all work
4. Connect USB-C to your host
5. Apply DC power while pins 9-10 are still shorted
6. Hold for 2-3 seconds after power applies, then remove the jumper
Verify on your host:
```bash
lsusb | grep -i nvidia
# You want: NVIDIA Corp. APX (idVendor 0955, idProduct 7523)
```
If you see `APX`, you're in recovery mode. If you see nothing, the jumper didn't make contact at the moment of power-on — try again.
### Flash with SDK Manager
Install NVIDIA SDK Manager on the Ubuntu host, log in with your Developer account, and let it auto-detect the device. Critical things to set:
**Target selection:** "Jetson Orin Nano [8GB developer kit version]" — module P3767-0005, carrier P3768-0000. The bare module variants will not flash correctly on the dev kit carrier.
**Pre-config (this is the one that bites people):** In the OS configuration step, fill in username, password, hostname, timezone, language, keyboard layout. **Do not skip this.** Without pre-config, the first boot runs an OEM wizard that needs serial or display console input, and you'll be unreachable over network until you complete it. With pre-config, the system boots straight to login with networking up.
**Storage device:** NVMe. Don't accept the default if it shows microSD or eMMC.
**Component selection:** I recommend unchecking "Jetson SDK Components" for the first run. Flash only the base OS, verify it boots and is reachable, then install components in a second pass. This isolates failures — if the network install of CUDA/cuDNN/TensorRT hits a snag, you don't lose the working OS flash.
Flash time: roughly 20-30 minutes for OS to NVMe over USB-C.
### After flash
1. Disconnect USB-C from Jetson
2. Connect ethernet (your home router is easiest — gets the Jetson on the LAN with internet)
3. Power-cycle the Jetson (unplug DC, wait 10 sec, replug — no jumper)
4. Wait ~60 seconds for first boot
Find the Jetson's IP from your router's DHCP client list, or:
```bash
# From your host:
ping jetson.local
# If hostname pre-config worked, this resolves
ssh username@jetson.local
```
Welcome to your Jetson.
---
## Part 2: System Configuration
### Verify and lock in Super performance mode
Default boot may be in 15W mode. Switch to MAXN_SUPER:
```bash
# Check current and available modes:
sudo nvpmodel -q
sudo nvpmodel -p --verbose
# IDs vary across JetPack versions. On JP 7.2 you'll see something like:
# ID=0 NAME=15W
# ID=1 NAME=7W
# ID=2 NAME=MAXN_SUPER
# Set MAXN_SUPER (use whatever ID maps to MAXN_SUPER on your system):
sudo nvpmodel -m 2
sudo nvpmodel -q # confirm "MAXN_SUPER"
```
This persists across reboots.
### Pin clocks across reboots
`jetson_clocks` pins frequencies at max; the setting is not persistent by default. Create a systemd unit:
```bash
sudo tee /etc/systemd/system/jetson_clocks.service > /dev/null <<EOF
[Unit]
Description=Pin Jetson clocks at boot
After=nvpmodel.service
[Service]
Type=oneshot
ExecStart=/usr/bin/jetson_clocks
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable jetson_clocks.service
sudo systemctl start jetson_clocks.service
```
### Install jtop
Essential dashboard for GPU/CPU/memory/power/thermal. Note: jtop in current releases doesn't recognize JetPack 7.2 yet and shows "Jetpack NOT DETECTED" — this is cosmetic only; all live metrics work correctly.
```bash
sudo apt update
sudo apt install python3-pip
sudo pip3 install -U jetson-stats --break-system-packages
sudo reboot
# Then:
jtop
```
### Replace the default 2 GB swap with 16 GB on NVMe
JetPack 7.2 ships with `/swapfile` already configured at 2 GB. Replace it for LLM and SDXL headroom:
```bash
sudo swapoff /swapfile
sudo rm /swapfile
sudo fallocate -l 16G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
# Ensure the existing fstab entry isn't duplicated:
grep swapfile /etc/fstab
# If not present:
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
# Lower swappiness for AI workloads:
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
free -h # confirm ~15-16 GB Swap
```
### Set the hostname so mDNS works
If the hostname is `localhost` (which can happen if pre-config didn't capture it):
```bash
sudo hostnamectl set-hostname jetson
sudo sed -i 's/127.0.1.1.*/127.0.1.1\tjetson/' /etc/hosts
sudo systemctl restart avahi-daemon
```
After this, `jetson.local` resolves from any device on the LAN that speaks mDNS. Bonjour on macOS, Avahi on Linux, both work. On Windows, install "Bonjour Print Services" or just use the IP.
A practical note: macOS aggressively caches mDNS, so after a Jetson reboot the Mac may take a minute to re-discover or hold a stale entry. Quick fix:
```bash
# On the Mac:
sudo killall -HUP mDNSResponder
```
For a permanent fix, add the Jetson to `/etc/hosts` on the Mac with a static IP (set a DHCP reservation in your router).
---
## Part 3: What Works and What Doesn't on JP 7.2
This is the critical context that's not in any official doc.
### Works out of the box (after JetPack install)
- CUDA 13.2 toolkit, nvcc, cuda-libraries, samples
- cuDNN 9.20
- TensorRT 10.16.2
- Nsight Systems and Nsight Compute
- GNOME desktop, Firefox, snap, apt
- Docker with NVIDIA Container Runtime
- Basic PyTorch operations (matmul, etc.) via the SBSA aarch64+cu126 wheel
- `nv-power-mode`, `nvfancontrol`, basic Jetson tooling
### Does NOT work (or works only on CPU)
| Tool | Status | Root cause |
|---|---|---|
| dustynv `jetson-containers` (Ollama, ComfyUI, etc.) | Either crash or silent CPU fallback | Container CUDA 12.6 runtime can't bridge to host CUDA 13.2 driver — Error 801 "operation not supported" |
| Ollama prebuilt binary install | CPU-only inference | Install script hardcodes JetPack version detection; sees "unsupported" and disables GPU code path entirely |
| ComfyUI / Diffusers diffusion on GPU | Silent NaN output (blank images) | PyTorch SBSA wheel was built `except {8.7}` — Orin's compute capability isn't included; some UNet kernels don't have PTX fallback that JIT-compiles correctly |
| jtop's "JetPack version" display | Shows "NOT DETECTED" | jetson-stats version table doesn't include JP 7.2 yet |
### Does work via build-from-source
| Tool | Build approach | Works because |
|---|---|---|
| **llama.cpp / llama-server** | `cmake -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87` | Compiles native sm_87 kernels against your CUDA 13 |
| **stable-diffusion.cpp / sd-cli** | Same pattern | Same reason |
| **PyTorch (limited)** | SBSA wheel install | Loads, runs basic ops, has unstable kernels for diffusion |
| **ComfyUI on CPU** | Native pip install in venv | CPU bypasses the sm_87 kernel issue entirely |
The lesson: **anything that compiles against your local CUDA 13.2 toolchain works; anything prebuilt against CUDA 12.6 has issues.** This will resolve as the ecosystem catches up over the coming weeks/months.
---
## Part 4: LLM Inference (llama.cpp Native Build)
### Build llama.cpp
```bash
sudo apt install -y build-essential cmake git libcurl4-openssl-dev
cd ~
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87 -DLLAMA_CURL=ON
cmake --build build --config Release -j$(nproc)
```
Build time: ~15 minutes. The CUDA compilation step is the long part — heavy template instantiation for tensor core kernels.
### Download a model
For 8 GB unified memory, Q4_K_M is the sweet spot. Two solid choices:
```bash
mkdir -p ~/models
cd ~/models
# Qwen 2.5 7B Instruct (proven, fast):
wget https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct-Q4_K_M.gguf
# Qwen 3.5 9B (newer, reasoning-capable — fits with 16k context):
# Check bartowski's repo for current URL.
```
### Benchmark
```bash
cd ~/llama.cpp
./build/bin/llama-bench -m ~/models/Qwen2.5-7B-Instruct-Q4_K_M.gguf -ngl 99
```
Expected on Orin Nano Super in MAXN_SUPER mode:
- Qwen 2.5 3B Q4_K_M: ~23 tok/s
- Qwen 2.5 7B Q4_K_M: ~12 tok/s
- Qwen 3.5 9B Q4_K_M: ~10 tok/s
### Run as a systemd service
```bash
sudo tee /etc/systemd/system/llama-server.service > /dev/null <<'EOF'
[Unit]
Description=llama.cpp server
After=network-online.target nvargus-daemon.service
Wants=network-online.target
[Service]
Type=simple
User=YOUR_USERNAME
Environment="LD_LIBRARY_PATH=/usr/local/cuda/lib64"
ExecStart=/home/YOUR_USERNAME/llama.cpp/build/bin/llama-server \
-m /home/YOUR_USERNAME/models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
-ngl 99 \
--host 0.0.0.0 \
--port 8080 \
-c 16384
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now llama-server
sudo systemctl status llama-server
```
Replace `YOUR_USERNAME`. Verify:
```bash
sudo journalctl -u llama-server -n 50 --no-pager | grep -iE "cuda|orin|offload"
# Want: ARCHS = 870, offloaded 29/29 layers to GPU, CUDA0: Orin
```
From any device on the LAN: `http://jetson.local:8080`. Built-in web UI, plus OpenAI-compatible API at `/v1/chat/completions`.
### Quantization notes
Don't be tempted by Q5/Q6/Q8 just because they fit on disk. On 8 GB unified memory:
- **Q4_K_M is the sweet spot.** Genuine quality is high; memory budget leaves room for context (16k), KV cache, and the rest of the system.
- **Q5_K_M** marginally better quality, ~10-15% slower if it fits with `-ngl 99`, doesn't fit at 16k context with full GPU offload — drops you to partial CPU.
- **Q6_K** doesn't fit at all with full GPU offload — auto-fit either rejects it or you drop dozens of layers to CPU, killing throughput.
Better quality moves at the same memory budget:
- **Larger context** (16k > 4k) for real conversation/RAG work
- **Specialized models** (Qwen Coder for code, etc.)
- **Different model family** at Q4_K_M (Qwen 3.5 9B vs Qwen 2.5 7B)
---
## Part 5: Image Generation (stable-diffusion.cpp Native Build)
Direct GPU SD inference. Works flawlessly on JP 7.2 because it compiles native sm_87 kernels.
### Build
```bash
cd ~
git clone --recursive https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp
cmake -B build -DSD_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87
cmake --build build --config Release -j$(nproc)
```
Build time: ~15 minutes.
### Models
```bash
mkdir -p ~/models/sd
cd ~/models/sd
# SD 1.5 (fast, 512px, ~5-10 sec/image):
wget https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors
# SDXL base (high quality, 1024px, ~90-120 sec/image with offload):
wget https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0_0.9vae.safetensors
# Better SDXL VAE (avoids saturation artifacts):
wget https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors
```
### Generate
SD 1.5 baseline:
```bash
./build/bin/sd-cli \
-m ~/models/sd/v1-5-pruned-emaonly.safetensors \
--type f16 \
-p "your prompt" \
-o ~/sd15.png \
--steps 20 --cfg-scale 7.5 -W 512 -H 512
```
SDXL with memory offload (required on 8 GB):
```bash
# Stop llama-server first to free memory:
sudo systemctl stop llama-server
./build/bin/sd-cli \
-m ~/models/sd/sd_xl_base_1.0_0.9vae.safetensors \
--vae ~/models/sd/sdxl_vae.safetensors \
--type f16 \
-p "your prompt" \
-o ~/sdxl.png \
--steps 25 --cfg-scale 6 -W 1024 -H 1024 \
--clip-on-cpu --diffusion-fa
```
Key flags for SDXL on 8 GB:
- `--clip-on-cpu`: keeps the 1.5 GB CLIP text encoders in RAM, frees GPU
- `--diffusion-fa`: flash attention reduces UNet activation memory
- `--vae-on-cpu`: only if `--clip-on-cpu --diffusion-fa` isn't enough — slows VAE decode significantly (5x+ longer)
Without offload flags, SDXL OOMs at the UNet allocation.
### Convenient alias
```bash
# Add to ~/.bashrc:
sdxl() {
cd ~/stable-diffusion.cpp
./build/bin/sd-cli \
-m ~/models/sd/sd_xl_base_1.0_0.9vae.safetensors \
--vae ~/models/sd/sdxl_vae.safetensors \
--type f16 --steps 25 --cfg-scale 6 -W 1024 -H 1024 \
--clip-on-cpu --diffusion-fa \
-p "$1" -o "$HOME/sdxl-$(date +%s).png"
cd -
}
# Then:
sdxl "your prompt here"
```
---
## Part 6: ComfyUI (CPU-only on JP 7.2)
ComfyUI works on JP 7.2 with the PyPI SBSA PyTorch wheel — but only on CPU for inference. GPU paths produce blank/NaN output due to the sm_87 kernel coverage issue described earlier. CPU mode is genuinely slow (~60-120 sec for SD 1.5 at 512px) but useful when you need the node-graph workflow UI.
If you want it:
```bash
# Create a Python 3.12 venv:
python3 -m venv ~/venv-comfyui
source ~/venv-comfyui/bin/activate
# Install PyTorch SBSA aarch64+cu126:
pip install torch --index-url https://download.pytorch.org/whl/cu126
# Plus torchvision/torchaudio from the same index to avoid CUDA version mismatch:
pip install --force-reinstall --index-url https://download.pytorch.org/whl/cu126 \
torchvision torchaudio
# Install ComfyUI:
cd ~
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
pip install -r requirements.txt
# Run in CPU-only mode:
python main.py --listen 0.0.0.0 --cpu
```
Browse to `http://jetson.local:8188`.
**When NVIDIA publishes JP 7.2 PyTorch wheels** (likely later in 2026), GPU inference in ComfyUI should "just work" by upgrading torch in this venv. Until then, use sd-cli for any real image generation work and ComfyUI only when the node graph specifically helps.
---
## Part 7: Practical Tips
### Memory management on 8 GB unified
The Jetson's 8 GB is shared between CPU and GPU. There's no separate "VRAM." Practical implications:
- **Stop services you're not using.** llama-server idle still holds the model in memory (~5 GB for a 7B Q4). Stop it before heavy SDXL work.
- **Watch jtop's GPU MEM column.** Tells you what's actually resident on the GPU side of allocations.
- **Swap is your safety net.** 16 GB swap on NVMe handles brief peaks; sustained swap thrashing means you've over-committed and need to back off.
- **First boot defaults can be misleading.** Default 2 GB swap, default 15W mode, default no clock pinning. Configure intentionally.
### Power supply matters
The Super dev kit can sustain ~25W under load (MAXN_SUPER + active workload + headroom for peaks). The included supply handles this. If you swap to a generic supply or an older Orin Nano supply (~27W max), you'll see OC throttle warnings under heavy multi-workload use. Symptoms:
- GUI notification "system throttled due to over-current"
- Brief clock reductions visible in jtop
Check throttle counters to distinguish real events from cosmetic notifications:
```bash
cat /sys/devices/platform/soctherm-oc-event/hwmon/hwmon4/oc1_event_cnt
cat /sys/devices/platform/soctherm-oc-event/hwmon/hwmon4/oc2_event_cnt
# 0 = no actual events; large numbers = real throttling
```
### LAN access and remote use
- mDNS works after hostname is set and Avahi is running
- macOS sometimes caches stale mDNS — `sudo killall -HUP mDNSResponder` fixes
- For permanent stability, add Jetson to `/etc/hosts` on your client machines (set a DHCP reservation in your router)
- For access from outside your LAN, Tailscale is the easiest path: 5 minutes to install, works from anywhere, end-to-end encrypted
### What to expect from local LLMs at this size class
Qwen 3.5 9B Q4_K_M (~10 tok/s) is genuinely useful for:
- General Q&A
- Code explanation, small functions, code review
- Writing tasks where coherence matters
- Reasoning through tradeoffs and decisions
- "First draft" of anything you'd otherwise paste into a chatbot
It's less suited to:
- Code generation across multiple files
- Long technical documents requiring sustained coherence
- Heavy math beyond high school
- Anything where you can't verify the answer
Calibrate expectations against the model size, not against frontier APIs. The setup is impressive for sub-25W; it's not a 70B+ replacement.
---
## Part 8: Things I'd do differently
If I were doing this setup again from scratch:
1. **Pre-config the OS image during SDK Manager flash.** Skip the OEM wizard. Without this, the first boot hangs unreachable.
2. **Plan for the lack of display.** Either have a DP→HDMI active adapter on hand, or a USB-TTL serial adapter for first-boot debug. The Jetson reference carrier has no HDMI and DP can be finicky with random monitors.
3. **Don't waste time on the prebuilt ecosystem first.** Skip dustynv containers, skip Ollama prebuilt. Go straight to llama.cpp + sd.cpp from source. Saves hours.
4. **Set up swap and MAXN_SUPER on day one** before doing any AI workload.
5. **Lock the hostname before assuming `.local` works.** mDNS depends on it.
6. **Buy a known-good case from someone who tested it with the Super specifically.** Third-party metal cases often don't account for the Super's taller heatsink or revised carrier; 3D-printed designs from people who own the kit are more reliable.
---
## Sources and credits
This guide reflects a setup session on June 5-6, 2026, working through what does and doesn't function on JP 7.2 in real time. The key technical findings (PyTorch SBSA wheel sm_87 behavior, container CUDA forward-compat failure mode, Ollama JetPack detection limits) aren't documented elsewhere yet because the JP 7.2 release is recent. Updates welcome — file issues or send corrections as the ecosystem matures.
Thanks. I appreciate your help.
@darin5 this is truly amazing. thank you!
question- do you think i’ll be able to get yolo running with the new jetpack? like you, we haven’t been able to get the yolo container running, but since your list shows TensorRT 10.16.2 as woirking, perhaps I’ll be able to run yolo directly?
For those interested, I’ve posted a comprehensive solution for running Ollama with full CUDA acceleration on JetPack 7.2 in the JetPack 7.2 feedback thread. The solution is based on the valuable information shared by @darin5. Feel free to check it out if you’re facing similar issues!
@ts121 , you should be able to get yolo running with a similar approach. Jetson is especially good at that sort of thing.
Another potential gotcha: I lost my Jetson clocks after a reboot because nvpmodel failed to set what was already set, and therefore failed to set the clocks in the next line in /etc/systemd/system/jetson_clocks.service
Note the ‘-’ prefix in the nvpmodel line. That will prevent it from stopping and not setting the clocks:
[Unit]
Description=Apply MAXN_SUPER and pin Jetson clocks at boot
After=multi-user.target
[Service]
Type=oneshot
ExecStart=-/usr/sbin/nvpmodel -m 2
ExecStart=/usr/bin/jetson_clocks
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
why are you using the cu126 wheel?
if we use just use the normal pytorch index like you do, they have cuda 13.2 right?
i understand the jetson index only has cuda 13 SBSA at the latest, so we woulnd’t be able to get 13.2 if we wanted to use the jetson index, but you didnt link that one, you just said to use the 12.6 from the normal index which I don’t understand
Very fair question, and I should have pointed it out: cu130 doesn’t appear to be on PyPI yet, and cu128 was actually slower than cu126 (~1.8x slower)
They have CUDA 13.2 wheels but you still will get things like this:
Warning (from warnings module):
File “/home/dave/.local/lib/python3.12/site-packages/torch/cuda/init.py”, line 418
_warn_unsupported_code(d, device_cc, code_ccs)
UserWarning: Found GPU0 Orin which is of compute capability (CC) 8.7.
The following list shows the CCs this version of PyTorch was built for and the hardware CCs it supports:
- 8.0 which supports hardware CC >=8.0,<9.0 except {8.7}
- 9.0 which supports hardware CC >=9.0,<10.0
- 10.0 which supports hardware CC >=10.0,<11.0 except {10.1}
- 11.0 which supports hardware CC >=11.0,<12.0
- 12.0 which supports hardware CC >=12.0,<13.0
- 12.0 which supports hardware CC >=12.0,<13.0
No published PyTorch CUDA builds for release 2.13.0.dev20260608+cu132 support this GPU. Visit Get Started to find a compatible release.
johnny says it’s fine and we can ignore the warning they just missed to remove, pytorch don’t build by specific codegen, build by family, so you can see 75, 80 90 100 110 120
Interesting. Thank you.
| Tool | Status | Notes |
|---|---|---|
| PyTorch SBSA aarch64+cu126 — general use | ✅ Works | Basic ops, LLMs, classification, training. PTX JIT from sm_80 to sm_87 covers most code paths despite the wheel’s except {8.7} warning. |
| PyTorch SBSA — diffusion UNets | ⚠️ Produces NaN | Specific high-performance kernel paths in attention/conv used by diffusion models appear to be SASS-only without sm_87 coverage. Workaround: use sd-cli or wait for native Jetson wheels. |
Thank you for this. So just say at this point 7.2 is basically useless?