NVIDIA GPU in Kubernetes (MicroK8s) on Nvidia Jetson AGX Thor

Robotics & Edge Computing Jetson Systems Jetson Thor
1

Hello all,

MIG works on the Nvidia Thor at the driver level, but on the MicroK8s path you can’t schedule pods per-slice, every pod gets the whole GPU. Details and exact commands below.

My setup:

  • Nvidia Jetson Thor, 20 SMs
  • JetPack 7.x, driver 595.78, CUDA 13.2
  • MicroK8s 1.35, nvidia-device-plugin v0.16.2

Step 1. Check your JetPack version

MIG is not functional on 7.0 or 7.1. If you’re on either, that’s your problem:

nvidia-smi mig -i 0 -lgip

You’ll get:

Failed to display GPU instance profiles: Unknown Error

You need JetPack 7.2 or newer.

Step 2. Make it headless

The display manager holds the GPU. If gdm3/Xorg is running, MIG creation fails with “In use by another client”. So do this over SSH:

sudo systemctl isolate multi-user.target

Then enable persistence mode and MIG:

sudo nvidia-smi -pm 1
sudo nvidia-smi -i 0 -mig 1

One thing that confused me at first: the profiles all show 0gb / 0.00 memory. That’s not a limit.

Step 3. Make the slices

The Thor only has 20 SMs, so a 3g slice (profile 0) uses the whole thing and can’t be paired with anything. For two workloads I used a 2g (12 SMs) plus a 1g (6 SMs), which do fit together:

sudo nvidia-smi mig -i 0 -cgi 83 -C    # 2g.0gb+gfx -> Device 0
sudo nvidia-smi mig -i 0 -cgi 78 -C    # 1g.0gb+me  -> Device 1

Check they’re there:

nvidia-smi -L

You should see both MIG devices listed. Note MIG does not survive a reboot — you have to recreate the slices each time. I wrapped this in a small mig-setup.sh so I can just rerun it.

Setup on the Thor

sudo systemctl isolate multi-user.target     # stop gdm3/Xorg
sudo nvidia-smi -pm 1                        # persistence mode
sudo nvidia-smi -i 0 -mig 1                  # enable MIG

MIG is not persistent, recreate after reboot (a small mig-setup.sh handles this).

MicroK8s GPU setup

sudo snap install microk8s --classic && sudo snap install kubectl --classic && sudo snap install helm --classic
sudo microk8s status --wait-ready
sudo microk8s config > /tmp/kubeconfig && sudo install -m 600 -o root -g root /tmp/kubeconfig /root/.kube/config

# point containerd at the host runtime binary 
sudo sed -i 's|BinaryName = "nvidia-container-runtime"|BinaryName = "/usr/bin/nvidia-container-runtime"|' \
  /var/snap/microk8s/current/args/containerd-template.toml
sudo microk8s stop && sleep 5 && sudo microk8s start

# RuntimeClass
kubectl apply -f - <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: nvidia
handler: nvidia-container-runtime
EOF

# device plugin 

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update

helm upgrade -i nvidia-device-plugin \
  nvidia/nvidia-device-plugin \
  -n gpu-operator \
  --version 0.9.0 \
  --set image.tag=v0.19.3 \
  --set deviceIDStrategy=index

kubectl describe node localhost.localdomain | grep nvidia

MIG partially works at the driver level on Jetson Thor (JetPack 7.2). However, with the MicroK8s and jetpack (CSV/Tegra) path, MIG slices cannot be scheduled individually from pods. Pods are assigned the entire integrated GPU rather than individual MIG slices. NVIDIA Device Plugin on Thor should support nvidia device plugin in advertising and allocating MIG slices to pods and thor must have full kernel-level support for MIG. I raised a github issue.: [Bug]: k8s-device-plugin fails to detect Jetson platform · Issue #1963 · NVIDIA/k8s-device-plugin · GitHub

3

You could try to build nvidia/nvidia-device-plugin from source. v0.19.3 was released June 22 and this commit that is on point was merged into main on July 21.

gh pr view 1621 \
  --repo NVIDIA/k8s-device-plugin \
  --json number,title,baseRefName,mergedAt,mergeCommit
{
  "baseRefName": "main",
  "mergeCommit": {
    "oid": "88a79d7e98c146e13a6bbb48fff6effdc87e541d"
  },
  "mergedAt": "2026-07-21T20:39:24Z",
  "number": 1621,
  "title": "Add configurable allocation policy (packed/distributed) 
for replicated and MIG resources"
}
4

Hi, @whitesscott, I think we need a response from NVIDIA, since we are awaiting full MIG kernel support on the Jetson Thor. Once it is ready, MIG support should also be added to the NVIDIA Device Plugin.

6

Hi, @shahizat

MIG support is added in JetPack 7.2+.

Regarding your comment that “MIG partially works at the driver level,” do you mean there are some issues in nvidia-device-plugin when deploying MicroK8s?

Are you able to use profile 78,83 with CUDA apps directly?
We are currently checking the Kubernetes issue, but just want to make sure no other issues were missed.

Thanks!