Open Voice OS on Jetson Orin Nano: Offline AI Assistant with LLM + TTS + STT on K3s!

Hello Jetson community!

I would like first to say thank you for all the answers & questions in this forum, I would not have been able to go that far without it (especially the one with Ollama needed to be =< 0.5.7).

In this video, we demonstrate Open Voice OS running a fully offline AI voice assistant stack on the ultra-compact NVIDIA Jetson Orin Nano, orchestrated with K3s and managed by Rancher.

Get a glimpse of a truly modern, edge-native AI assistant powered entirely on-device, featuring:

🧠 Local LLM (Large Language Model) for smart, private conversations
🗣️ High-quality Text-to-Speech (TTS) with natural voice generation
🎙️ Fast and accurate Speech-to-Text (STT) recognition
🧩 All deployed as containers with K3s lightweight Kubernetes
🖥️ Centralized lifecycle management via Rancher

This setup combines the best of open-source voice tech and cloud-native tooling for a modular, scalable, and 100% private experience — no internet required!

🔧 Tech Stack Highlights:

  • Open Voice OS (voice assistant)
  • Whisper (Speech-To-Text)
  • Piper (Text-To-Speech)
  • Ollama + Qwen2.5:1.5b (LLM)
  • NVIDIA Jetson Orin Nano (8GB)
  • K3s (lightweight Kubernetes)
  • Rancher (K8s management UI)
2 Likes

Please find a bit more details.

Only three skills are running:

  • Date & Time
  • Weather
  • Fallback ChatGPT
goldyfruit@jetson01:~$ jetson_release 
Software part of jetson-stats 4.3.2 - (c) 2024, Raffaello Bonghi
Model: NVIDIA Jetson Orin Nano Engineering Reference Developer Kit Super - Jetpack 6.2 [L4T 36.4.3]
NV Power Mode[2]: MAXN_SUPER
Serial Number: [XXX Show with: jetson_release -s XXX]
Hardware:
 - P-Number: p3767-0005
 - Module: NVIDIA Jetson Orin Nano (Developer kit)
Platform:
 - Distribution: Ubuntu 22.04 Jammy Jellyfish
 - Release: 5.15.148-tegra
jtop:
 - Version: 4.3.2
 - Service: Active
Libraries:
 - CUDA: 12.6.68
 - cuDNN: 1.0
 - TensorRT: 10.3.0.30
 - VPI: 3.2.4
 - Vulkan: 1.3.204
 - OpenCV: 4.8.0 - with CUDA: NO

goldyfruit@jetson01:~$ sudo kubectl get pods -o wide -A
NAMESPACE             NAME                                                              READY   STATUS    RESTARTS       AGE     IP            NODE       NOMINATED NODE   READINESS GATES
cattle-ai             ollama-1-1743641548-687546cb6c-xspgf                              1/1     Running   0              43h     10.42.0.206   jetson01   <none>           <none>
cattle-fleet-system   fleet-agent-0                                                     2/2     Running   28 (46h ago)   9d      10.42.0.196   jetson01   <none>           <none>
cattle-system         cattle-cluster-agent-7c455cf84f-w742q                             1/1     Running   43 (46h ago)   8d      10.42.0.192   jetson01   <none>           <none>
cattle-system         rancher-webhook-c8b6646fb-r4tf7                                   1/1     Running   16 (46h ago)   9d      10.42.0.195   jetson01   <none>           <none>
cattle-system         system-upgrade-controller-56696956b-9t9xr                         1/1     Running   15 (46h ago)   9d      10.42.0.183   jetson01   <none>           <none>
gpu-operator          gpu-operator-1743642830-node-feature-discovery-gc-65d9d7f5865kj   1/1     Running   12 (46h ago)   9d      10.42.0.190   jetson01   <none>           <none>
gpu-operator          gpu-operator-1743642830-node-feature-discovery-master-5b85dhl54   1/1     Running   4 (46h ago)    2d10h   10.42.0.181   jetson01   <none>           <none>
gpu-operator          gpu-operator-1743642830-node-feature-discovery-worker-b7l42       1/1     Running   6 (46h ago)    2d10h   10.42.0.191   jetson01   <none>           <none>
gpu-operator          gpu-operator-78c479c76f-wsz4f                                     1/1     Running   4 (46h ago)    2d10h   10.42.0.186   jetson01   <none>           <none>
kube-system           coredns-9b9499f65-zm6jn                                           1/1     Running   15 (46h ago)   9d      10.42.0.194   jetson01   <none>           <none>
kube-system           local-path-provisioner-86d975c679-ng7jx                           1/1     Running   15 (46h ago)   9d      10.42.0.193   jetson01   <none>           <none>
metallb-system        metallb-0-1743641145-controller-78fb5ffb8f-95tsv                  1/1     Running   12 (46h ago)   9d      10.42.0.197   jetson01   <none>           <none>
metallb-system        metallb-0-1743641145-speaker-n6z79                                4/4     Running   48 (46h ago)   9d      10.17.2.201   jetson01   <none>           <none>
ovos                  ovos-audio-697c84cdf9-d75vt                                       1/1     Running   3 (46h ago)    2d13h   10.17.2.201   jetson01   <none>           <none>
ovos                  ovos-core-6b878f775f-6tzq4                                        1/1     Running   2 (46h ago)    2d9h    10.17.2.201   jetson01   <none>           <none>
ovos                  ovos-listener-746f447d4b-phrt9                                    1/1     Running   0              12h     10.17.2.201   jetson01   <none>           <none>
ovos                  ovos-messagebus-57446bdfbd-p2bbm                                  1/1     Running   9 (46h ago)    3d19h   10.17.2.201   jetson01   <none>           <none>
ovos                  ovos-phal-7b6b47fbcb-qjsbn                                        1/1     Running   3 (46h ago)    2d13h   10.17.2.201   jetson01   <none>           <none>
ovos                  ovos-skill-fallback-chatgpt-7df8fb4d55-4n5bv                      1/1     Running   0              13h     10.17.2.201   jetson01   <none>           <none>
ovos                  stt-whisper-659ffdb5df-6wrcv                                      1/1     Running   2 (46h ago)    2d      10.42.0.188   jetson01   <none>           <none>
ovos                  tts-piper-564b99687d-tcms2                                        1/1     Running   3 (46h ago)    2d13h   10.42.0.184   jetson01   <none>           <none>
goldyfruit@jetson01:~$ sudo kubectl describe nodes
Name:               jetson01
Roles:              control-plane,etcd,master,worker
Labels:             beta.kubernetes.io/arch=arm64
                    beta.kubernetes.io/instance-type=k3s
                    beta.kubernetes.io/os=linux
                    cattle.io/os=linux
                    feature.node.kubernetes.io/cpu-cpuid.AES=true
                    feature.node.kubernetes.io/cpu-cpuid.ASIMD=true
                    feature.node.kubernetes.io/cpu-cpuid.ASIMDDP=true
                    feature.node.kubernetes.io/cpu-cpuid.ASIMDHP=true
                    feature.node.kubernetes.io/cpu-cpuid.ASIMDRDM=true
                    feature.node.kubernetes.io/cpu-cpuid.ATOMICS=true
                    feature.node.kubernetes.io/cpu-cpuid.CPUID=true
                    feature.node.kubernetes.io/cpu-cpuid.CRC32=true
                    feature.node.kubernetes.io/cpu-cpuid.DCPOP=true
                    feature.node.kubernetes.io/cpu-cpuid.EVTSTRM=true
                    feature.node.kubernetes.io/cpu-cpuid.FLAGM=true
                    feature.node.kubernetes.io/cpu-cpuid.FP=true
                    feature.node.kubernetes.io/cpu-cpuid.FPHP=true
                    feature.node.kubernetes.io/cpu-cpuid.ILRCPC=true
                    feature.node.kubernetes.io/cpu-cpuid.LRCPC=true
                    feature.node.kubernetes.io/cpu-cpuid.PACA=true
                    feature.node.kubernetes.io/cpu-cpuid.PACG=true
                    feature.node.kubernetes.io/cpu-cpuid.PMULL=true
                    feature.node.kubernetes.io/cpu-cpuid.SHA1=true
                    feature.node.kubernetes.io/cpu-cpuid.SHA2=true
                    feature.node.kubernetes.io/cpu-cpuid.USCAT=true
                    feature.node.kubernetes.io/cpu-hardware_multithreading=false
                    feature.node.kubernetes.io/cpu-model.family=15
                    feature.node.kubernetes.io/cpu-model.id=54305
                    feature.node.kubernetes.io/cpu-model.vendor_id=ARM
                    feature.node.kubernetes.io/kernel-config.NO_HZ_IDLE=true
                    feature.node.kubernetes.io/kernel-config.PREEMPT=true
                    feature.node.kubernetes.io/kernel-version.full=5.15.148-tegra
                    feature.node.kubernetes.io/kernel-version.major=5
                    feature.node.kubernetes.io/kernel-version.minor=15
                    feature.node.kubernetes.io/kernel-version.revision=148
                    feature.node.kubernetes.io/memory-swap=true
                    feature.node.kubernetes.io/pci-10ec.present=true
                    feature.node.kubernetes.io/storage-nonrotationaldisk=true
                    feature.node.kubernetes.io/system-os_release.ID=ubuntu
                    feature.node.kubernetes.io/system-os_release.VERSION_ID=22.04
                    feature.node.kubernetes.io/system-os_release.VERSION_ID.major=22
                    feature.node.kubernetes.io/system-os_release.VERSION_ID.minor=04
                    kubernetes.io/arch=arm64
                    kubernetes.io/hostname=jetson01
                    kubernetes.io/os=linux
                    node-role.kubernetes.io/control-plane=true
                    node-role.kubernetes.io/etcd=true
                    node-role.kubernetes.io/master=true
                    node-role.kubernetes.io/worker=true
                    node.kubernetes.io/instance-type=k3s
                    plan.upgrade.cattle.io/system-agent-upgrader=75d7f5c8c727aa0851795b67d0069012225814d640ce436707f15035
                    rke.cattle.io/machine=4eb3b210-47eb-4944-b4d1-b482aacde079
Annotations:        alpha.kubernetes.io/provided-node-ip: 10.17.2.201
                    cluster.x-k8s.io/cluster-name: ai-jetson-ai
                    cluster.x-k8s.io/cluster-namespace: fleet-default
                    cluster.x-k8s.io/labels-from-machine: 
                    cluster.x-k8s.io/machine: custom-ff349468a93f
                    etcd.k3s.cattle.io/local-snapshots-timestamp: 2025-04-12T07:56:07-04:00
                    etcd.k3s.cattle.io/node-address: 10.17.2.201
                    etcd.k3s.cattle.io/node-name: jetson01-9e3697ca
                    flannel.alpha.coreos.com/backend-data: {"VNI":1,"VtepMAC":"b6:e4:cc:46:06:7e"}
                    flannel.alpha.coreos.com/backend-type: vxlan
                    flannel.alpha.coreos.com/kube-subnet-manager: true
                    flannel.alpha.coreos.com/public-ip: 10.17.2.201
                    k3s.io/hostname: jetson01
                    k3s.io/internal-ip: 10.17.2.201
                    k3s.io/node-args:
                      ["server","--agent-token","********","--cluster-init","true","--disable","servicelb","--disable","traefik","--disable","metrics-server","-...
                    k3s.io/node-config-hash: 47OAZ5FSS5HJUFSALIOJWCEH2SBFJXO2FF2DF6VZAYX6YTKNLQPA====
                    k3s.io/node-env: {"K3S_DATA_DIR":"/var/lib/rancher/k3s"}
                    management.cattle.io/pod-limits: {"cpu":"500m","memory":"6152Mi"}
                    management.cattle.io/pod-requests: {"cpu":"415m","memory":"490Mi","pods":"21"}
                    nfd.node.kubernetes.io/feature-labels:
                      cpu-cpuid.AES,cpu-cpuid.ASIMD,cpu-cpuid.ASIMDDP,cpu-cpuid.ASIMDHP,cpu-cpuid.ASIMDRDM,cpu-cpuid.ATOMICS,cpu-cpuid.CPUID,cpu-cpuid.CRC32,cpu...
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 02 Apr 2025 17:28:37 -0400
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  jetson01
  AcquireTime:     <unset>
  RenewTime:       Sat, 12 Apr 2025 08:02:44 -0400
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  EtcdIsVoter      True    Sat, 12 Apr 2025 07:58:03 -0400   Wed, 02 Apr 2025 17:29:48 -0400   MemberNotLearner             Node is a voting member of the etcd cluster
  MemoryPressure   False   Sat, 12 Apr 2025 08:01:53 -0400   Wed, 02 Apr 2025 17:28:37 -0400   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Sat, 12 Apr 2025 08:01:53 -0400   Wed, 02 Apr 2025 17:28:37 -0400   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Sat, 12 Apr 2025 08:01:53 -0400   Wed, 02 Apr 2025 17:28:37 -0400   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Sat, 12 Apr 2025 08:01:53 -0400   Wed, 02 Apr 2025 17:28:38 -0400   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  10.17.2.201
  Hostname:    jetson01
Capacity:
  cpu:                6
  ephemeral-storage:  981879348Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  hugepages-32Mi:     0
  hugepages-64Ki:     0
  memory:             7802780Ki
  nvidia.com/gpu:     0
  pods:               110
Allocatable:
  cpu:                6
  ephemeral-storage:  955172228986
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  hugepages-32Mi:     0
  hugepages-64Ki:     0
  memory:             7802780Ki
  nvidia.com/gpu:     0
  pods:               110
System Info:
  Machine ID:                 5dbfb12414a3456d9014d88183e338b1
  System UUID:                5dbfb12414a3456d9014d88183e338b1
  Boot ID:                    3a7019ad-5fc8-48e7-88a9-e100c398947a
  Kernel Version:             5.15.148-tegra
  OS Image:                   Ubuntu 22.04.5 LTS
  Operating System:           linux
  Architecture:               arm64
  Container Runtime Version:  docker://28.0.4
  Kubelet Version:            v1.31.7+k3s1
  Kube-Proxy Version:         v1.31.7+k3s1
PodCIDR:                      10.42.0.0/24
PodCIDRs:                     10.42.0.0/24
ProviderID:                   k3s://jetson01
Non-terminated Pods:          (21 in total)
  Namespace                   Name                                                               CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                                               ------------  ----------  ---------------  -------------  ---
  cattle-ai                   ollama-1-1743641548-687546cb6c-xspgf                               0 (0%)        0 (0%)      0 (0%)           0 (0%)         43h
  cattle-fleet-system         fleet-agent-0                                                      0 (0%)        0 (0%)      0 (0%)           0 (0%)         9d
  cattle-system               cattle-cluster-agent-7c455cf84f-w742q                              0 (0%)        0 (0%)      0 (0%)           0 (0%)         8d
  cattle-system               rancher-webhook-c8b6646fb-r4tf7                                    0 (0%)        0 (0%)      0 (0%)           0 (0%)         9d
  cattle-system               system-upgrade-controller-56696956b-9t9xr                          0 (0%)        0 (0%)      0 (0%)           0 (0%)         9d
  gpu-operator                gpu-operator-1743642830-node-feature-discovery-gc-65d9d7f5865kj    10m (0%)      0 (0%)      128Mi (1%)       1Gi (13%)      9d
  gpu-operator                gpu-operator-1743642830-node-feature-discovery-master-5b85dhl54    100m (1%)     0 (0%)      128Mi (1%)       4Gi (53%)      2d10h
  gpu-operator                gpu-operator-1743642830-node-feature-discovery-worker-b7l42        5m (0%)       0 (0%)      64Mi (0%)        512Mi (6%)     2d10h
  gpu-operator                gpu-operator-78c479c76f-wsz4f                                      200m (3%)     500m (8%)   100Mi (1%)       350Mi (4%)     2d10h
  kube-system                 coredns-9b9499f65-zm6jn                                            100m (1%)     0 (0%)      70Mi (0%)        170Mi (2%)     9d
  kube-system                 local-path-provisioner-86d975c679-ng7jx                            0 (0%)        0 (0%)      0 (0%)           0 (0%)         9d
  metallb-system              metallb-0-1743641145-controller-78fb5ffb8f-95tsv                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         9d
  metallb-system              metallb-0-1743641145-speaker-n6z79                                 0 (0%)        0 (0%)      0 (0%)           0 (0%)         9d
  ovos                        ovos-audio-697c84cdf9-d75vt                                        0 (0%)        0 (0%)      0 (0%)           0 (0%)         2d13h
  ovos                        ovos-core-6b878f775f-6tzq4                                         0 (0%)        0 (0%)      0 (0%)           0 (0%)         2d9h
  ovos                        ovos-listener-746f447d4b-phrt9                                     0 (0%)        0 (0%)      0 (0%)           0 (0%)         12h
  ovos                        ovos-messagebus-57446bdfbd-p2bbm                                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         3d19h
  ovos                        ovos-phal-7b6b47fbcb-qjsbn                                         0 (0%)        0 (0%)      0 (0%)           0 (0%)         2d13h
  ovos                        ovos-skill-fallback-chatgpt-7df8fb4d55-4n5bv                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         13h
  ovos                        stt-whisper-659ffdb5df-6wrcv                                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         2d
  ovos                        tts-piper-564b99687d-tcms2                                         0 (0%)        0 (0%)      0 (0%)           0 (0%)         2d13h
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                415m (6%)   500m (8%)
  memory             490Mi (6%)  6152Mi (80%)
  ephemeral-storage  0 (0%)      0 (0%)
  hugepages-1Gi      0 (0%)      0 (0%)
  hugepages-2Mi      0 (0%)      0 (0%)
  hugepages-32Mi     0 (0%)      0 (0%)
  hugepages-64Ki     0 (0%)      0 (0%)
  nvidia.com/gpu     0           0
Events:              <none>
1 Like