NVIDIA TOKKIO

Omniverse Isaac Sim
1

Hi everyone,

I’m hoping to get some community feedback on an issue I’m facing while deploying the NVIDIA ACE / Tokkio 5.0.0-GA suite on a bare-metal Kubernetes cluster.

The High-Level Goal: We are setting up the full Tokkio stack on Kubernetes to power a digital human, including the Unreal renderer, WebRTC streaming, and the Tokkio UI/API.

The Core Problem: While the base services are running and I can access the Tokkio UI (the NVIDIA logo appears), the streaming session fails to start. The browser’s developer console consistently shows the error: Failed to start inbound stream for VST (VST_INBOUND_STREAM_ERROR)

Diagnosis & Key Findings: After some investigation, I believe this WebRTC error is a symptom of a deeper issue with the backend services. Here’s a summary of what I’ve found:

  • What’s Working:

    • The Kubernetes cluster is healthy, and the NVIDIA GPU Operator is running correctly.

    • Core microservices like ia-unreal-renderer, vms-vms, riva-speech, and tokkio-ui are deployed and in a Running state.

    • Firewall rules and NodePorts are configured correctly; I’ve verified with tcpdump that traffic is reaching the host.

  • Where It Fails:

    • API Endpoints Return 404: All HTTP requests to the Tokkio API and ACE Configurator endpoints are met with a 404 Not Found response from the OpenResty/Nginx ingress.

    • Missing Deployments: My main suspicion is that the tokkio-api and streambridge deployments are missing or were not enabled during the installation. Running kubectl get deployments seems to confirm their absence.

    • Redis Connectivity: Some pods are logging “Waiting for redis,” even though the redis-timeseries pod and its service are running correctly with a valid endpoint.

My Current Theory: The VST_INBOUND_STREAM_ERROR occurs because the Tokkio UI cannot communicate with the necessary backend APIs (tokkio-api, streambridge) to complete the WebRTC handshake. These APIs appear to be not deployed, which is why the ingress controller returns a 404.

My Question for the Community: Has anyone encountered a similar situation where the tokkio-api or streambridge services don’t deploy by default with the standard charts/manifests? Are there specific configuration values I should double-check to ensure these critical components are enabled and correctly routed by the ingress?

Any guidance or suggestions would be greatly appreciated. I’m happy to provide more detailed logs or configs if needed.

Thanks in advance for your help!

2

Hi @cquiala12,

This question is about NVIDIA ACE / Tokkio deployment, which is a separate product from Isaac Sim. This forum category is for Isaac Sim-specific questions.

For Tokkio deployment issues, I’d recommend:

Regarding your specific issue — missing tokkio-api and streambridge deployments are typically caused by certain Helm sub-charts not being enabled. Double-check your Helm values file for flags like tokkio-api.enabled and streambridge.enabled,
and ensure your chart version matches the 5.0.0-GA deployment guide.

Closing this as it’s outside the scope of the Isaac Sim forum.