Streaming Isaac Sim 5.1.0 to Web Browser with Character Controller

Omniverse Isaac Sim
1

Hi, I am working on a project that requires streaming Isaac Sim simulation content to web browsers, where one or multiple users can control their own first-person view (FPV) characters through a web interface. If multiple user are enabled. The requirements are to enable WASD and mouse controls from the web interface and to provide each user with an independent FPV camera view.

I noticed that NVIDIA provides a livestream client application with WebRTC support. I’m wondering if I can develop a custom web-based client instead of using the Windows application.

Additionally, I don’t need to stream the entire Isaac Sim GUI, I only want to stream the viewport showing the simulation scene (the 3D environment content).

Are there any ways to implement this, possibly using WebRTC?

Thanks for your help.

Isaac Sim Version

[ ] 5.1.0
5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 24.04
Ubuntu 22.04
Ubuntu 20.04
[ ] Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: RTX 5000 Ada
  • Driver Version:
3

  1. Developing a web-based client instead of using NVIDIA’s WebRTC streaming client

    Yes, this should be technically possible, but I have not personally tried implementing a custom web-based client.

  2. Streaming only the simulation viewport instead of the full Isaac Sim GUI

    Isaac Sim supports full-screen mode (for example, via F11), which effectively hides most GUI elements and presents the simulation view as the primary output.