Local streaming via WebRTC dont' work

Omniverse Core Platform Kit App Streaming
1

Hello, I have a problem with my WebRTC

Operating System:
Linux
Kit Version:
110 (Kit App Template)
Kit Template:
USD Composer
GPU Hardware:
A series (Blackwell)
GPU Driver:
Latest

Work Flow: Local streaming via WebRTC

Main Issue: White screen after clicking “Next” on the web viewer when selecting “UI for any streaming app”. The page loads the NVIDIA header but the stream never appears.

image
image1907×1019 19.1 KB

Reproduction Steps: Kit app is running on a remote Linux server (Ubuntu 24.04, ARM aarch64) accessed via SSH tunnel through VS Code Remote SSH

  1. Web viewer (web-viewer-sample) is running locally on the server via npm run dev on port 5173, forwarded to my Windows PC via VS Code
  2. stream.config.json is set to "source": "local" with "server": "127.0.0.1" and "signalingPort": 49100
  3. Kit is launched with: ./repo.sh launch kuleuven.des_composer_streaming.kit -- --no-window
  4. Opening localhost:5173 → selecting “UI for any streaming app” → clicking Next → white screen

How to solve this problem ?

2

Ok so some questions…

  1. Does this work locally, meaning running the server and the viewer on the same Windows PC?
  2. Does this work fully on the remote Linux Server, meaning running both the web server and the http page on it?
  3. If so to both, does it work in reverse? Run the webserver on the Windows PC and run the http on the Linux Server.

Make sure ALL of those work fully, on both machines and then get back to me.

3

Hi,

Thanks for your questions — here’s a clear update on my side:

  • The Linux server I’m working with is headless, meaning it has no graphical interface. Because of this, I cannot directly visualize or run a viewer on it. This is precisely why I’m trying to use WebRTC, to stream the interface remotely.

  • Due to this limitation, I’m currently unable to fully test scenarios 2 and 3 (running both the web server and viewer on the Linux server, or running the viewer from the Linux server).

  • However, I did test the setup using npm run dev:

    • When launching the streaming from my Windows PC, I can access the viewer.

    • When launching it from the Linux server (via terminal) and accessing the viewer from my Windows PC, I also reach the interface.

  • In both cases, I get the option “UI for any streaming app”, but the screen remains completely white and nothing is displayed.

So at this stage, the connection seems to initialize, but the stream itself is not rendering.

Let me know how you’d like me to proceed or if you want me to investigate a specific part further.

Best regards,

4

Ok, so you cannot have a completely headless system with Omniverse. You can run the kit app headless, meaning no GUI, but the SYSTEM itself needs to be a fully GUI system. Monitor, or virtual monitor, fully OS GUI enabled. So again, you can run Omniverse headless, but the RTX GPU needs to attach to a system level GUI. We cannot run Omniverse like a low level SSH server.

This explains why your Windows PC is ready to go but there is no stream. The Linux Server cannot generate a visual stream without RTX and RTX cannot run without a system level GUI. So for Linux you need at least basic graphic drivers like ServerX.

5

Here is a full detailed breakdown from AI:

Omniverse Kit can run “headless”, but it still needs a proper NVIDIA GPU driver and a display stack (or explicit no‑window mode). On a bare Linux server with no GUI/X server, Kit will usually fail to start or give GLFW/display errors. forums.developer.nvidia

Why headless Kit often fails

  • Kit uses OpenGL/Vulkan and expects either:
    • An X11/Wayland session with a working NVIDIA driver, or
    • Headless mode with --no-window and a full GUI driver installed (not just the compute driver). docs.omniverse.nvidia
  • NVIDIA’s own guidance: the system must be configured “as if you have a monitor present”; otherwise, GLFW initialization fails because there is no display or swapchain. forums.developer.nvidia

Common symptoms:

  • Errors like GLFW initialization failed, “no displays found,” or similar.
  • Isaac Sim / Kit apps exiting immediately on startup on a server install.

Options: lightweight GUI vs true headless

1) Install a lightweight GUI + X server

On something like Ubuntu Server, the simplest reliable path is to add a small desktop (Xfce, LXQt) so the NVIDIA driver has an X display to bind to. cherryservers

Typical steps (Ubuntu/Debian style):

  1. Install desktop environment and display manager, e.g. Xfce + lightdm:
    sudo apt update
sudo apt install xfce4 lightdm
    or use another desktop from the guide you prefer. youtube
  2. Install the proprietary NVIDIA driver (matching Omniverse’s recommendations).
  3. Reboot, log into the GUI once (locally or via something like xRDP/VNC).
  4. Run Kit/Isaac Sim from that session, or start it under that X display (e.g. via DISPLAY=:0 over SSH).

This gives you a full X11 environment so Kit thinks a monitor is attached, even if you only access it remotely.

2) X server but no full desktop

If you do not want a full desktop:

  • Install the NVIDIA driver and xorg, and configure a headless X server with AllowEmptyInitialConfiguration so X starts even without a physical monitor. forums.developer.nvidia
  • Optionally use a virtual display/Xvfb or a dummy HDMI plug to simulate a monitor.

Then start Kit with something like:

DISPLAY=:0 /path/to/kit.app.sh

3) True no‑window headless

For some streaming/automation scenarios, you can run Kit with the --no-window flag (names can vary slightly by app) so it does not create any visible window. Even then: forums.developer.nvidia

  • You still need the full NVIDIA driver stack, and
  • Depending on the build, a display or at least proper EGL headless support.

Check your specific Kit/Isaac app docs for the exact headless flags and requirements.