Enabling WebRTC Connection in Dockerized Python App: Configurations Needed for Browser Access

Omniverse Isaac Sim
1

Hello,

I have a Python application that needs to run in Docker’s headless mode. However, I’ve encountered an issue where the application can only be accessed via a Streaming Client rather than through WebRTC. Are there any configurations I need to adjust in order to enable WebRTC connections via a web browser?

Thank you

Below is an example that I’ve been using for testing purposes, which currently only connects through the Streaming Client.

my_application.py

from omni.isaac.kit import SimulationApp
simulation_app = SimulationApp({"headless": True}) # we can also run as headless.

from omni.isaac.core import World
from omni.isaac.core.objects import DynamicCuboid
import numpy as np
import time

world = World()
world.scene.add_default_ground_plane()
fancy_cube =  world.scene.add(
    DynamicCuboid(
        prim_path="/World/random_cube",
        name="fancy_cube",
        position=np.array([0, 0, 1.0]),
        scale=np.array([0.5015, 0.5015, 0.5015]),
        color=np.array([0, 0, 1.0]),
    ))

world.reset()
for i in range(5000):  
    time.sleep(0.1)
    position, orientation = fancy_cube.get_world_pose()
    linear_velocity = fancy_cube.get_linear_velocity()
    # will be shown on terminal
    print("Cube position is : " + str(position))
    print("Cube's orientation is : " + str(orientation))
    print("Cube's linear velocity is : " + str(linear_velocity))
    world.step(render=True) # execute one physics step and one rendering step
simulation_app.close()

docker-compose.yml

version: "3.8"
services:
  isaac-sim:
    container_name: isaac-sim
    image: nvcr.io/nvidia/isaac-sim:2023.1.1
    environment:
      - ACCEPT_EULA=Y
      - PRIVACY_CONSENT=Y
      - OMNI_KIT_ALLOW_ROOT=1
    volumes:
      - ./store/cache/kit:/isaac-sim/kit/cache:rw
      - ./store/cache/ov:/root/.cache/ov:rw
      - ./store/cache/pip:/root/.cache/pip:rw
      - ./store/cache/glcache:/root/.cache/nvidia/GLCache:rw
      - ./store/cache/computecache:/root/.nv/ComputeCache:rw
      - ./store/logs:/root/.nvidia-omniverse/logs:rw
      - ./store/data:/root/.local/share/ov/data:rw
      - ./store/documents:/root/Documents:rw
      - ./tm-apps/omni.isaac.sim.headless.webrtc.kit:/isaac-sim/apps/omni.isaac.sim.headless.webrtc.kit
      - ./tm-exts:/isaac-sim/exts
      - ./python-apps:/root/python-apps

    entrypoint: ["bash", "./python.sh", "/root/python-apps/my_application.py"]
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    network_mode: host
2

Hi @darien.wei - Have you referred to this documentation?

https://docs.omniverse.nvidia.com/isaacsim/latest/installation/install_faq.html#isaac-sim-headless-app-via-webrtc-browser-client
https://docs.omniverse.nvidia.com/isaacsim/latest/installation/manual_livestream_clients.html#webrtc-browser-client

3

Hi @rthanker,

I’ve reviewed the documents again. Today, I attempted another testing by enabling the livestream.py[Ref.1] with WebRTC within Docker headless. It appears that the service is running, but the screen is intermittently blank when viewed in the browser[Ref.2].

I’ve enabled verbose logging and noticed some errors towards the end[Ref.3]. I’ve attached the full log[Ref.4] for your reference.

Could you provide a solution for the issue?
Thank you

[Ref.1]
livestream.py

from omni.isaac.kit import SimulationApp
from datetime import datetime
import time

CONFIG = {
    "width": 1280,
    "height": 720,
    "window_width": 1920,
    "window_height": 1080,
    "headless": True,
    "renderer": "RayTracedLighting",
    "display_options": 3286,  # Set display options to show default grid
}

kit = SimulationApp(launch_config=CONFIG)
from omni.isaac.core.utils.extensions import enable_extension

# Default Livestream settings
kit.set_setting("/app/window/drawMouse", True)
kit.set_setting("/app/livestream/proto", "ws")
kit.set_setting("/app/livestream/websocket/framerate_limit", 120)
kit.set_setting("/ngx/enabled", False)


# enable_extension("omni.kit.livestream.native")
# enable_extension("omni.services.streamclient.websocket")
# Enable WebRTC Livestream extension Only
enable_extension("omni.services.streamclient.webrtc")

while kit._app.is_running() and not kit.is_exiting():
    kit.update()
kit.close()

[Ref.2]

webrtc_screen_blank
webrtc_screen_blank1466×947 21.4 KB

[Ref.3]

....
isaac-sim  | 2024-03-27 05:57:22 [10,826ms] [Error] [carb.livestream.plugin] Could not initialize streaming components
isaac-sim  | 2024-03-27 05:57:22 [10,836ms] [Error] [carb.livestream.plugin] Couldn't initialize the capture device.

[Ref.4]
livestream-webrtc.log (39.9 KB)