Running python scripts headlessly with Isaac Sim 4.5

Omniverse Isaac Sim
,
1

Isaac Sim Version

4.5.0

Operating System

Ubuntu 20.04

GPU Information

  • Model: Running in AWS g5.2xlarge instance
  • Driver Version: 535.183.06 CUDA Version: 12.2

Topic Description

When running a python script in a headless mode, streaming client connection cannot be established.

Detailed Description

I am using a container installation of Isaac Sim 4.5, set up according to instructions Container installation.
I want to reproduce the livestream example. After launching the suggested command:

./python.sh standalone_examples/api/isaacsim.simulation_app/livestream.py

I am expecting to be able to connect to Omniverse Streaming client and see an empty scene. However, the connection to Omniverse Streaming Client fails.

image
image865Ă—261 15.3 KB

I would appreciate your help in resolving the issue.
I want to emphasize that the same example for Isaac Sim 4.2 works on my machine.
My goal is simple: be able to run python scripts and visualize the output via Streaming Client.

1 Like
2

hi @d.humeniuk , is the same workflow working as expected with Isaac Sim 4.2?

3

@rchadha hi! Yes, On IsaacSim 4.2 when launching the livestream example I can connect to the UI via Streaming Client and I see an empty scene.

4

For Isaac Sim 4.5.0, please ensure you follow the container installation documentation:

and pay attention to this note:

  • Isaac Sim WebRTC Streaming Client is recommended to be used within the same network as an Isaac Sim headless instance.
  • See Omniverse Streaming Client [DEPRECATED] for an alternative streaming client if you have issue livestreaming remotely in the Cloud.
5

Hi @d.humeniuk , I had the same problem here.

The solution I found to run interactive python commands while isaac-sim 4.5.0 streams from a container installation on a AWS EC2 instance is to use isaacsim’s jupyter notebook extension.

Here’s a brief step-by-step guide (replace <TRUSTED_IP> and <EC2_PUBLIC_IP> with your actual trusted IP address and the public IP address of your EC2 instance, respectively.):

  1. Open port 8228 on your EC2 VM’s firewall:
sudo ufw allow from <TRUSTED_IP> to any port 8228 proto tcp
  1. Launch the container using host networking, as recommended in Isaac Sim’s AWS AWS deployment documentation:
docker run --name isaac-sim --entrypoint bash -it --runtime=nvidia --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v ~/docker/isaac-sim/documents:/root/Documents:rw \
    nvcr.io/nvidia/isaac-sim:4.5.0
  1. Start the streaming service inside the container with Jupyter notebook enabled:
./runoldstreaming.sh --enable isaacsim.code_editor.jupyter

  1. Connect to IsaacSim using the Omniverse Streaming Client [DEPRECATED]

  2. Access the Jupyter interface from your web browser by navigating to:

http://<EC2_PUBLIC_IP>:8228/lab
  1. Create a notebook using the omniverse kernel. The python commands passed in the notebook reflect interactively in isaacsim’s streaming session.

Best,

6

Hi @VickNV ! Thank you for your reply. My Isaac Sim headless instance is located on AWS. My WebRTC Client is installed on my host machine. I am not sure how can I make sure they are located in the same network. Can you give an example of such setup?

7

Thank you @t.reid ! I think it can work as a temporary solution!

8

In your case, it’s not in the same network so you have to use the Omniverse Streaming Client.

1 Like
9

Hi all. Just a heads up, we have a fix for streaming over the Internet using the WebRTC Streaming Client in our upcoming Isaac Sim release. Hence, support for Omniverse Streaming Client will be removed.

Meantime, we have also updated the AWS deployment guide as a Virtual Desktop experience.

2 Likes
10

@VickNV Thank you for the answer! Yes, ./isaac-sim.old_streaming.sh works but only to launch the Isaac Sim interface and not the standalone python apps with livestreaming.

11

@Sheikh_Dawood thank you for the answers! This sounds good! Thank you for the additional instructions!