Connect Isaac Sim on a docker container through Omniverse streaming client remotely

Omniverse Isaac Sim
, , , , , , , ,
1

Hello, I want to connect to Isaac sim on a docker container in a HPC running Ubuntu Server 20.04 through my desktop running Ubuntu 20.04 with the Omniverse streaming client.
I run the Isaac Sim docker container with the following command on the HPC (server-side):

docker run --name isaac-sim --entrypoint bash -it --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
    -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache/Kit:rw \
    -v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v ~/docker/isaac-sim/documents:/root/Documents:rw \
    nvcr.io/nvidia/isaac-sim:2022.2.1

And then start Isaac Sim with native livestream mode:
./runheadless.native.sh

After that I got the Isaac Sim Headless Native App is loaded. message on the server-side, I run the kit-remote.sh through terminal on my desktop (client) with the following command:

./kit-remote.sh apps/omni.create.kit \
   --enable omni.services.streaming.manager \
   --enable omni.kit.livestream.native \
   --no-window \
   --allow-root

After I enter server IP and attempt to connect, I get a black window that closes automatically after a few seconds.

The kit-remote.log file contents are as follows:

~/ov/pkg/kit_remote-103.1.1 ~/ov/pkg/kit_remote-103.1.1
[2023-04-25 18:52:48,980]=13:52:48=   DEBUG [Geronimo: GeronimoRemoteConfig] {6E231140} - Cannot open config file RemoteConfig.json.
[2023-04-25 18:52:48,980]=13:52:48=    INFO [Tracing] {6E231140} - Starting tracing for gfn-gamestream.
[2023-04-25 18:52:48,981]=13:52:48=    INFO [OV-Streaming-Client: main] {6E231140} - Command line: "./kit-remote --width 1920 --height 1080 --server <ip_address>"
[2023-04-25 18:52:48,981]=13:52:48=    INFO [OV-Streaming-Client: main] {6E231140} - Setting log level to 'E'
I-!$> [2023-04-25 18:52:49,272]=13:52:49=    INFO [BifrostClient: Scheduler] {68665700} - Successfully set TLS with value 0x55ef84828060
I-!$> [2023-04-25 18:52:49,272]=13:52:49=    INFO [BifrostClient: Scheduler] {68E66700} - Successfully set TLS with value 0x55ef84828060
I-!$> [2023-04-25 18:52:49,272]=13:52:49=    INFO [BifrostClient: Scheduler] {69667700} - Successfully set TLS with value 0x55ef84828060
I-!$> [2023-04-25 18:52:49,272]=13:52:49=    INFO [BifrostClient: Scheduler] {69E68700} - Successfully set TLS with value 0x55ef84828060
I-!$> [2023-04-25 18:52:49,273]=13:52:49=   DEBUG [BifrostClient: Interface] {6E231140} - nvbCreateClient( 0x55ef84bf51a0 )
[2023-04-25 18:52:49,435]=13:52:49=   ERROR [Geronimo::Analytics] {6E231140} - Failed to load dll with error: /home/fuzzy/ov/pkg/kit_remote-103.1.1//NvTelemetryAPI.so: cannot open shared object file: No such file or directory
[2023-04-25 18:52:49,706]=13:52:49=   ERROR [BifrostClient: Streamer] {6E231140} -  updateVideoSettingsForNVbProfile: profile 8 is not handled
[2023-04-25 18:52:49,836]=13:52:49=   ERROR [NVST:ClientSession] {69E68700} -  Number of channels(2) is not valid for surround configuration
[2023-04-25 18:52:49,838]=13:52:49=   ERROR [NVST:NvNetworkLinux] {69E68700} -  Address family 1 for Network address: 0x7fb469e64880 not supported
[2023-04-25 18:52:50,022]=13:52:50=   ERROR [NVST:NattHolePunch] {4CFF9700} -  NattHolePunch: Failed to receive hole punching packet
[2023-04-25 18:52:50,022]=13:52:50=   ERROR [NVST:ClientSession] {69E68700} -  Control channel hole punching failed.
[2023-04-25 18:52:50,039]=13:52:50=   ERROR [NVST:ServerControl] {3F7FE700} -  Unknown server notification
[2023-04-25 18:52:50,066]=13:52:50=   ERROR [GIOInterface] {3D7FA700} - GERONIMO_ERROR 0xC0020003 GERONIMO_IOINTERFACE_INVALID_AUDIO_FUNC: Audio Renderer is NULL.
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:ClientLibraryWrapper] {1BFFF700} -  Stream config not updated after rtsp handshake
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:ClientLibraryWrapper] {1BFFF700} -  Cannot find streamEventRaised for stream.media type 1
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:ClientLibraryWrapper] {1BFFF700} -  Cannot find streamEventRaised for stream.media type 2
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:DataChannelBuffers] {3F7FE700} -  Tried to read enet data when none is available
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:ServerControlEnet] {3F7FE700} -  Failed to read data from Enet socket with result 8000003C
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:ServerControlEnet] {3F7FE700} -  Read failed due to other enet read errors
[2023-04-25 18:52:50,308]=13:52:50=   ERROR [NVST:ClientSession] {3F7FE700} -  Unable to read server command due to NVST_NETERR_CLIENT_DISCONNECT_UNINTENDED (2147745797)
[2023-04-25 18:52:50,314]=13:52:50=   ERROR [RemoteGridApp] {68E66700} - STREAMING: FAILED: NVB_SN_STOPPED_UNINTENTIONALLY (800e840c)
[2023-04-25 18:53:03,986]=13:53:03=   ERROR [NVST:ClientLibraryWrapper] {69E68700} -  Cannot find streamEventRaised for stream.media type 1
[2023-04-25 18:53:03,987]=13:53:03=   ERROR [NVST:ClientLibraryWrapper] {69E68700} -  Cannot find streamEventRaised for stream.media type 2
[2023-04-25 18:53:03,987]=13:53:03=   ERROR [NVST:EnetSocket] {69E68700} -  Failed to send message, run loop is continuing, socket is disconnecting, peer is (nil)
[2023-04-25 18:53:03,987]=13:53:03=   ERROR [NVST:ClientLibraryWrapper] {69E68700} -  Failed to send client session start timestamp stream 0 nvtime::sessionStartTime[us] on client: 8298075166063, error: -2146762752
I-!#>
 [2023-04-25 18:53:03,990]=13:53:03=    INFO [BifrostClient: Interface] {6E231140} - nvbDestroyClient(). Successfully destroyed NVbClient
[2023-04-25 18:53:03,995]=13:53:03=    INFO [Tracing] {6E231140} - Sending final distributed tracing report.
[2023-04-25 18:53:03,995]=13:53:03=    INFO [Tracing] {6E231140} - Distributed tracing ended.

In the following, I attached the logs when I execute ./runheadless.native.sh on HPC.
runheadless-native.log (37.1 KB)

Thanks in advance.

HPC Specs (Server-Side):
Ubuntu Server 20.04 LTS
Tesla A100 40G
NVIDIA-Driver Version: 525.60.13
Cuda Version: 12.0
Isaac-sim: 2022.2.1

Client-Side:
Omniverse Streaming Client Version: 103.1.1
GeForce GTX 1080 8G

More info: I connect to the HPC through ssh e.g. : ssh user@<ip_address> -p<port_number>

1 Like
3

I have the same problem

image
image719×431 22.3 KB

1 Like
4

Hi, the Headless Native mode with Omniverse Streaming Client does not work with A100 because it does not have NVENC. If you both machines are on the same network, the websocket with H264 software encoding is recommended instead.
See Livestream Clients — Omniverse Robotics documentation.

2 Likes
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.