Tutorial Issac application opencv_edge_detection fails to connect to Isaac Sim

I’m not able to follow the “Building Isaac Applications” tutorial at step “Start Unity3D”: Building Isaac Applications — ISAAC 2021.1 documentation . Below is what I tried.

The tutorial starts the simulator by running a builds/sample.x86_64 executable:

bob@desktop:~/isaac_sim_unity3d$ cd builds
bob@desktop:~/isaac_sim_unity3d/builds$ ./sample.x86_64 --scene small_warehouse

I can find several build directories, but can’t find any sample.x86_64 executable:

alan@alan-MS-7D25:~/.local/share/ov/pkg$ find . -name "build"
./isaac_sim-2021.1.0/kit/python/lib/python3.7/site-packages/pip/_internal/operations/build
./isaac_sim-2022.1.0/kit/python/lib/python3.7/site-packages/pip/_internal/operations/build
./isaac_sim-2022.1.0/kit/data/Kit/Isaac-Sim/2022.1/pip3-envs/default/pip/_internal/operations/build
alan@alan-MS-7D25:~/.local/share/ov/pkg$ find . -name "sample.x86_64"
alan@alan-MS-7D25:~/.local/share/ov/pkg$ 

I think this tutorial uses an old version of Isaac Sim per the copyright tag © Copyright 2018-2020, NVIDIA Corporation. But Omniverse launcher only has 2021 and 2022 versions of Isaac Sim, which I can use.

So I started the simulator through Omniverse launcher and Isaac Sim UI and manually added references of the USDs and clicked play (I can’t find asset small_warehouse so I used warehouse_multiple_shelves instead):

robot usd: omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Robots/Carter/carter_v2.usd
enrionment usd: omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Environments/Simple_Warehouse/warehouse_multiple_shelves.usd

Then I started the Isaac SDK docker and the Isaac application using

alan@alan-MS-7D25:~/isaac$ docker run --network=host --device=/dev/video0 --mount source=isaac-sdk-build-cache,target=/root -v `pwd`:`pwd` -w `pwd`/sdk --runtime=nvidia -it isaacbuild:latest /bin/bash
root@alan-MS-7D25:/home/alan/isaac/sdk# bazel run //apps/tutorials/opencv_edge_detection:opencv_unity3d

and the command line shows the app failed to connect to the simulator at localhost:55000:

2022-06-16 22:13:11.530 INFO  packages/engine_tcp_udp/components/TcpPublisher.cpp@71: Starting TCP publisher on port 55001
2022-06-16 22:13:11.530 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.interface/input' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.530 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.interface/output' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.530 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.interface/time_sync' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.530 INFO  packages/engine_tcp_udp/components/TcpSubscriber.cpp@133: TCP receiver trying to connect to remote (localhost:55000)...
2022-06-16 22:13:11.530 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.noisy_flatscan_1.flatscan_noiser/Random' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.530 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.noisy_flatscan_2.flatscan_noiser/Random' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.531 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.noisy_flatscan_3.flatscan_noiser/Random' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.531 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'simulation.noisy_flatscan_4.flatscan_noiser/Random' was not added to scheduler because no tick method is specified.
2022-06-16 22:13:11.531 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.
2022-06-16 22:13:11.536 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2022-06-16 22:13:11.536 WARN  packages/navsim/components/ScenarioManager.cpp@32: No scene specified. The default one loaded by navsim will be used.
2022-06-16 22:13:11.536 DEBUG external/com_nvidia_isaac_engine/engine/alice/backend/event_manager.cpp@40: Stopping node 'simulation.scenario_manager' because it reached status 'SUCCESS'
2022-06-16 22:13:12.031 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.
2022-06-16 22:13:12.531 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.
2022-06-16 22:13:13.031 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.
2022-06-16 22:13:13.531 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.
2022-06-16 22:13:14.031 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.
2022-06-16 22:13:14.531 WARN  packages/engine_tcp_udp/components/TcpSubscriber.cpp@158: Failed to connect to remote. Will try again in 0.500000 seconds.

Does this mean the simulator is not publishing on localhost:55000 ? Can anyone help me figure out how to connect this Isaac app to Isaac sim?

Isaac SDK 2021.1 requires Isaac Sim 2021.1 and is not compatible with Isaac Sim 2022.1 unfortunately.

I did try Isaac Sim 2021.1.0 and Isaac Sim 2021.1.1. And I got the same error.

Could you try loading the following USD omniverse:/Isaac/Samples/Isaac_SDK/Scenario/carter_warehouse_with_forklifts.usd? We suspect that the USD you’re using for the multi-shelf warehouse is not compatible with Isaac Sim 2021.1 (previous version). If you could attach a journal of your commands and any logs you’ve obtained, it would help us reproduce the issue.