Issues with extensions moving from isaac 4.2-4.5

Isaac Sim Version

4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: NVIDIA RTX A2000 12GB
  • Driver Version: 550.120 CUDA Version: 12.4

Issues with Moving from 4.2-4.5

Im trying to get a custom extension working on 4.5 which was running fine on 4.2. However now the extension manager is complaining about dependencies? I have at least a few interfaces Im struggling with here

  • ros bridge
  • isaac sensor
  • conveyor utility
    All seem to be not working. Any ideas or approaches?

Here’s one example:

This might be due to renaming, did you check this Renaming Extensions in Isaac Sim — Isaac Sim Documentation

Holy smokes! Ok what an adventure but I have rooted out much of the issues. I tried to jot down a few notes of issues I had to chase down in the hopes that others may have an easier time:

For context: this is a custom extension generated by the 4.2 template that Im attempting to get working in 4.5. The extension is an application involving a gantry machine and a bunch of conveyors (imported URDF models as articulations) and rangefinder sensors. There is also a bunch of interaction with ROS2 nodes subscribing sensor data and publishing robot commands. Whole thing is in a docker container, 2 services, 1 for isaac, 1 for ros2.

Issues and how I solved:

  1. Renamed extensions in my scenario.py script. I had a bunch of methods that interacted with action graph nodes. I had to manually rename them to the new definitions, outlined here:

https://docs.isaacsim.omniverse.nvidia.com/latest/overview/extensions_renaming.html

This also specifically helped renaming the sensor interfaces:

https://docs.isaacsim.omniverse.nvidia.com/latest/py/source/extensions/isaacsim.sensors.physx/docs/ogn/OgnIsaacReadLightBeamSensor.html

  1. Changed my docker setup to include new flag for ros bridge package and ld lib path. Docker compose for that now looks like this:
isaac_sim:
    image: nvcr.io/nvidia/isaac-sim:4.5.0
    entrypoint: bash
    stdin_open: true
    tty: true
    environment:
      # For omniverse EULA
      ACCEPT_EULA: Y
      PRIVACY_CONSENT: Y
      #For activating ros humble bridge
      ROS_DISTRO: humble
      RMW_IMPLEMENTATION: rmw_cyclonedds_cpp #rmw_fastrtps_cpp  to Change to FastDDS instead of CycloneDDS
      LD_LIBRARY_PATH: /isaac-sim/exts/isaacsim.ros2.bridge/humble/lib
      OMNI_SERVER: omniverse://localhost/NVIDIA/Assets/Isaac/4.5"
      # Necessary for display passthrough
      DISPLAY: $DISPLAY
    network_mode: host
    volumes:
      - $VOLUMES_DIRECTORY/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw
      - $VOLUMES_DIRECTORY/isaac-sim/cache/ov:/root/.cache/ov:rw
      - $VOLUMES_DIRECTORY/isaac-sim/cache/pip:/root/.cache/pip:rw
      - $VOLUMES_DIRECTORY/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw
      - $VOLUMES_DIRECTORY/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw
      - $VOLUMES_DIRECTORY/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw
      - $VOLUMES_DIRECTORY/isaac-sim/data:/root/.local/share/ov/data:rw
      - $VOLUMES_DIRECTORY/isaac-sim/documents:/root/Documents:rw      
      # Necessary for display passthrough
      - "/tmp/.X11-unix:/tmp/.X11-unix:rw"
      - "./launch-profiles/${LAUNCH_PROFILE:- 'set in docker/_shared'}:/robot/launch-profile/"
    command: ./runapp.sh -v
  1. On startup, i enabled the deprecated issac ext helper. I then opened and saved each loaded USD file so the helper could comb thru the file and updated deprecated nodes.

  2. Activated all necessary new extensions for my app. For me this involved finding the new appropriate conveyor, isaacc ros2 bridge and sensor extensions and enabling them

After all that, my app now functions correctly. yay.

ONE ISSUE REMAINS:

However, there is ONE FINAL BOSS ISSUE:

[Error] Failed to create MDL shade node for prim '/World/full_rl_picker_base/Looks/Steel_Galvanized/Shader'. Empty identifier: '' 	and/or subIdentifier: ''

And:

[Error] [asyncio] Task exception was never retrieved
future: <Task finished name='Task-6382' coro=<UsdShadeBaseWidget._load_mdl_modules.<locals>.load_module() done, defined at /isaac-sim/extscache/omni.kit.property.material-1.10.17+d02c707b/omni/kit/property/material/scripts/widgets/usdshade/base_widget.py:336> exception=ErrorException(Error in 'pxrInternal_v0_22__pxrReserved__::Neuray::LoadModule' at line 242 in file ../../source/plugins/omni.usdMdl/neuray.cpp : ''rtx::neuraylib::MdlModuleId' for '' is Invalid')>

And:

[Warning] Error populating 'omniverse://localhost': Error listing directory 'omniverse://localhost': Result.ERROR_CONNECTION
node_name=omni.kit.widget.filebrowser.thumbnails
[Warning] Failed to stat url omniverse://localhost

I cant get my material shaders to load. And the nucleus server that I have running locally seems to be unreachable from isaac 4.5 (but still connects fine to issac4.2 in the docker container). I have tried:

  • adding the local nucleus server to my docker env: “OMNI_SERVER: omniverse://localhost/NVIDIA/Assets/Isaac/4.5”
  • trying to find materials in the pre-loaded assets

Anyone have any ideas here?

Ok, scratched a little harder and solved it!

Turns out you now need to add default username and password env var for issac to log into nucleus:

      OMNI_USER: admin
      OMNI_PASS: admin

Added that to the docker container environment and presto, everything now works.

We are up and running on 4.5!

2 Likes