Guidance in implementing a custom ROS2 Extension

Hi, I am relatively new to Isaac Sim in general, so I would like to ask for guidance for something you guys might find relatively trivial to implement.

What I want:
A component like ROS_JointState, but instead of publishing joint states, I want it to expose a ROS2 service that I can call from my system. It should expose 2 ros2 services

  • one to add an rigid body from a known asset path at a given pose to a specified environment path
  • one to remove an object from a specified environment path

So that I can dynamically add / remove different objects for my robot to pick in the simulation

What have I discovered so far:
After looking around into the source files and the documentations

  • Given that the ros2 <-> issac side is taken care of, I know how to add object from an asset path to an environment path with wanted translation and rotation (similar to what’s inside the omni.isaac.samples extension)
  • How to expose my extension and add it to the menu bar + how to remove it when the extension is shutting down

What I need help with
I need guidance with the following topics

  • How do I establish a communication with ROS2? I tried looking for source code online on how the ROS_JointStates or its TF tree counterpart is written, but so far I have not found any source code and it seems like the access I only get is the already-compiled .so file for the extension
  • How is the ros2 architecture structured on the Isaac side? should I just go and create a new node for myself? or should I get an existing singleton node from somewhere and define my services on it (as I found a nodeName config in the ROS2 bridge .toml)
  • I tried importing rclpy from the ros2_bridge extension (which works), but when I tried calling rclpy.init() inside Isaac, I got an error where it cannot find librclpy_common.so. Which is very strange to me as I can verify the following
    1. The LD_LIBRARY_PATH is correctly pointing to the directory of the librclpy_common.so
    1. when I tried to load the shared object on my own using the provided python.sh and cdll.LoadLibrary, it manages to load without any issues, but I am given a “No such file or directory” error when I tried the same thing inside Isaac Sim’s console (with ROS2 Bridge enabled)
  • How do one create a component that I want? I would imagine it being a customly-defined component that runs a custom script once the simulation starts. But I cannot find any examples that implement something similar to a ROS_JointState block in my files. Any pointers to examples would be appreciated

Please let me know on what I did incorrectly, or where should I look into next, thank you.

1 Like

ROS2 is a bit different in how it resolves library paths at runtime, compared to ROS.
I need to see if there is a better way to do this but for now if you run:

declare -x LD_LIBRARY_PATH="/path/to/exts/omni.isaac.ros2_bridge/bin/"
./isaac-sim.sh

you should be able to disable the ros extension, enable the ros2 extension and then in the script editor call:

import rclpy
rclpy.init()

Going to try to answer the rest of the questions here, please let me know if I missed something :)

  • The ROS2 nodes that are precompiled are meant to provided access to data (like camera/lidar) that would otherwise be slow to publish from python.
    It should be possible to use rclpy + python bindings to existing extensions to write your own nodes in an extension to control the full pipeline.

  • There should be python bindings to pretty much everything in isaac sim. we can provide links to docs or snippets if you have questions. If there are any bindings missing we can look into adding them for future releases.

  • I will add a task to document a rclpy sample for the next release, and try to solve the .so not loading error in a more user friendly way.