Isaac Sim Camera Orientation Issue With Ros

Omniverse Isaac Sim
,
1

Important: Isaac Sim support

Note: For Isaac Sim support, the community is gradually transitioning from this forum to the Isaac Sim GitHub repository so that questions and issues can be tracked, searched, and resolved more efficiently in one place. Whenever possible, please create a GitHub Discussion or Issue there instead of starting a new forum topic.

Note: For any Isaac Lab topics, please submit your topic to its GitHub repo ( GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim · GitHub ) following the instructions provided on Isaac Lab’s Contributing Guidelines ( Contribution Guidelines — Isaac Lab Documentation ).

Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.

6.0.0
5.1.0
5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 24.04
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: A5000
  • Driver Version: 580.142

Topic Description

I have camera created in isaacsim which is attatched to a robot. The camera have some translation & orientation with the robot. The orientation is (90,-90,0). I am publishing the camera data with action graphs created using Tools->Robotics->Ros2Omnigraphs.

When i check the camera feed in rviz2 by adding the topic camera/rgb/ the image is coming properly with correct orientation. But when i use the same camera topics with some algorithms like rtabmap for visual slam the orientation is not correct. I also have tf node attached to camera action graphs for publishing the tf between robot body and camera frame.

Screenshots or Videos

Camera Prim In Usd

camera_prim
camera_prim1920Ă—1080 299 KB

Camera Action Graph

action_graph_cam
action_graph_cam1920Ă—1080 263 KB

The tf getting published is also different from usd orientation

tf_cam
tf_cam720Ă—558 57.6 KB

In rviz and rtabmapviz i am getting the map as follows:

rviz_orient
rviz_orient2560Ă—1440 226 KB

rtab_orient
rtab_orient1920Ă—1080 144 KB

Additional Information

What I’ve Tried

  • The raw tf publisher node expects quaternions which i am reading from the camera prim attribute and also tried giving manually.

2

Hi @umer.k, thanks for the detailed report and screenshots!

The root cause here is a coordinate frame convention mismatch between Isaac Sim and ROS2. Isaac Sim uses an OpenGL/USD camera convention where the camera looks down the -Z axis with Y up, while ROS2 expects the standard optical frame convention where the camera looks down the +Z axis with Y down (i.e., camera_optical_frame). The ROS2 TF publisher in the action graph publishes the raw USD transform, which does not automatically apply this convention correction — this is why the image looks fine in RViz (the image data itself is correct) but algorithms like RTAB-Map that rely on the TF for 3D geometry get confused.

To fix this, you have a couple of options:

  1. Add a static TF broadcaster between your camera frame and a camera_optical_frame that applies the standard ROS optical frame rotation (a 90° rotation around X, then -90° around Z, i.e., the transform that converts from ROS camera frame to optical frame). You can do this in your launch file:

    Node(
    package='tf2_ros',
    executable='static_transform_publisher',
    arguments=['0', '0', '0', '-1.5708', '0', '-1.5708',
               'camera_frame', 'camera_optical_frame']
)

    Then configure RTAB-Map to subscribe to camera_optical_frame instead of camera_frame.

  2. Adjust the camera orientation in USD so that the published TF already matches ROS conventions. The typical corrective rotation to apply on top of your existing orientation is to rotate the camera prim by an additional 90° around the X-axis to align Z-forward.

  3. Use the isaac_ros_visual_slam or isaac_ros_rtabmap packages from the Isaac ROS suite if available for your setup, as they are designed to handle this convention difference natively.

Option 1 (static TF) is generally the most robust and least invasive approach. Please let us know if this resolves the issue!

4

Thanks for the reply @VickNV ,

This is my rtab launch command

ros2 launch rtabmap_launch rtabmap_launch.py depth_topic:=/depth rgb_topic:=/rgb camera_info:=/camera_info frame_id:=body odom_topic:=/odom approx_sync:=true rviz:=true use_sim_time:=true
  • I tried publishing static tf between camera and camera_optical with tf (0,0,0 ,-1.507,0,-1.5705).
  • In the above launch command i tried changing frame_id to camera_optical still i am having that orientation issue.
  • Am i missing something
5

Hi, I think there’s a common mix-up between two different frame_ids. Both your changes were on the wrong one, which is why nothing changed.

The two frame_ids in play:

  1. frame_id in rtabmap_launch.py → this is the robot base / fixed frame (the frame RTAB-Map will treat as “the robot”). Your original value body is most likely correct — keep it as body. Don’t set it to camera_optical. RTAB-Map uses this frame to compute odometry → map alignment, not to interpret camera data.
  2. frame_id in the image / CameraInfo message header → this is what tells RTAB-Map which TF frame the camera is in. This is the one that must equal camera_optical_frame (or whatever you named your optical frame). It is set inside the OmniGraph, not via the launch file — specifically the frameId field on the ROS2 Camera Helper (or the underlying ROS2 Publish Image / ROS2 Publish Camera Info) nodes.

Could you check the current value? Run:

 ros2 topic echo /rgb --field header --once
 ros2 topic echo /camera_info --field header --once
 ros2 topic echo /depth --field header --once

If all three say frame_id: body (or anything other than camera_optical_frame), that’s your problem. Open each ROS2 Camera Helper / publisher node in the OmniGraph and set its frameId input to camera_optical_frame.

Then verify the TF chain is intact:

   ros2 run tf2_ros tf2_echo body camera_optical_frame

You should get a non-zero translation + the optical rotation. If this errors with “frame does not exist,” then either the static TF isn’t running or your camera_link isn’t in the chain.

I’d also recommend removing the (90, -90, 0) orientation on the camera prim and putting the camera back to its natural USD orientation. The static TF between camera_link and camera_optical_frame is doing the convention conversion — having the prim pre-rotated on top of it makes the geometry hard to reason about and can cause exactly the kind of partial-correct, partial-wrong behavior you’re seeing (image looks OK in RViz, but 3D reconstruction is rotated).

6

Thanks for the reply @PeterNV

i changed the frame id in ros2 camera helper action graph from sim_camera to sim_camera_optical along with publishing static tf between sim_camera and sim_camera optical.

For now i have kept the orientation on camera prim and i able to generate proper 3d map using rtab..

Once again thanks @PeterNV and @VickNV for the guidance.