Orientation of Camera

Hello!

I want to attach a camera to custom objects and there are some problems regarding the orientation of the camera. The parent is a XForm prim that is there for proper orientation of the camera, but whatever I try, the orientation does not really change as I would expect. The XForm is also not changing the orientation as I am trying to do using set_local_pose. So, now the problem is that I cannot fix the orientation of the camera such that I can capture synthetic data.

It does not matter, if I try to change the orientation of the XForm or directly the camera, the result is somehow weird and not logical to me.

How is it supposed to work?

Best Regards,
Christof

Hey Christof,

This is a great question! There are three different frames that the Isaac Sim/Omniverse supports wrt to the camera:

  1. USD frame: Defined as +Y up, -Z forward.
    • this is the frame data being displayed in the Property tab of the scene. This is the convention used by the computer graphics community.
  2. world frame: Defined as +Z up, +X forward.
    • The omni.isaac.sensor Camera class sets/gets the pose of the cameras in the world frame via set_world_pose() and get_world_pose()
  3. ROS camera frame: Defined as +Y up, +Z forward
    • This is the frame that ROS camera-related messages are published in via OmniGraph.

If you take the pose of the camera from the Property tab of the scene and put it into omni.isaac.sensor Camera object via get/set, then it would result in weird behaviors because the pose does not match.

To work around this, you can do one of two things:

  1. If you know the pose of the camera that you like in the world frame, directly use omni.isaac.sensor.Camera . Example:

    from omni.isaac.sensor import Camera 
    from omni.isaac.core.utils.rotations import euler_angles_to_quat
    
    pos_world = np.array([1, 1, 1])
    quat_world =euler_angles_to_quat([0, -90, 0], degrees=True)
    camera = Camera(position=pos_world, 
    								orientation=quat_world)
    
    camera.get_world_pose() # should give the same as pos_world and quat_world above 
    camera.set_world_pose(pos_world, quat_world) # same as initialization
    
  2. If you would like to set the pose of the camera using the USD pose, directly use the XForm and set pose using the data from Property panel.

We are actively looking to improve the camera interface for Isaac Sim and we have improvements coming in the next Isaac Sim release. The features that will help you with this issue are as follows:

  1. The omni.isaac.sensor camera will allow you to get/set of the camera pose in all of the different frames
  2. We will introduce a new GUI extension, Camera Inspector, which will display the camera pose in all of these frames, so that you can take them from the scene and put it into code with ease. This extension will also include some other functionalities, e.g., viewports to allow you to adjust the camera on the fly.
  3. We will introduce a new set of camera tutorials to help you interact with the camera better via GUI and API.
1 Like