I want to attach a camera to custom objects and there are some problems regarding the orientation of the camera. The parent is a XForm prim that is there for proper orientation of the camera, but whatever I try, the orientation does not really change as I would expect. The XForm is also not changing the orientation as I am trying to do using
set_local_pose. So, now the problem is that I cannot fix the orientation of the camera such that I can capture synthetic data.
It does not matter, if I try to change the orientation of the XForm or directly the camera, the result is somehow weird and not logical to me.
How is it supposed to work?
This is a great question! There are three different frames that the Isaac Sim/Omniverse supports wrt to the camera:
USD frame: Defined as +Y up, -Z forward.
- this is the frame data being displayed in the
Property tab of the scene. This is the convention used by the computer graphics community.
world frame: Defined as +Z up, +X forward.
omni.isaac.sensor Camera class sets/gets the pose of the cameras in the
world frame via
ROS camera frame: Defined as +Y up, +Z forward
- This is the frame that ROS camera-related messages are published in via OmniGraph.
If you take the pose of the camera from the
Property tab of the scene and put it into
omni.isaac.sensor Camera object via get/set, then it would result in weird behaviors because the pose does not match.
To work around this, you can do one of two things:
If you know the pose of the camera that you like in the
world frame, directly use
omni.isaac.sensor.Camera . Example:
from omni.isaac.sensor import Camera
from omni.isaac.core.utils.rotations import euler_angles_to_quat
pos_world = np.array([1, 1, 1])
quat_world =euler_angles_to_quat([0, -90, 0], degrees=True)
camera = Camera(position=pos_world,
camera.get_world_pose() # should give the same as pos_world and quat_world above
camera.set_world_pose(pos_world, quat_world) # same as initialization
If you would like to set the pose of the camera using the
USD pose, directly use the XForm and set pose using the data from
We are actively looking to improve the camera interface for Isaac Sim and we have improvements coming in the next Isaac Sim release. The features that will help you with this issue are as follows:
omni.isaac.sensor camera will allow you to get/set of the camera pose in all of the different frames
- We will introduce a new GUI extension, Camera Inspector, which will display the camera pose in all of these frames, so that you can take them from the scene and put it into code with ease. This extension will also include some other functionalities, e.g., viewports to allow you to adjust the camera on the fly.
- We will introduce a new set of camera tutorials to help you interact with the camera better via GUI and API.