I want to attach a camera to custom objects and there are some problems regarding the orientation of the camera. The parent is a XForm prim that is there for proper orientation of the camera, but whatever I try, the orientation does not really change as I would expect. The XForm is also not changing the orientation as I am trying to do using set_local_pose. So, now the problem is that I cannot fix the orientation of the camera such that I can capture synthetic data.
It does not matter, if I try to change the orientation of the XForm or directly the camera, the result is somehow weird and not logical to me.
This is a great question! There are three different frames that the Isaac Sim/Omniverse supports wrt to the camera:
USD frame: Defined as +Y up, -Z forward.
this is the frame data being displayed in the Property tab of the scene. This is the convention used by the computer graphics community.
world frame: Defined as +Z up, +X forward.
The omni.isaac.sensor Camera class sets/gets the pose of the cameras in the world frame via set_world_pose() and get_world_pose()
ROS camera frame: Defined as +Y up, +Z forward
This is the frame that ROS camera-related messages are published in via OmniGraph.
If you take the pose of the camera from the Property tab of the scene and put it into omni.isaac.sensor Camera object via get/set, then it would result in weird behaviors because the pose does not match.
To work around this, you can do one of two things:
If you know the pose of the camera that you like in the world frame, directly use omni.isaac.sensor.Camera . Example:
from omni.isaac.sensor import Camera
from omni.isaac.core.utils.rotations import euler_angles_to_quat
pos_world = np.array([1, 1, 1])
quat_world =euler_angles_to_quat([0, -90, 0], degrees=True)
camera = Camera(position=pos_world,
orientation=quat_world)
camera.get_world_pose() # should give the same as pos_world and quat_world above
camera.set_world_pose(pos_world, quat_world) # same as initialization
If you would like to set the pose of the camera using the USD pose, directly use the XForm and set pose using the data from Property panel.
We are actively looking to improve the camera interface for Isaac Sim and we have improvements coming in the next Isaac Sim release. The features that will help you with this issue are as follows:
The omni.isaac.sensor camera will allow you to get/set of the camera pose in all of the different frames
We will introduce a new GUI extension, Camera Inspector, which will display the camera pose in all of these frames, so that you can take them from the scene and put it into code with ease. This extension will also include some other functionalities, e.g., viewports to allow you to adjust the camera on the fly.
We will introduce a new set of camera tutorials to help you interact with the camera better via GUI and API.
Hi, I found the similar problem with you, and I wrote a pose of it, hope it can help you out.
Solution
According to the discussion in the form, I think its a designed mechanism to automatically keep the same camera coordinate system(+Y up) across different USD files.
And if you want to cancel this “auto-rotation” effect, you can use camera_axes="usd" args in set_world_pose().
from omni.isaac.sensor import Camera
camera = Camera(
prim_path="/World/camera",
position=np.array([0, 0, 0]),
orientation=np.array([1, 0, 0, 0]),
)
camera.set_world_pose(np.array([0, 0, 0]), np.array([1, 0, 0, 0]), camera_axes="usd")
my_world.step(render=True)
This way, the default camera pose will aligned with the world frame.
