Isaac Sim camera orientation auto rotate in set_world_pose()

Issue

The default coordinate system in Isaac Sim is Z-up, and the cameras in USD are always y-up.

As many issues in Forums have mentioned, when you creating a camera and set its world pose to dafult using set_world_pose(),

from omni.isaac.sensor import Camera
camera = Camera(
    prim_path="/World/camera",
    position=np.array([0, 0, 0]),
    orientation=np.array([1, 0, 0, 0]),
)
camera.set_world_pose(np.array([0, 0, 0]), np.array([1, 0, 0, 0]))
my_world.step(render=True)

the camera pose is not aligned with the world frame in Isaac Sim, as it theoretically should be. Instead, the camera automatically rotates itself to +Y up. Specifically, the XYZ rotation is (90, -90, 0).
Pasted image 20240501180515 Pasted image 20240501180530

That causes a lot of troubles. For example, when you set the camera pose under the world frame, the camera will first be set to the correct transformation, an then apply a rotation automatically like mentioned above.

Solution

According to the discussion in the form, I think its a designed mechanism to automatically keep the same camera coordinate system(+Y up) across different USD files.

And if you want to cancel this “auto-rotation” effect, you can use camera_axes="usd" args in set_world_pose().

from omni.isaac.sensor import Camera
camera = Camera(
    prim_path="/World/camera",
    position=np.array([0, 0, 0]),
    orientation=np.array([1, 0, 0, 0]),
)
camera.set_world_pose(np.array([0, 0, 0]), np.array([1, 0, 0, 0]), camera_axes="usd")
my_world.step(render=True)

This way, the default camera pose will aligned with the world frame.
Pasted image 20240501181344 Pasted image 20240501181355

You can find more detailed information about set_world_pose() in the API docs of it here.

Same issues in the forum

Hope this features could be mentioned in the following official docs

https://docs.omniverse.nvidia.com/isaacsim/latest/reference_conventions.html#isaac-sim-conventions

https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/isaac_sim_sensors_camera.html#camera

1 Like