Isaac Sim camera orientation auto rotate in set_world_pose()

[voyager.yupeng]

Issue

The default coordinate system in Isaac Sim is Z-up, and the cameras in USD are always y-up.

As many issues in Forums have mentioned, when you creating a camera and set its world pose to dafult using set_world_pose(),

from omni.isaac.sensor import Camera
camera = Camera(
    prim_path="/World/camera",
    position=np.array([0, 0, 0]),
    orientation=np.array([1, 0, 0, 0]),
)
camera.set_world_pose(np.array([0, 0, 0]), np.array([1, 0, 0, 0]))
my_world.step(render=True)

the camera pose is not aligned with the world frame in Isaac Sim, as it theoretically should be. Instead, the camera automatically rotates itself to +Y up. Specifically, the XYZ rotation is (90, -90, 0).

That causes a lot of troubles. For example, when you set the camera pose under the world frame, the camera will first be set to the correct transformation, an then apply a rotation automatically like mentioned above.

Solution

According to the discussion in the form, I think its a designed mechanism to automatically keep the same camera coordinate system(+Y up) across different USD files.

And if you want to cancel this “auto-rotation” effect, you can use camera_axes="usd" args in set_world_pose().

from omni.isaac.sensor import Camera
camera = Camera(
    prim_path="/World/camera",
    position=np.array([0, 0, 0]),
    orientation=np.array([1, 0, 0, 0]),
)
camera.set_world_pose(np.array([0, 0, 0]), np.array([1, 0, 0, 0]), camera_axes="usd")
my_world.step(render=True)

This way, the default camera pose will aligned with the world frame.

You can find more detailed information about set_world_pose() in the API docs of it here.

Same issues in the forum

Orientation of Camera

Hello! I want to attach a camera to custom objects and there are some problems regarding the orientation of the camera. The parent is a XForm prim that is there for proper orientation of the camera, but whatever I try, the orientation does not really change as I would expect. The XForm is also not changing the orientation as I am trying to do using set_local_pose. So, now the problem is that I cannot fix the orientation of the camera such that I can capture synthetic data. It does not matter, …

Different methods for creating camera

I found two methods to create a camera. Method 1: from omni.isaac.core.utils.stage import get_current_stage from pxr import UsdGeom import omni.replicator.core as rep camera_path = "/Camera" camera = get_current_stage().DefinePrim(camera_path, "Camera") camera.GetAttribute("clippingRange").Set(Gf.Vec2f(0.001, 100.0)) camera.GetAttribute("focalLength").Set(30) UsdGeom.Xformable(camera).AddTranslateOp().Set((0.0, 0.0, 1.0)) UsdGeom.Xformable(camera).AddRotateXYZOp().Set((0.0, 0.0, 0.0)) render_…

Replicator Camera Orientation

Hello, I am using the replicator create camera function to generate synthetic data. This camera is randomized using the approach mentioned in the documentation. I already have a list of orientations and positions parameters for the camera to use. These parameters have been collected manually while navigating in my scene using Isaac Sim, and annotating the postion / orientation in the desired locations. While the positions seems to be correct, there is a major problem in the orientation. None o…

Use omniverse api to add camera in scene got rotation problem

i’m using the code below to create a camera, i set up the position and rotation tags, and then i find that a Camera_Xform is created, and a Camera instance is under it. For this step there’s no problem, and the position is only set to Camera_Xform, this also has no problem, but!! the rotation is set up to both Camera_Xform and Camera instance, which means that the camera is rotated twice, why and how to solve this problem? camera = rep.create.camera(position=(4.0, 0, 1.0),rotation=(90,0,90),foc…

How do I set camera rotation in Issac Sim Python standalone app?

Hi, I’m currently using Isaac Sim Python 2022.2.0. I would like to know how to adjust the rotation property of the camera, which can be seen on the screenshot: [image] However, I could only set the camera_position. How can I set the initial camera rotation to [7, 0, -90] in Python standalone app? My code is: class MyEnv(gym.Env): # ... def set_initial_camera_params(self, camera_position=[22, 35, 220], camera_target=[0, 0, 0]): from omni.isaac.core.utils.viewports import se…

Issac Sim | Python API - Camera orientation ignored

Creating a Camera sensor using the Python API, sets a default orientation, that cannot be changed. Trying to set orientation, is just being ignored. Here is a reproduction: from omni.isaac.sensor import Camera right_camera = Camera("/camera_right", name="camera_right", translation=np.array([0, 0, 0]), orientation=np.array([1, 0, 0, 0])) right_camera.set_world_pose(orientation=np.array([1, 0, 0, 0])) right_camera.set_default_state(orientation=np.array([1, 0, 0, 0])) [image]

Hope this features could be mentioned in the following official docs

https://docs.omniverse.nvidia.com/isaacsim/latest/reference_conventions.html#isaac-sim-conventions

https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/isaac_sim_sensors_camera.html#camera

[user146382]

I encountered the same problem when collecting data, and was troubled by this mechanism for a long time, It prevented me from collecting the camera pose correctly.