Replicator Camera Orientation

Omniverse Synthetic Data Generation (SDG)
1

Hello,
I am using the replicator create camera function to generate synthetic data. This camera is randomized using the approach mentioned in the documentation. I already have a list of orientations and positions parameters for the camera to use. These parameters have been collected manually while navigating in my scene using Isaac Sim, and annotating the postion / orientation in the desired locations. While the positions seems to be correct, there is a major problem in the orientation. None of the annotated angles give me the same angle as I earlier saw them in the viewport.

What might be the problem?
Thank you in advance

Best
Anthony

2

@toninsemaan is it possible for you to upload a screenshot or do a side by side comparison to show what you are getting from the annotated angles vs from within the viewport?

3

Hi @toninsemaan ,

More information would be needed to help debug your issue.

Screenshots/images, how you are setting the position and orientations for the new camera, also how you are getting the orientations.

4

Sure!

Create a camera

import omni.replicator.core as rep
rep_camera = rep.create.camera()

Get the orientation / position of all the set cameras in the usd scene and launch the randomizer

stage = omni.usd.get_context().get_stage()
cams_positions = []
cams_rotations = []
camera_paths = ["/World/Camera", "/World/Camera_01"]
for cam_path in camera_paths:
        prim: Usd.Prim = stage.GetPrimAtPath(cam_path)
        orient = prim.GetAttribute("xformOp:orient").Get()
        translate = prim.GetAttribute("xformOp:translate").Get()

        #Then, since the above command gives me the orientation in quad, I am transforming them to euler using the following:
        rotation = quat_to_euler_angles(quat=gf_quat_to_np_array(orient), degrees=True).tolist()

        cams_rotations.append(rotation)
        cams_positions.append(translate)

cams_positions_float = []
cams_rotations_float = []
for i in range(len(cams_positions)):
        if cams_positions[i] is not None and cams_rotations[i] is not None:
                cams_positions_float.append((cams_positions[i][0], cams_positions[i][1], cams_positions[i][2]))
                cams_rotations_float.append((cams_rotations[i][0], cams_rotations[i][1], cams_rotations[i][2]))

# Randomize Camera
with rep.trigger.on_frame(num_frames=len(cams_positions_float)):
        with rep_camera:
                rep.modify.pose(position=rep.distribution.sequence(cams_positions_float),
                                rotation=rep.distribution.sequence(cams_rotations_float))
rp = rep.create.render_product(rep_camera, (512,512), True)
writer.attach([rp])

While the current viewport of the camera are as follows:

image
image1243×659 84 KB

image
image1242×713 40.2 KB

I am the getting rgb synthetic data:

image
image544×542 14.9 KB

image
image538×544 16.5 KB

It feels like the orientation of the camera is following a completely different axis.

Thank you for you time,
Best

Anthony

6

Hello @Simplychenable @dennis.lynch,

Is there any updates on the following issue?

Apologies for the inconvenience.

Thank you,
Best
Anthony

7

just fyi, i am just another user so you can take my input as a grain of salt. always air on the side of what the dev has to say.

that said, regarding the orientation, i am not sure what you are seeing pertains to the z-up discrepancy mentioned in this post. and another optional step if see further mismatch in terms of what you are seeing in the render output, you could try capturing additional camera attributes such as focusDistance, focalLength, clippingRange, etc so it matches exactly to the camera setting used in Stage.