Mesh loading for replicator

Omniverse Synthetic Data Generation (SDG)
1

Hey there!
I’m currently getting into the groove with Omniverse and have some questions about how to properly load objects and use precomposed scenes with the replicator.
So what I’ve tried is loading an environment using

env = rep.create.from_usd()

to load a predefined usd file and then use

shapes = rep.get.prims(path_pattern=prim_paths)

to get the predefined objects in the scene and modify them with replicator.
But this doesnt work, it seems like the objects are only imported as references and for some reason replicator can’t interact with those. I also noticed that it is not possible to see or move the object through the UI.

Next I tried to import the usd objects correctly through the UI. Drag and drop did not work, the objects were only created as references - visible in the stage but not in the viewport. Note here that I can see and interact with the objects properly when I open the USD file directly. I played around a bit and found that the object is visible and can be moved through the UI (either by the arrows or by setting translation values) when I choose “Insert as sublayer”. I then used this code to create a new sublayer and load each object in a separate layer:

omni.kit.commands.execute(‘CreateSublayer’,
layer_identifier=stage.GetRootLayer().identifier,
sublayer_position=0,
new_layer_path=text,
transfer_root_content=False,
create_or_insert=False,
layer_name=Object
)

Then I tried to move the objects with replicator by loading the prims through their paths, but again the objects did not move. I tried moving the objects to the main layer by using the Flatten function, but nothing seemed to change. I’ve also tried dragging the meshes from their object xforms to the world xform but I couldn’t figure out how to do that.

I’ve also tried loading the objects using code similar to this:

def env_conference_table(size=5):
confTable = rep.randomizer.instantiate(
rep.utils.get_usd_files(conference_tables, recursive=False),
size=size,
mode=“scene_instance”,
)
with confTable:
rep.modify.pose(
position=rep.distribution.uniform((-500, 0, -500), (500, 0, 500)),
rotation=rep.distribution.uniform((-90, -180, 0), (-90, 180, 0)),
)
return confTable.node

with rep.trigger.on_frame(num_frames=5):
rep.randomizer.env_conference_table(10)

where conference_tables is the path to a folder with my usd object files, but the files were not loaded into the scene.

What worked in the end was creating a scene with Blender, exporting the whole scene as usd and opening that scene and then loading the prims as before. I think the issue with the previous attempts is that the object’s mesh is not in the World Xform, but I’m not sure… maybe I’m also making it way too complicated and I’m the issue :D

Now I’ve got a few questions:

  1. How can I properly work with references (both in the UI and through replicator)
  2. Is there a way to load objects without having to build the scene manually?
  3. How can I move a mesh from one Xform to another?
  4. What exactly went wrong with my previous attempts?
  5. Somewhere along the way I was also at a stage where I could see the objects, but it wasn’t possible to move or rotate them with the arrows you see when you select the object. It worked when I manually added a TransformOp, but I never figured out how to do that through python. I’m not sure how I got to that point, but maybe you can tell me what went wrong and why an object can be imported like that.

Best regards
Sally

2

Hello @sally.zeitler1! I’ve contacted the dev team and informed them of your post. We should hear back from them shortly!

3

Hello @sally.zeitler1! Thank you for reaching out. There’s a lot going on here that isn’t expected, but I’ll provide some ideas that may help us debug this together.

  1. As you noted, using rep.create.fron_usd() attaches the USD as a reference to a new stage in the prim. That means that the prim paths in the stage will be different from the original prim_paths of the referenced USD. If you follow this approach, you can try modifying your prim_paths in the call shapes = rep.get.prims(path_pattern=prim_paths) with a path that doesn’t include the reference’s default prim. You should however have been able to interact with the references in the UI, so there may be more at play here.
  2. The layer approach is a good one, and it maintains the prim_paths you expect. If the prims aren’t being moved, verify that the paths provided to rep.get.prims are correct.
  3. You can manually move a mesh to be a child of an Xform by dragging it in the Stage panel. In script, you can use the following command: omni.kit.commands.execute("MovePrim", path_from="/Replicator/MyMesh", path_to="/Parent/MyXform")
  4. Unfortunately, without the USD to examine, I can only speculate. However one idea that comes to mind is that your referenced USD may not have a default prim set, which can cause issues when referencing. You should see the words (default prim) next to the highest level prim in the Stage panel. If not, you can set it using a right click and selecting Set as Default Prim.
  5. Normally, using the UI to move/scale/rotate a mesh will automatically add transform ops, so there may be something else going on. Replicator will add these automatically as well, but you can do so using USD scripting as follows: UsdGeom.Xformable(prim).AddTranslateOp()

I hope these help, and please reach out again if you’re still encountering issues.