Best way to instance random USD objects for use in scatter calls?

Omniverse Synthetic Data Generation (SDG)
1

I’m trying to do something simple on paper, but seemingly difficult with the replicator API.

I need to generate a set of n images that depict the YCB objects on a table. It needs the following randomizations per generated frame:

  • Random n objects on table (no duplicates)
  • Random positions and rotations (within defined boundary)

The trouble comes needing a collision-free placement of objects. It’s not clear to me how you’re supposed to do this in an elegant way. Other than bringing in physics, it seems like the only guaranteed way to place the objects flush on the table surface, without other object collisions, is make a plane for each object. This is because of the different centroid offsets relative to the table surface height. Once you get them, you have to put in the funny sequential block with the incrementing no_coll_prims like so:

        with rep.utils.sequential():
            with cracker_box:
                rep.randomizer.scatter_2d(
                    cracker_plane_samp,
                    seed=1,
                    no_coll_prims=[],
                    check_for_collisions=True,
                )
            with sugar_box:
                rep.randomizer.scatter_2d(
                    sugar_plane_samp,
                    seed=2,
                    no_coll_prims=[
                        rep.get.prims(path_match="03_cracker_box", prim_types=["Mesh"])
                    ],
                    check_for_collisions=True,
                )
            with soup_can:
                rep.randomizer.scatter_2d(
                    soup_plane_samp,
                    seed=3,
                    no_coll_prims=[
                        rep.get.prims(path_match="03_cracker_box", prim_types=["Mesh"]),
                        rep.get.prims(path_match="04_sugar_box", prim_types=["Mesh"]),
                    ],
                    check_for_collisions=True,
                )
            with mustard_bottle:
                rep.randomizer.scatter_2d(
                    mustard_plane_samp,
                    seed=4,
                    no_coll_prims=[
                        rep.get.prims(path_match="03_cracker_box", prim_types=["Mesh"]),
                        rep.get.prims(path_match="04_sugar_box", prim_types=["Mesh"]),
                        rep.get.prims(
                            path_match="05_tomato_soup_can", prim_types=["Mesh"]
                        ),
                    ],
                    check_for_collisions=True,

So I can get pretty far with this:

usd_files = rep.utils.get_usd_files(OBJECTS_DIR, recursive=False)
    
# this is the structure of dict for each YCB file mapped to a semantic class and height value to plug in 
objects_data = {"002_master_chef_can.usd": {"semantic_class": "blue Master Chef coffee bean can", "height": 0.070089,},}  

def randomize_objects():
        num_objects = random.randint(1, 6)
        selected_assets = random.sample(usd_files, num_objects)
        print(f"Selected assets: {selected_assets}")

        objects = []
        no_coll_prims = []

        with rep.utils.sequential():
            for i, asset in enumerate(selected_assets):
                asset_name = asset.split('/')[-1]
                object_data = objects_data.get(asset_name, {"semantic_class": "unknown object", "height": 0})
                semantic_class = object_data["semantic_class"]
                height = object_data["height"]
                print(f"Adding asset: {asset_name} with semantic class: {semantic_class} and height: {height}")

                obj = rep.create.from_usd(asset, semantics=[("class", semantic_class)])
                plane_samp = rep.create.plane(scale=1, position=(0, 0, height), rotation=(90, 0, 0), visible=False)

                with obj:
                    print(f"Modifying pose for {asset_name}")
                    rep.modify.pose(rotation=rep.distribution.uniform((-90, 0, -180), (-90, 0, 180), seed=i))
                    print(f"Scattering {asset_name}")
                    sys.stdout.flush()
                    try:
                        rep.randomizer.scatter_2d(
                            plane_samp, 
                            seed=i, 
                            no_coll_prims=no_coll_prims, 
                            check_for_collisions=True
                        )
                        print(f"Successfully scattered {asset_name}")
                    except Exception as e:
                        print(f"Error scattering {asset_name}: {e}")

                no_coll_prims.append(obj.node)
                objects.append(obj.node)
                print(f"Completed adding {asset_name}")

        return objects

But the objects and quantity obviously are going to be same for each frame trigger because that selected_assets list only get made once. Only the poses change per frame.

I’ve tried using instantiate, but I can’t seem to get it to work at all with the overall loop structure here. I get a bunch of errors related to parsing the ReplicatorItem made by calling instantiate. Can’t figure out how to get the instantiate prims to play as nicely as the ones I can get by calling create.from_usd.

Is instantiate the correct approach, and I just need to reevaluate my control flow?