For my master thesis I’m trying to load an environment and randomize the scene for synthetic data generation. Specifically, I want to randomize some production modules in a production environment. However, I struggle to implement the avoidance of collisions. In particular, collisions with walls and other production modules of the environment.
I’m trying to use rep.modify.pose() to randomize the position and rotation of the modules in the environment. Even though I’m using rep.physics.collider() the randomized modules appear to collide with the surrounding objects and the environment (as seen in the attached image).
The flat white module shouldn’t touch either the wall or the other module. The green “box” is supposed to be the approximation_shape of the collider, but is always shown a frame too late.
After I read that similar problems were solved with the rep.randomizer.scatter_2d function, I tried to solve it the same way. I made sure that check_for_collision=True. It still results in the same output, that production modules collide with the surrounding environment (e.g. walls and other production modules). Furthermore, the modules are placed in a random position, but the rotation doesn’t change.
This is the Code I’m using so far, but might be hard to recreate as the data are generated with usd files of the production site:
import omni.replicator.core as rep with rep.new_layer(): # Load in asset ENVS = "/path/to/production_environment.usd" # Setup the static elements env = rep.create.from_usd(ENVS) # identify modules # Logistic module_logistic_name = "Logistic" module_logistic_path = "/Replicator/Ref_Xform/Ref/Assembly/Logistic" current_module_path = module_logistic_path class_name = module_logistic_name current_module = rep.get.prim_at_path(current_module_path) # Define randomizer function for the module assets. def module_props(curr_m): instances = curr_m with instances: rep.modify.pose( rotation=rep.distribution.uniform((0, 0, -180), (0, 0, 180)), position=rep.distribution.uniform((-10, 12, 0.9), (-5, 20, 0.9)), ) rep.physics.collider(approximation_shape="boundingCube") return instances.node # Register randomization rep.randomizer.register(module_props) # Setup camera and attach it to render product camera1 = rep.create.camera() render_product1 = rep.create.render_product(camera1, resolution=(1024, 1024), force_new=False) # Initialize and attach writer writer = rep.WriterRegistry.get("BasicWriter") writer.initialize(output_dir=f"/path/to/output", rgb=True, bounding_box_2d_tight=False, bounding_box_2d_loose=True, semantic_segmentation=False) writer.attach([render_product1]) with rep.trigger.on_frame(num_frames=10, rt_subframes=20): rep.randomizer.module_props(current_module) with camera1: rep.modify.pose(position=rep.distribution.uniform((-11, 11, 1), (-2, 11, 3)), look_at=current_module_path) # Run the simulation graph rep.orchestrator.run() #import asyncio #asyncio.ensure_future(rep.orchestrator.step_async())
asyncio.ensure_future(rep.orchestrator.step_async()) slowed down the data generation and only generated one image of the origin. However, it didn’t solve the collisions.
For my setup, I’m using:
Isaac Sim: 2022.2.0
@hclever are you the right person to talk to?
I’m happy if you can help me out or push me in the right direction. Thanks a lot!