Hey there,
For my master thesis I’m trying to load an environment and randomize the scene for synthetic data generation. Specifically, I want to randomize some production modules in a production environment. However, I struggle to implement the avoidance of collisions. In particular, collisions with walls and other production modules of the environment.
I’m trying to use rep.modify.pose() to randomize the position and rotation of the modules in the environment. Even though I’m using rep.physics.collider() the randomized modules appear to collide with the surrounding objects and the environment (as seen in the attached image).
The flat white module shouldn’t touch either the wall or the other module. The green “box” is supposed to be the approximation_shape of the collider, but is always shown a frame too late.
After I read that similar problems were solved with the rep.randomizer.scatter_2d function, I tried to solve it the same way. I made sure that check_for_collision=True. It still results in the same output, that production modules collide with the surrounding environment (e.g. walls and other production modules). Furthermore, the modules are placed in a random position, but the rotation doesn’t change.
This is the Code I’m using so far, but might be hard to recreate as the data are generated with usd files of the production site:
import omni.replicator.core as rep
with rep.new_layer():
# Load in asset
ENVS = "/path/to/production_environment.usd"
# Setup the static elements
env = rep.create.from_usd(ENVS)
# identify modules
# Logistic
module_logistic_name = "Logistic"
module_logistic_path = "/Replicator/Ref_Xform/Ref/Assembly/Logistic"
current_module_path = module_logistic_path
class_name = module_logistic_name
current_module = rep.get.prim_at_path(current_module_path)
# Define randomizer function for the module assets.
def module_props(curr_m):
instances = curr_m
with instances:
rep.modify.pose(
rotation=rep.distribution.uniform((0, 0, -180), (0, 0, 180)),
position=rep.distribution.uniform((-10, 12, 0.9), (-5, 20, 0.9)),
)
rep.physics.collider(approximation_shape="boundingCube")
return instances.node
# Register randomization
rep.randomizer.register(module_props)
# Setup camera and attach it to render product
camera1 = rep.create.camera()
render_product1 = rep.create.render_product(camera1, resolution=(1024, 1024), force_new=False)
# Initialize and attach writer
writer = rep.WriterRegistry.get("BasicWriter")
writer.initialize(output_dir=f"/path/to/output", rgb=True, bounding_box_2d_tight=False, bounding_box_2d_loose=True, semantic_segmentation=False)
writer.attach([render_product1])
with rep.trigger.on_frame(num_frames=10, rt_subframes=20):
rep.randomizer.module_props(current_module)
with camera1:
rep.modify.pose(position=rep.distribution.uniform((-11, 11, 1), (-2, 11, 3)), look_at=current_module_path)
# Run the simulation graph
rep.orchestrator.run()
#import asyncio
#asyncio.ensure_future(rep.orchestrator.step_async())
Substituting rep.orchestrator.run() with asyncio.ensure_future(rep.orchestrator.step_async()) slowed down the data generation and only generated one image of the origin. However, it didn’t solve the collisions.
For my setup, I’m using:
Isaac Sim: 2022.2.0
Replicator: 1.6.4
@hclever are you the right person to talk to?
I’m happy if you can help me out or push me in the right direction. Thanks a lot!
