Avoid collisions with replicator

Hey there,

For my master thesis I’m trying to load an environment and randomize the scene for synthetic data generation. Specifically, I want to randomize some production modules in a production environment. However, I struggle to implement the avoidance of collisions. In particular, collisions with walls and other production modules of the environment.

I’m trying to use rep.modify.pose() to randomize the position and rotation of the modules in the environment. Even though I’m using rep.physics.collider() the randomized modules appear to collide with the surrounding objects and the environment (as seen in the attached image).

The flat white module shouldn’t touch either the wall or the other module. The green “box” is supposed to be the approximation_shape of the collider, but is always shown a frame too late.

After I read that similar problems were solved with the rep.randomizer.scatter_2d function, I tried to solve it the same way. I made sure that check_for_collision=True. It still results in the same output, that production modules collide with the surrounding environment (e.g. walls and other production modules). Furthermore, the modules are placed in a random position, but the rotation doesn’t change.

This is the Code I’m using so far, but might be hard to recreate as the data are generated with usd files of the production site:

import omni.replicator.core as rep

with rep.new_layer():

    # Load in asset
    ENVS = "/path/to/production_environment.usd"
    # Setup the static elements
    env = rep.create.from_usd(ENVS)

    # identify modules
    # Logistic
    module_logistic_name = "Logistic"
    module_logistic_path = "/Replicator/Ref_Xform/Ref/Assembly/Logistic"

    current_module_path = module_logistic_path
    class_name = module_logistic_name

    current_module = rep.get.prim_at_path(current_module_path)
    # Define randomizer function for the module assets.
    def module_props(curr_m):
        instances = curr_m

        with instances:            
                rotation=rep.distribution.uniform((0, 0, -180), (0, 0, 180)),
                position=rep.distribution.uniform((-10, 12, 0.9), (-5, 20, 0.9)),

        return instances.node

    # Register randomization 

    # Setup camera and attach it to render product
    camera1 = rep.create.camera()
    render_product1 = rep.create.render_product(camera1, resolution=(1024, 1024), force_new=False)
    # Initialize and attach writer
    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize(output_dir=f"/path/to/output", rgb=True, bounding_box_2d_tight=False, bounding_box_2d_loose=True, semantic_segmentation=False)

    with rep.trigger.on_frame(num_frames=10, rt_subframes=20):	
    	with camera1:
    	    rep.modify.pose(position=rep.distribution.uniform((-11, 11, 1), (-2, 11, 3)), look_at=current_module_path)

    # Run the simulation graph
    #import asyncio

Substituting rep.orchestrator.run() with asyncio.ensure_future(rep.orchestrator.step_async()) slowed down the data generation and only generated one image of the origin. However, it didn’t solve the collisions.

For my setup, I’m using:
Isaac Sim: 2022.2.0
Replicator: 1.6.4

@hclever are you the right person to talk to?

I’m happy if you can help me out or push me in the right direction. Thanks a lot!

1 Like

I worked on an example, which shows the same issues that I have with the simulation and you can also try on your own.

I found this post and the code example you gave and adapted it a little bit for my example.

So this is my example code:

import omni.replicator.core as rep

import omni.usd

with rep.new_layer():

    camera = rep.create.camera(position=(1000, 1000, 1000), look_at=(0,0,0))

    render_product = rep.create.render_product(camera, (1024, 1024))
    torus = rep.create.torus(semantics=[('class', 'torus')] , position=(0, -200 , 100),count=3)

    sphere = rep.create.sphere(semantics=[('class', 'sphere')], position=(0, 100, 100),count=3)

    cube = rep.create.cube(semantics=[('class', 'cube')],  position=(100, -200 , 100),count=3)

    plane = rep.create.plane(scale=10, visible=True)

    sphere = rep.create.sphere(semantics=[('class', 'Test_sphere')], position=(1, -1, 0), scale=3)
    with sphere:

    def get_shapes():
        shape = rep.get.prims(semantics=[('class', 'cube'), ('class', 'sphere'), ('class', 'torus')])
        with shape:
            rep.randomizer.scatter_2d(surface_prims=plane, check_for_collisions=True)
            rep.randomizer.color(colors=rep.distribution.uniform((0, 0, 0), (1, 1, 1)))
        return shape.node


    with rep.trigger.on_frame(num_frames=10,rt_subframes=10):
        # with camera:
        #     rep.modify.pose(position=rep.distribution.uniform((-500, 200, -500), (500, 500, 500)), look_at=(0,0,0))
    # Initialize and attach writer
    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize(output_dir=f"/path/to/output/Data", rgb=True, bounding_box_2d_tight=False, bounding_box_2d_loose=True, semantic_segmentation=False)

    import asyncio

with Isaac Sim 2022.2.1 and Replicator 1.7.8 The result is shown in the picture below:

Is there any option to avoid the collision with the bigger sphere?

Hi Hugo,

Unfortunately, this version of replicator (1.7.8) only supports collision checking between the sampled input prims – not any other prims in the scene. You have three options -
(1) wait for the next version of Code / Isaac to be released that has replicator 1.9 (about a month I think)
(2) step physics for 1 step, which should move things a bit out of collision
(3) use my workaround that is detailed in a previous post – here is a link – Scatter_2d collisions with static objects - #18 by jorge29 . You’ll have to replace a number of the scripts in replicator with ones I provided:
OgnScatter2D.py (16.8 KB)

randomizer.py (17.8 KB)

utils.py (44.1 KB)

viewport_manager.py (12.2 KB)

You can run this script to test it:

import omni.replicator.core as rep
with rep.new_layer():
    sphere1 = rep.create.sphere(semantics=[('class', 'sphere')], scale=3, position=(-100, 50, -100), visible=True)
    plane2 = rep.create.plane(scale=4, position = (0, 100, -150), rotation=(45, 0, 0), visible=True)

    def randomize_spheres():
        spheres = rep.create.sphere(scale=0.4, count=30)
        with spheres:
            rep.randomizer.scatter_2d([plane2, sphere1], check_for_collisions=True)
        return spheres.node


    with rep.trigger.on_frame(interval=10, num_frames=1):


The first item in the surface_prims list is the item sampled on (the plane) and the second and every one thereafter are checked for collisions.

Let me know if this works-

I am facing an issue when trying to rotate the scattered prims randomly around the Global Z-Axis. I am using the following Script:

# Randomize Part Position and Rotatation
        def random_Parts():
            parts = rep.get.prims(
                semantics=[('class', 'cube'), ('class', 'keil'), ('class', 'stuetztraeger'), ('class', 'traeger'), ('class', 'verbinder'), ('class', 'profil')]
            floor_prim = rep.get.prims(semantics=('class', 'floor'))

            with parts:
                # Randomize the rotation of the parts
                    min_angle=(0, 0, -180),
                    max_angle=(1, 1, 180)

                # rep.modify.pose(
                #     rotation=rep.distribution.uniform((0, 0, -180), (0, 0, 180))
                # )

                # Scatter it across the plane
            return parts.node

If I am using rep.modify.pose() the objects collide with each other. If i use rep.randomizer.rotation(), the collision detection works, however I need to set all the max_angles larger than the min_angles, otherwise i will get an error message. Maybe someone knows a way to only modify the rotation around the Z-Axis randomly.

Can you just make the difference from min and max to be very small? I.e. try min=(0,0,-180) and max=(0.001, 0.001, 180)? Or is this small change of x and y significant?

If I make the difference very small (e.g. 0.1), Omniverse stops at “RTX ready” and it does not seem to start. I waited for 45min and then it rendered the first frame. After that the rendering continuous, but is super slow (like 500 seconds) per frame.


At a difference of 1 degree, the effect is not very huge if you look straight from the top. But if you look from the side, the rotation is very noticeable, as the parts start to intersect with the floorplane.


I fixed this in Replicator 1.10.5, so that version and anything after it should be good.