Collision between two Triangle meshs collider or two convext hall colliders not detected

[danielle.sisserman]

Hello, I am writing my own scattering function with the pxr library. Now I am at the point where I need to detect if any two prims are colliding (and if they are, randomize their position again). Every prim which I randomize has a collider, a trigger, a triggerstate, and a kinematic rigidbody. I came across an odd bug( or feature?); The collision between any two colliders of the same approximation type is never detected!
Some combination of types however do detect collision. for example if one mesh is convex hall and the other is triangle mesh then the collision is detected. However, in my case, all the randomized objects must have the same collision type (each mesh must have the ability to detect it’s collisions and re-randomize its position).

[lbenhorin]

Hey,
I can suggest looking at the simulation example scenes - “Objects de-penetration”
This will demo how using OmniGraph nodes you can de-penetrate as many objects you want, I think this will work faster then with standard python calls - I might be wrong here.
You can see this video demonstrating that - Done be intimidated by the large network there, you can a) extract just the fraction you need without the interactivity etc and b) you can code dynamically code this with python
Example video

[lbenhorin]

I made a more clear and simple video - also including the usd file
Simpler video
de-penetration_simplified_more.zip (181.6 KB)

The script to fire a custom event to trigger the graph:

import carb.events
import omni.kit.app

MY_CUSTOM_EVENT = carb.events.type_from_string("omni.my.extension.MY_CUSTOM_EVENT")
bus = omni.kit.app.get_app().get_message_bus_event_stream()
bus.push(MY_CUSTOM_EVENT, payload={"data": 0})

As said in the vid, I assume this can all be implemented in code alone (without OG nodes), maybe other can advise.

[danielle.sisserman]

@lbenhorin thank you, this is an interesting approach.

If I understand correctly, in my extension i trigger a custom event whenever I randomize the positions of the prims.

Two questions come to mind:

1. where do i specify to the action graph for which prims it should check overlaping (we wouldnt want objects like trees, walls, or the robot to be moved)?
2. how can I specify how to modify the position of the prims which are overlapped (we wouldnt want floating objects)?
3. how can I make the graph itierate itself until the prims are not overlapping (X iterations for 1 event trigger)?

These questions might be hard to answer, I think one of the following would be the better approache:

either:

1. get the actual python code of this node (assuming the input to the function is only one prim and not two, we wouldnt want to check for each prim a collision with each other prim, that would be an O^2 calculation):
   

or:
2) get the action graph to trigger an event if there is an overlapping:

and listen to this event in my extension. the event delegate should pass the overlapping pair.

Is either approach 1 or approach 2 possible? If not, then this approach, unfortionatly is none applicable for simple scattering without replicator.

Thank you again,

[lbenhorin]

Great questions -

1. where do i specify to the action graph for which prims it should check overlapping (we wouldn’t want objects like trees, walls, or the robot to be moved)?
   The read prims nodes expects a path, note how the path ends with *, that lets it list every prim of type “Mesh” from its ancestors. Your code can generate all prims into this root, so the graph will always collect them

2. how can I specify how to modify the position of the prims which are overlapped (we wouldn’t want floating objects)?
   This you will have to adept in the separation logic, maybe for forcing Y values to zero or something.

3. how can I make the graph iterate itself until the prims are not overlapping (X iterations for 1 event trigger)?
   I would set a pre-defined, say 10 iterations, which will most likely be enough.

[danielle.sisserman]

@lbenhorin
thank you for your answers but here is why this approach in not applicable for our case:

1. where do i specify to the action graph for which prims it should check overlapping (we wouldn’t want objects like trees, walls, or the robot to be moved)?
*The read prims nodes expects a path, note how the path ends with , that lets it list every prim of type “Mesh” from its ancestors. Your code can generate all prims into this root, so the graph will always collect them

But then how can I check collision between a randomized prim and a wall, but only modify the position of the prim (and not the wall)?

2. how can I specify how to modify the position of the prims which are overlapped (we wouldn’t want floating objects)?
This you will have to adept in the separation logic, maybe for forcing Y values to zero or something.

I am not sure what the “seperation logic” is. Of course I will need control of how to move the objects that I rendomized. The only thing the action graph should do is trigger an even that a pair is overlapping and pass them in the delegate.

3. how can I make the graph iterate itself until the prims are not overlapping (X iterations for 1 event trigger)?
I would set a pre-defined, say 10 iterations, which will most likely be enough.

This could cause problems, at the 10th iteration, prim A from overlapping pair (A,B) could be overlapping with another prim (not B). As stated before, the moving of the prims in case of a collision should be done only from the code. the graph should only trigger an event in the case that there is an overlapping.

That is exactly the problem I am trying to solve here. Omniverse comes with a Collider+Trigger components that should trigger when the mesh is colliding with another collider. That is not working. That is why I need another way to trigger the event of a triggered collosion.

[danielle.sisserman]

Maybe my understanding of how triggered colliders in omniverse work is flaud.

I am looking at the TriggerStateAPI example in Physics Demo Scenes

the boxTrigger has a trigger and a trigger state components and the boxActor has only a collider (no trigger):

If i add a trigger to the boxActor, then the detection not longer works.

Is it possible that trigger colliders collisions are detected only with none-triggered colliders?

[lbenhorin]

Actually there is a better simple way then what i recommended before
based on this - Scatter Examples — Omniverse Extensions latest documentation
You can simply use OG node that handles scattering - it will also take care of self collisions + you can specify objects to avoid.

So what i would do is have your code spawn the randomized objects into a certain scope,
The graph will use what ever objects in the scope, randomize them across the target mesh(s), and handle collisions for you.

[danielle.sisserman]

@lbenhorin this is great for scattering a group of prims and avoid them colliding with each other. But if I understand correctly, the prims can still collide with other meshes in my stage such as the robot and trees/walls. am I wrong?

by the way i’ve made great progress with some amazing help:

Retrieving replicator value

@ahaidu while prepering a simple test case I realized the Issue. First, the setup of the test case: I’ve set up an empty stage with two spheres with colliders and added a PhysicsScene: [Screenshot 2024-04-17 151725] this is the code I paste in the Script Editor: import omni.isaac.core.utils.bounds as bounds_utils import carb from pxr import Gf import omni.timeline timeline = omni.timeline.get_timeline_interface() from omni.physx import get_physx_scene_query_interface def report_hit(hit)…

I believe this is the right approach to detecting triggered collisions (given that they cannot be detected with good’old triggered colliders).

Thank you again for you help

[lbenhorin]

This node also knows to avoid specified meshes, such as your robot.
Take a closer look ;)

Lior Ben Horin
Partner Manager, Developer Relations - Omniverse
NVIDIA

[hclever]

Hi Danielle,

The scatter nodes (scatter2D and scatter3D) have the capability of checking for collisions within sampled prims and also from a list of other provided prims in the scene. You can use scatter2d for surfaces (e.g. unstacked bricks on a pallet) and scatter3d for randomizing within volumes (e.g. confetti in a room)

Here’s an example with scatter2d that uses a sequence of scatters to put objects of heterogeneous types on various surfaces and check for collisions with other things in the scene.

import omni.replicator.core as rep
import asyncio
 
async def main():
    with rep.new_layer():

        mat1 = rep.create.material_omnipbr(
           diffuse=(0.1, 0.1, 0.1),
           roughness=0.99,
           metallic=0.99,
           emissive_color=(0.1, 0.1, 0.7),
           emissive_intensity=500,
        )
        mat2 = rep.create.material_omnipbr(
           diffuse_texture=rep.distribution.choice(rep.example.TEXTURES),
           roughness_texture=rep.distribution.choice(rep.example.TEXTURES),
           metallic_texture=rep.distribution.choice(rep.example.TEXTURES),
           emissive_texture=rep.distribution.choice(rep.example.TEXTURES),
           emissive_intensity=1000,
        )

        mat3 = rep.create.material_omnipbr(
           diffuse_texture=rep.distribution.choice(rep.example.TEXTURES),
           roughness_texture=rep.distribution.choice(rep.example.TEXTURES),
           metallic_texture=rep.distribution.choice(rep.example.TEXTURES),
           emissive_texture=rep.distribution.choice(rep.example.TEXTURES),
           emissive_intensity=200,
        )

        cylinder = rep.create.cylinder(semantics=[('class', 'cylinder')], material=mat1, scale=(2.7, 4, 2.7), position=(-100, 50, -50), visible=True)
        plane_samp = rep.create.plane(scale=3, material=mat2, rotation=(20, 0, 0), visible=True)
        sphere_samp = rep.create.sphere(scale=2.4, material=mat3, position = (0, 100, -180), visible=True)


        def randomize_spheres():
            spheres = rep.create.sphere(position=rep.distribution.uniform((0,0,0), (100,100,100)), scale=0.4, count=20) #48
            cubes = rep.create.cube(position=rep.distribution.uniform((0,0,0), (100,100,100)), scale=0.33, count=20) #48
            cones = rep.create.cone(position=rep.distribution.uniform((0,0,0), (100,100,100)), scale=0.33, count=20) #48
            with rep.utils.sequential():
                with spheres:
                    rep.randomizer.color(colors=rep.distribution.uniform((0, 0, 0), (0.4, 0.4, 0.4)))
                    rep.randomizer.scatter_2d([plane_samp, sphere_samp], no_coll_prims=[cylinder], check_for_collisions=True)
                with cubes:
                    rep.randomizer.color(colors=rep.distribution.uniform((0, 0, 0), (0.4, 0.4, 0.4)))
                    rep.randomizer.scatter_2d([plane_samp, sphere_samp], no_coll_prims=[cylinder, spheres], check_for_collisions=True)
                with cones:
                    rep.randomizer.color(colors=rep.distribution.uniform((0, 0, 0), (0.4, 0.4, 0.4)))
                    rep.randomizer.scatter_2d([plane_samp, sphere_samp], no_coll_prims=[cylinder, spheres, cubes], check_for_collisions=True)
            return spheres.node

        rep.randomizer.register(randomize_spheres)

        with rep.trigger.on_frame(num_frames=10):
            rep.randomizer.randomize_spheres()

asyncio.ensure_future(main())

[hclever]

the “no_coll_prims” arg in the scatter nodes is key to having the scattered objects not collide with other meshes in the scene. In a future release we will have some binary flag to check all prims in the scene for collision but that’s not in yet.

[danielle.sisserman]

@hclever thank you for your input. The issue for us really is that replicator cannot get the robot’s real time position and thus cannot avoid collision with it when using rep.randomizer.scatter_2d.

that is why this thread was created:

Retrieving replicator value

Hi, i have a simple issue I need help with: I have rocks in my scene moved randomly with replicator in my scene. I also have an autonomous robot and I want the rocks to not be positioned within a radius of the robot. def randomize_rocks(stage, robot_position, radius = 1): rocks = rep.get.prims(semantics=[('class', 'rock')]) with rocks: while True: _position = rep.distribution.uniform((-3, -9.5, 0.135), (4, -3.5, 0.135)) # Calculate the distance between t…

That is why, with Lior’s help, we have shiftted from replicator to USD randomization.
ever since then, we are trying to understant how to detect collisions between a trigger collider and X other trigger colliders (as apperantly that functionality does not exist in omniverse?). Any insight on this would be great as we will still need this basic ability.

It does seem that your code should work with avoiding collision with the robot given the robot’s prim path?
how can i get a replication item instance of the robot in my stage?

Thank you again

[danielle.sisserman]

@hclever I ren a quick test of this approach and it does not seem to be working as expected:

this is my code:

plane_path = "/World/Plane"
robot_path = "/World/LawnMower_withTriggerCollider/LawnMowerUSD/Body/CATIA_STL"

def scatter():
    hoses_usd = rep.utils.get_usd_files(HOSE_ON_GROUND_ASSETS_DIR)
    dolls_usd = rep.utils.get_usd_files(DOLLS_ASSETS_DIR)

    traversable_plane = rep.get.prim_at_path(plane_path)

    hoses = rep.randomizer.instantiate(hoses_usd, size=5, use_cache = False)
    dolls = rep.randomizer.instantiate(dolls_usd, size=5, use_cache = False)

    robot = rep.get.prim_at_path(robot_path)

    with rep.utils.sequential():
        with hoses:
            rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [robot, dolls], check_for_collisions=True)
        with dolls:
            rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [robot, hoses], check_for_collisions=True)

when I run it, the hoses are avoiding collisions with each other and so are the dolls. But, dolls and hoses are colliding:

Is there something wrong in the code I wrote?

Thank you

[hclever]

@danielle.sisserman

    with rep.utils.sequential():
        with hoses:
            rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [robot, dolls], check_for_collisions=True)
        with dolls:
            rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [robot, hoses], check_for_collisions=True)

Do you get any errors in the terminal?

I see one issue in the code - the scatters should be written like this:

    with rep.utils.sequential():
        with hoses:
            rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [robot], check_for_collisions=True)
        with dolls:
            rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [robot, hoses], check_for_collisions=True)

where dolls does not appear in the first scatter2d. the dolls have not been scattered yet (they are done second) when hoses are scattered, so they should not be checked in the first scatter.

[hclever]

can you send me your USD files that are imported so I can try this? for:

HOSE_ON_GROUND_ASSETS_DIR

and

DOLLS_ASSETS_DIR

[lbenhorin]

Hey Danielle,
My suggestion is for you to use the scatter node - which will do exactly what you need, but, do not create it using replicator methods, since we understood you need more control.

You should create the node and graph yourself - either by referencing a pre made usd file with a graph or creating it using python.
You can learn how to build graphs with python here - https://docs.omniverse.nvidia.com/kit/docs/omni.graph.docs/latest/Overview.html

Lior Ben Horin
Partner Manager, Developer Relations - Omniverse
NVIDIA

[danielle.sisserman]

@hclever I ren a couple of test cases of the approach you provided. It does work well in case1 but does not work in case2:

case 1:

if i run this in the script editor:

plane_path = "/World/Plane"
cube_path = "/World/Cube"
sphere_path = "/World/Sphere"



with rep.new_layer():
    with rep.trigger.on_time(interval = 1):
        traversable_plane = rep.get.prim_at_path(plane_path)

        cube = rep.get.prim_at_path(cube_path)
        sphere = rep.get.prim_at_path(sphere_path)
        
        with rep.utils.sequential():
            with sphere:
                rep.randomizer.scatter_2d(traversable_plane, check_for_collisions=True)
            with cube:
                rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [sphere], check_for_collisions=True)

then it works great!

case 2:

but if i run this:

plane_path = "/World/Plane"
cube_path = "/World/Cube"
sphere_path = "/World/Sphere"
torus_path = "/World/Torus"


with rep.new_layer():
    with rep.trigger.on_time(interval = 1):
        traversable_plane = rep.get.prim_at_path(plane_path)

        cube = rep.get.prim_at_path(cube_path)
        sphere = rep.get.prim_at_path(sphere_path)
        torus = rep.get.prim_at_path(torus_path)

        with rep.utils.sequential():
            with sphere:
                rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [torus], check_for_collisions=True)
            with cube:
                rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [sphere, torus], check_for_collisions=True)

then it does not.

there are no errors or warnings in the console.

any ideas on how to fix this?

thanks again for all of the much needed help

[danielle.sisserman]

@lbenhorin this might be a good approach. I’m pretty sure the node is exactly the replicator function scatter_2d. are you certain that this replicator node can be triggered by a custom event from an extension that does not use replicator? from an async function that triggers every x seconds?

[danielle.sisserman]

@hclever I’ve looks at this further.

it seems that the issue is here:

 with cube:
    rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [sphere, torus], check_for_collisions=True)

if i switch between sphere and torus:

 with cube:
    rep.randomizer.scatter_2d(traversable_plane, no_coll_prims = [torus, sphere], check_for_collisions=True)

then now the cube and the sphere will collide (instead of the cube and the torus).

If however, i dont use:

cube = rep.get.prim_at_path(cube_path)
sphere = rep.get.prim_at_path(sphere_path)
torus = rep.get.prim_at_path(torus_path)

and instead use:

cylinder = rep.create.cylinder()
sphere = rep.create.sphere()
cube = rep.create.cube()

then everything works as expected:

cylinder = rep.create.cylinder()
sphere = rep.create.sphere()
cube = rep.create.cube()
plane_path = "/World/Plane"


with rep.new_layer():
    with rep.trigger.on_time(interval = 1):
        traversable_plane = rep.get.prim_at_path(plane_path)

        with rep.utils.sequential():
            with sphere:
                rep.randomizer.scatter_2d([traversable_plane], no_coll_prims=[cylinder], check_for_collisions=True)
            with cube:
                rep.randomizer.scatter_2d([traversable_plane], no_coll_prims=[cylinder, sphere], check_for_collisions=True)

It seems that maybe there is an issue with rep.get.prim_at_path?