Objects clipping through gripper

Omniverse Isaac Sim
1

Greetings,

Currently, I am in the process of implementing a robotics application that involves inserting a pin into a hole. However, I am encountering a problem where the pin is passing through the gripper fingers, even though both the pin and gripper fingers are defined as sdf mesh. I have experimented with various settings such as altering the colliders of the gripper fingers and pin, adjusting the joint force of the gripper, modifying the time steps per second value, and locking all positional and rotational axes of the pin. Unfortunately, none of these measures have proven effective, and the pin continues to pass through the gripper fingers. It is worth noting that collisions seem to function properly when I designate the pin as a kinematic actor, but this is not a viable solution for simulation purposes. I have included a video below that showcases this behavior.

I would appreciate any insights into the cause of this issue.

Best regards

1 Like
2

If I use a cube as object, the collision works properly (convex hull collider on cube), a simple cylinder with convex hull does not work:

5

Hi Axel,

We have been working on improving SDF collision behavior for the next release so there are improvements coming.

But I am still surprised that this case is not working for you. Just to confirm:

  • Gripper: SDF, Pin Mesh: SDF - does not work
  • Gripper: SDF, Cube Mesh: Convex Hull - works
  • Gripper: SDF, Cylinder Mesh: Convex Hull - does not work

I would like to make sure that we either have fixed your issue already or can address it in time for the next release. Is there any chance that you could share a minimal repro for the gripping issues?

Do you see any warnings in the log that could hint an an issue? Notable warnings would include GPU contact buffers not being sized large enough.

Thank you,
Philipp

6

Hi
The collider of the pin, visualized with the green lines, looks a bit like a triangle mesh collider and not an SDF collider but it is hard to see in the video. Triangle mesh colliders work on static or kinematic objects but not on dynamic rigid objects. Dynamic rigids need an SDF collider for the Pin and the Grippers.
Sharing an usd to reproduce the issue would be great. You could delete everything except the gripper and the pin. Saving using “Save Flattened as” would be useful since everything will end up in a single file that you can easily share. If you save as usda, you can even check in a text editor that there is no confidential data in the file that you share.

1 Like
7

Hi Phillip and twidmer,

yes, that is correct:

  • Gripper: SDF, Pin Mesh: SDF - does not work
  • Gripper: SDF, Cube Mesh: Convex Hull - works
  • Gripper: SDF, Cylinder Mesh: Convex Hull - does not work

Here is the example usd:
example_gripper_collisions.usd (14.8 MB)

Once the simulation is started, rotate the robot by 90°. This action will correctly position the robot. Additionally, I have prepared a small script for opening and closing the grippers:

from pxr import Usd, UsdGeom, UsdPhysics, UsdShade, Sdf, Gf, Tf, PhysxSchema

stage = omni.usd.get_context().get_stage()

finger_right_path = "/World/ur5e/egk_40_ei_m_b/schunk_egk_40_ei_m_b/main_body/backe_right"
finger_left_path = "/World/ur5e/egk_40_ei_m_b/schunk_egk_40_ei_m_b/main_body/backe_left"

finger_right = stage.GetPrimAtPath(finger_right_path)
finger_left = stage.GetPrimAtPath(finger_left_path)

gripper_open = 0

if gripper_open == 0:
    finger_right.GetAttribute("drive:linear:physics:targetPosition").Set(0)
    finger_left.GetAttribute("drive:linear:physics:targetPosition").Set(0)
else:
    finger_right.GetAttribute("drive:linear:physics:targetPosition").Set(0.04)
    finger_left.GetAttribute("drive:linear:physics:targetPosition").Set(-0.04)

Here is a short video demonstrating the process:

The only warning I encountered, that could be connected to this, is something about inertia tensors. You will probably see it in the example.

Thank you,
Axel

8

Hi Axel,

Great, thank you very much for the repro - we will have a look and get back to you.

Philipp

9

Hi,

I tested a bit with the scene. I was not able to get it working but there are a couple of issues that need to get fixed. The console shows a couple of problems. I would recommend to load the console when running Isaac Sim since it sometimes provides useful debug information. The messages I get are the following

2023-05-22 08:25:33 [38,771ms] [Warning] [omni.physx.plugin] The rigid body at /World/ur5e/base has a possibly invalid inertia tensor of {1.0, 1.0, 1.0} and a negative mass, small sphere approximated inertia was used. Either specify correct values in the mass properties, or add collider(s) to any shape(s) that you wish to automatically compute mass properties for. If you do not want the objects to collide, add colliders regardless then disable the 'enable collision' property.
2023-05-22 08:25:33 [38,775ms] [Warning] [omni.physx.plugin] The rigid body at /World/ur5e/egk_40_ei_m_b/schunk_egk_40_ei_m_b/grundbacke_left has a possibly invalid inertia tensor of {1.0, 1.0, 1.0} and a negative mass, small sphere approximated inertia was used. Either specify correct values in the mass properties, or add collider(s) to any shape(s) that you wish to automatically compute mass properties for. If you do not want the objects to collide, add colliders regardless then disable the 'enable collision' property.
2023-05-22 08:25:33 [38,777ms] [Warning] [omni.physx.plugin] The rigid body at /World/ur5e/egk_40_ei_m_b/schunk_egk_40_ei_m_b/grundbacke_right has a possibly invalid inertia tensor of {1.0, 1.0, 1.0} and a negative mass, small sphere approximated inertia was used. Either specify correct values in the mass properties, or add collider(s) to any shape(s) that you wish to automatically compute mass properties for. If you do not want the objects to collide, add colliders regardless then disable the 'enable collision' property.
2023-05-22 08:25:33 [38,779ms] [Warning] [omni.physx.plugin] The rigid body at /World/ur5e/flange/tool0 has a possibly invalid inertia tensor of {1.0, 1.0, 1.0} and a negative mass, small sphere approximated inertia was used. Either specify correct values in the mass properties, or add collider(s) to any shape(s) that you wish to automatically compute mass properties for. If you do not want the objects to collide, add colliders regardless then disable the 'enable collision' property.
2023-05-22 08:25:33 [38,781ms] [Warning] [omni.physx.plugin] The rigid body at /World/ur5e/world has a possibly invalid inertia tensor of {1.0, 1.0, 1.0} and a negative mass, small sphere approximated inertia was used. Either specify correct values in the mass properties, or add collider(s) to any shape(s) that you wish to automatically compute mass properties for. If you do not want the objects to collide, add colliders regardless then disable the 'enable collision' property.
2023-05-22 08:27:09 [135,271ms] [Warning] [omni.physicsschema.plugin] ScaleOrientation is not supported for rigid bodies, prim path: /Cube/Pin_in_hole/zylinderstift_10x40/zylinderstift_10x40_1. You may ignore this if the scale is close to uniform.
2023-05-22 08:27:09 [135,275ms] [Warning] [omni.physicsschema.plugin] ScaleOrientation is not supported for rigid bodies, prim path: /Cube/Pin_in_hole/zylinderstift_10x40/zylinderstift_10x40_1. You may ignore this if the scale is close to uniform.
2023-05-22 08:27:09 [135,280ms] [Warning] [omni.physicsschema.plugin] ScaleOrientation is not supported for rigid bodies, prim path: /Cube/Pin_in_hole/zylinderstift_10x40/zylinderstift_10x40_1. You may ignore this if the scale is close to uniform.

This means that there are several rigid body APIs on meshes or XForms that don’t have a collider API (the collider API can be on the same prim or on a child prim). Rigid bodies must have a collider associated to calculate mass properties. Otherwise the rigid API does not really make sense or gets artificial mass values applied. The second and even more important point is that non-uniform scaling is only supported to some degree. If non-uniform scaling is applied to an xform and then child nodes use a rotation and another child node even further down the hierarchy uses scaling again, this can lead to skew. Skew is not supported for meshes that have physics applied. As a rule of thumb, non-uniform scaling should only be used on meshes that don’t have children or if there are child prims they should not have a transforms and inherit the transform from their parent.

Hope that helps to get a step further with debugging.

10

Hi twidmer,

thank you for the tips.

I have managed to fix the issue.

I discovered that the SDF collision-body does not rotate along with the robot (if the simulation is running). You can observe the problem in this video:

To address this, I defined a new SDF collision body after rotating the robot. Although the SDF appeared in the correct position, it remained stationary and did not move as expected. You can view the behavior in this video:

If I solely utilized the joints to manipulate the robot, everything worked as intended:

I have provided the updated usd-file for your reference (also fixed the issues in the console):
example_gripper_collisions_2.usd (14.8 MB)

I also noticed that the cube, originally defined as a static triangle mesh, does not collide properly with the gripper. However, when I define the cube as a convex hull instead, the collision issue is resolved. Are collisions between static triangle meshes and SDFs supported?

11

Static triangle meshes can collide against SDFs. Older versions of the PhysX SDK (which powers Isaac Physics) might not generate enough contact points. Newer versions will fix that. You can try to work around it by using static triangle meshes with more/smaller triangles. But the collisions do work. I tested your scene, duplicated a gripper part, added its own rigid body API and it did collide with the cube. But I still see the problems I described before in the scene like a mesh with child nodes (that is not supported in usd) and a chain with multiple non-uniform scalings across the hierarchy.

12

Thank you for sharing your findings and testing the scene.

Regarding the issues you mentioned, I have a few questions:

  • Newer Versions of Isaac Sim: Do you have any information on when the newer versions of the PhysX SDK, will be integrated into Isaac Sim? It would be helpful to know an estimated timeline for these improvements.

  • Child Nodes of Meshes: When you referred to child nodes of meshes, do you mean the GeomSubset nodes? As far as I can see, I couldn’t find any other child nodes on meshes. The GeomSubset child nodes are automatically generated when converting a step file to USD using the CAD importer extension. If there are other types of child nodes causing issues, please let me know so that I can investigate further.

  • Non-Uniform Scaling Prims: Besides the cube, I couldn’t locate any other prims with non-uniform scaling. If there are specific prims that exhibit this behavior, I would appreaciate if you could provide more details so that I can address and rectify the scaling inconsistency.

Regarding the CAD Importer: I’ve noticed that when converting a STEP file (metric system) to USD, the resulting object is 100 times larger than in the CAD program. My colleagues have encountered the same issue. We’ve been using the unit converter to scale the individual objects back to the correct size. Is this a bug, or is there a parameter that can be adjusted to resolve this scaling discrepancy?

13

Hi Axel,

  • @rthaker can comment/correct about an IsaacSim update timing, I think it should be in the next couple of months.
  • GeomSubsets are fine as descendants, but Gprims cannot be nested (USD Terms and Concepts — Universal Scene Description 23.05 documentation) I assume you get a warning in the console about this.
  • As per the warnings that @twidmer mentions above, the skew issue concerns the /Cube/Pin_in_hole/zylinderstift_10x40/zylinderstift_10x40_1. You should be able to go up the hierarchy to find where skew is introduced / or you can check if the scaling is close to uniform/unskewed on the zylinderstift directly as the warning mentions, if it’s close to uniform/unskewed you are also fine.

I am not familiar with the CAD importer, maybe @rthaker can help with this question.

Thank you,
Philipp

14

Thank you for the clarification.

I have conducted some experiments with the original implementation and managed to resolve the collision issues. However, I have to assign a physics material to the pin and the Gripper, switch to another collider type, and then switch back to the SDF collider in order for it to work. Interestingly, this step is not necessary in the simplified example and it works without assigning a physics material.

I’m unsure if this behavior is normal, but it appears that the convex hull collision is more stable than the SDF collisions. However, both methods still produce suboptimal results.

Here’s an example using the pin collider defined as convex hull and the gripper as SDF (with 120 time steps per second):

And here’s another example with the pin collider defined as SDF (also with 120 time steps per second):

As you can see, there is noticeable jittering. Are there any parameters that could potentially address this issue?

Additionally, there seems to be significant penetration between the rigid bodies, as evident in the following example (both gripper and pin colliders are defined as SDF, with 240 time steps per second):

Are there any parameters that I can adjust to mitigate this problem?

15

For what to work specifically? This sounds like a potential bug and if you could share detailed repro steps that would be appreciated.

For the jitter, it’s not easy to say - do you have a similar simplified repro available for this as you provided earlier?

Thank you,
Philipp

16

For the collisions to work properly. Otherwise the pin clips through the gripper, as seen in the video of the first post.

I will try to reproduce it in a simplified version.

Here is the file for this part:
example_gripper_collisions_3.usd (14.8 MB)

Kind regards

Axel

17

Unfortunately, I encountered difficulties while attempting to make the rmpflow-controller function in a simplified, standalone example

I instead experimented in a standalone example without the controller, but could not reproduce the issues.

Here is the example, maybe this helps to improve sdf-collisions:
simplified_example.usd (12.1 MB)

test_robot_control.py (3.4 KB)

Just save the usd-file in the same folder than the standalone python file, and then run the standalone script. For some reason gpu dynamics gets disabled in the standalone version and I could not change the physics scene attributes in python, so you have to manually activate gpu dynamics and set your desired time steps per seconds value.

In the provided example, I noticed that assigning a material appeared to enhance the performance of sdf-collisions, as demonstrated in the accompanying video. Additionally, when using a higher physics dt (time step), the convex hull collider seemed to outperform the sdf-collider. The video was recorded with 60 time steps per second.

I will provide an update if I manage to reproduce the issues in a simplified version.

Kind Regards

18

Found the reason, why the physics scene attributes would not change in the script.
The physics scene prim is overwritten when defining my_world. Defining my_world before changing the physics scene attributes fixes this issue.

Here is the updated standalone script:
test_robot_control.py (3.4 KB)

1 Like
19

I experimented a bit and found out, that some of the wired physics behavior and the jittering is caused by the rmpflow controller.

For my experiment, I still used rmpflow to controll the robot and left everything the same, except for when the gripper should be closed or opened. Instead of controlling the gripper through rmpflow, I directly set the target position of the linear drive in the gripper.

Those are the results (120 time steps per second), I would recomment to slow down the videos:

SDF-Collider on the pin, without setting a material:

Directly changing Target position:

rmpflow:

SDF-Collider on the pin, with material:

Directly changing Target position:

rmpflow:

Just for reference, when I control the gripper through rmpflow, I get the the controller action through the forward method of the paraller gripper object and then apply them with the articulation controller:

actions = self._gripper.forward(action="close")
self.articulation_controller.apply_action(actions)

The results of the direct joint position control look very good, but the control through rmpflow is unusable.

20

Thank you for the update - the material difference is still interesting to see for me; what is the difference between the material that you apply and the default material? Is the effective mass different for the case where you apply a material?

For rmp flow, I think @rthaker could provide/obtain more information.

21

I am not sure about the default material, but I can read the mass property of the pin:

default material: 0.003030497347936034 kg

new material (steel): 0.023789402097463608 kg

And here are the properties of the new material:

Dynamic friction: 0.4
Static friction: 0.4
Restitution: 0.2
Density: 7850 kg/mÂł
Friction combine mode: average
Restitution combine mode: average

You can also test this in the standalone example I posted earlier:

The same effect can be observed here. When the new material (Steel_Gripper, already in the scene and applied to the gripper fingers) is applied to the pin, the collisions seem more realistic.

22

Hi @axel.goedrich - then I would think that the better mass ratio between pin and robot gripper with the new material provides better collision fidelity. Do you need it to work well with the lower mass or not?