SurfaceGripper fails to create joint with articulated item: "PxD6JointCreate: actors must be different"

[_jon]

Hi All,

I’m currently trying to pick up an object with internal joints using the included UR10 and SurfaceGripper classes. The simulation runs fine (and picks up the object) when there is no joint in the object (i.e. just a simple cube) or if there are no joints in the object I’m trying to pick, but fails with the following message as soon as there is a joint connecting the rigid body that is picked up to something else (e.g., like in a box with a hinge where the lid is lifted or a simplified book with a single hinge). I’m using the standard PickPlaceController from omni.isaac.universal_robots.controllers. Anyone got an idea how to fix this?

This is the error log:

2023-09-18 12:10:47 [15,393,615ms] [Error] [omni.physx.plugin] PhysX error: PxD6JointCreate: actors must be different, FILE /buildAgent/work/16dcef52b68a730f/source/physxextensions/src/ExtD6Joint.cpp, LINE 1141
2023-09-18 12:10:47 [15,393,615ms] [Error] [omni.isaac.dynamic_control.plugin] Failed to create D6 joint
2023-09-18 12:10:47 [15,393,615ms] [Error] [omni.isaac.dynamic_control.plugin] DcSetD6JointProperties: Invalid or expired D6 Joint handle
2023-09-18 12:10:47 [15,393,615ms] [Error] [omni.isaac.dynamic_control.plugin] DcGetD6JointConstraintIsBroken: Invalid or expired D6 Joint handle

This is how I create the UR10, the gripper and the item:
UR10 (from omni.isaac.universal_robots import UR10):

self._ur10_robot = UR10(prim_path=ur10_prim_path, name=ur10_robot_name, attach_gripper=True)

Gripper (from omni.isaac.manipulators.grippers.surface_gripper import SurfaceGripper):

self._gripper = SurfaceGripper(end_effector_prim_path=self._end_effector_prim_path, translate=0.1611, direction="x")

My item:

self._book = scene.add(XFormPrim(
    prim_path = book_prim_path,
    name = book_name,
    position = book_pos,
    orientation = book_rot,
    scale = np.array([1, 1, 1])
))

And this is how the item (in this case a book) is defined in usd:

Picking this does not work resulting in the error message from above. However, if I remove the revolute joint connecting the front cover and page, I can pick the front cover off the book (as expected).

As far as I can tell, the actual implementation of the gripper’s close()-function is inside a compiled cpython file (omni.isaac.surface_gripper._surface_gripper), so I can’t check what’s happening there.

Any Ideas on what could be going wrong? If anyone needs more info, I’m happy to share.

EDIT: Is the UR10-implementation of the SurfaceGripper somehow different than the one in the Manipulation>SurfaceGripper example? Because there, I can duplicate the cube, connect it with a joint to the original cube, and the SurfaceGripper is able to pick them.

[rthaker]

Hi @_jon - The error message you’re seeing is coming from the PhysX physics engine, which is used by Isaac Sim. The error is saying that it’s trying to create a joint (in this case, a D6 joint, which is a type of joint that can represent a variety of mechanical joints) between two actors that are the same. In PhysX, an actor is a physical object that can participate in the simulation, such as a rigid body.

The error is likely occurring because the SurfaceGripper is trying to create a joint between the gripper and the object it’s picking up. If the object already has a joint connecting it to something else, this could potentially cause a conflict.

One possible solution could be to modify the SurfaceGripper class to handle objects with internal joints. This could involve checking if the object has any existing joints, and if so, handling them appropriately. For example, you could potentially disable the existing joints while the object is being picked up, and then re-enable them once the object is released.

Another possible solution could be to modify the object itself so that it doesn’t have any internal joints when it’s being picked up. For example, you could potentially split the object into separate parts, each with its own rigid body, and then use the SurfaceGripper to pick up each part individually.

As for your question about the UR10 implementation of the SurfaceGripper, it’s possible that there could be differences between the UR10 version and the one used in the Manipulation>SurfaceGripper example. The UR10 version might have additional features or modifications to handle the specific requirements of the UR10 robot.

[_jon]

@rthaker

The error is likely occurring because the SurfaceGripper is trying to create a joint between the gripper and the object it’s picking up. If the object already has a joint connecting it to something else, this could potentially cause a conflict.

Hi @rthaker, thank you for your reply. Is there a way for me to know (i.e., print in python) between which two actors the SurfaceGripper is trying to create the joint?

Unfortunately, disabling or removing the joints during pickup is not really an option.

[rgasoto]

Thanks for the detailed explanation of the issue and confirmation that it works when there’s no joint in the object being picked. This is an unexpected behavior and I’ve created a bug to track this issue. I’ll work on a reproduction of the issue using our UR10 sample.

Any implementation of the surface gripper uses the same backend code that generates a D6 Physics joint that simulates a fixed connection with some configurable slack to account for the flexibility of some surface grippers.

I’ll report back here when the issue gets resolved.

[_jon]

Thank you, looking forward to it! I’ll try to fix this on my end in the meantime and do the same.

[_jon]

Alright, I seem to have fixed the issue - although I still don’t quite understand what went wrong.

I removed both revolute joints from the book model and then recreated them. Now everything works.

When I ran a quick simulation with just the book itself, the model seemed to behave as expected (both joints working) before and after the recreation.

@rgasoto, does this help? Any idea what went wrong? For now, I’m happy that everything is running, but I’d like to prevent this from happening in the future.

[rgasoto]

Hah! I was just getting to this issue on my end.

Turns out jointed items are fine, but there’s one catch:

If the surface gripper is added to an articulation (as in the Physx Articulation API) - which robots are - the picked object cannot be an articulation - or needs to have its articulation API disabled - as you can’t join two articulations together. I’ll look into incorporating that in the code, but on the meanwhile just be sure to disable the articulation root of the picked object.

I’m glad it’s working for you now.