How to Add Surface Gripper

Omniverse Isaac Sim
1

Hello,
I’m trying to add a Surface Gripper to the fingertip link of the robot using the following code. However, I’m encountering the following error, and the gripper doesn’t behave as intended. The version of IsaacSim I’m using is 2022.2.1. Additionally, I’m trying to add the Surface Gripper to a parallelized learning environment based on OIGE.

    def set_up_scene(self, scene) -> None:
        # Import before environment parallelization
        self.add_robot()
        self.add_gripper()
        super().set_up_scene(scene, replicate_physics=False)
        self._robots = ArticulationView(prim_paths_expr="/World/envs/.*/robot", name="robot_view")
        scene.add(self._robots)

    def add_gripper(self):
        sgl = Surface_Gripper_Properties()
        sgl.d6JointPath = self.default_zero_env_path + "/robot/hand_left_link/SurfaceGripper"
        sgl.parentPath = self.default_zero_env_path + "/robot/hand_left_link"
        sgl.offset = _dynamic_control.Transform()
        sgl.offset.p.x = 0.016
        sgl.offset.p.y = 0.013
        sgl.offset.p.z = 0.025
        sgl.offset.r = [0.0, 0.0, 0.70710678, 0.70710678]
        sgl.gripThreshold = 0.01
        sgl.forceLimit = 1.0e2
        sgl.torqueLimit = 1.0e3
        sgl.bendAngle = np.pi/4
        sgl.stiffness = 1.0e4
        sgl.damping = 1.0e3

        self.surface_left_gripper = Surface_Gripper(self._dynamic_control)
        self.surface_left_gripper.initialize(sgl)

        sgr = Surface_Gripper_Properties()
        sgr.d6JointPath = self.default_zero_env_path + "/robot/hand_right_link/SurfaceGripper"
        sgr.parentPath = self.default_zero_env_path + "/robot/hand_right_link"
        sgr.offset = _dynamic_control.Transform()
        sgr.offset.p.x = 0.016
        sgr.offset.p.y = 0.013
        sgr.offset.p.z = 0.025
        sgr.offset.r = [0.0, 0.0, 0.0, 1.0]
        sgr.gripThreshold = 0.01
        sgr.forceLimit = 1.0e2
        sgr.torqueLimit = 1.0e3
        sgr.bendAngle = np.pi/4
        sgr.stiffness = 1.0e4
        sgr.damping = 1.0e3

        self.surface_right_gripper = Surface_Gripper(self._dynamic_control)
        self.surface_right_gripper.initialize(sgr)

    def pre_physics_step(self, actions) -> None:
        left_closed = self.surface_left_gripper.is_closed()
        right_closed = self.surface_right_gripper.is_closed()

        print('Left gripper closed:', left_closed)
        print('Right gripper closed:', right_closed)

        # Update gripper
        self.surface_left_gripper.update()
        self.surface_right_gripper.update()

Furthermore, when I launch the simulation, the is_closed() function returns False , but then I get the following error, and it becomes True . I believe based on this post that the offset of the Surface Gripper might not be set correctly and it ends up being placed inside the hand. Is there a good way to debug the placement position of the Surface Gripper?

Left gripper closed: False ### The result of self.surface_left_gripper.is_closed()
Right gripper closed: False ### The result of self.surface_right_gripper.is_closed()

2023-07-17 10:33:58 [13,420ms] [Error] [omni.physx.plugin] PhysX error: PxD6JointCreate: actors must be different, FILE /buildAgent/work/16dcef52b68a730f/source/physxextensions/src/ExtD6Joint.cpp, LINE 1141
2023-07-17 10:33:58 [13,420ms] [Error] [omni.isaac.dynamic_control.plugin] Failed to create D6 joint
2023-07-17 10:33:58 [13,420ms] [Error] [omni.isaac.dynamic_control.plugin] DcSetD6JointProperties: Invalid or expired D6 Joint handle

Left gripper closed: True ### The result of self.surface_left_gripper.is_closed()
Right gripper closed: False ### The result of self.surface_right_gripper.is_closed()
2023-07-17 10:33:58 [13,589ms] [Error] [omni.isaac.dynamic_control.plugin] DcGetD6JointConstraintIsBroken: Invalid or expired D6 Joint handle

I appreciate any assistance or insights you can provide. Thank you.

2

According to the omni.isaac.examples.bin_filling.py script, I believe I can add a Suction Gripper similar to UR10e.

However, I would like to use it in a parallelized learning environment similar to OIGE. Does the Surface Gripper support parallelization?

Thank you.

3

Hi @makoto.sato385 - Yes, the Surface Gripper in Isaac Sim does support parallelization. The Surface Gripper is implemented as an extension (omni.isaac.surface_gripper) and can be used in any environment, including those that are parallelized.

However, it’s important to note that when using the Surface Gripper in a parallelized environment, each instance of the environment will have its own instance of the Surface Gripper. This means that the state of the Surface Gripper in one environment will not affect the state of the Surface Gripper in another environment.

When using the Surface Gripper in a parallelized learning environment, you’ll need to ensure that the gripper’s state is properly initialized for each environment instance, and that any actions applied to the gripper are correctly applied in each environment instance.

In the context of reinforcement learning, this means that the gripper’s state should be included in the environment’s state representation, and any actions applied to the gripper should be part of the action space. This will allow the reinforcement learning algorithm to learn how to control the gripper based on its state and the state of the environment.

2 Likes
4

Hi, @rthaker - Thank you for your response. I will try using the Surface Gripper in a parallel environment.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.