Isaac Sim v5. Simple Surface Gripper

Omniverse Isaac Sim
1

System Details

  • Isaac Sim Version: 5.0.0
  • Operating System: Ubuntu 24.04.3 LTS
  • GPU Information:
    • Model: RTX 5080
    • Driver Version: nvidia-driver-580-open=580.65.06-0ubuntu0.24.04.4

Creating the simple surface gripper

Goal

To create a pneumatic device from the several STL files.

Detailed Description

There’s few documentation available on the Internet. Official manual presents:

Samples shipped within the IsaacSim:

  • $isaacsim_workdir/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/surface_gripper/surface_gripper.py
  • $isaacsim_workdir/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/ur10_palletizing/ur10_palletizing.py

Isaac Sim Asserts:

  • Robots → UniversalRobots → ur10 → grippers → {long,short}_gripper.usd

To be honest, the presented samples working quite good. Presented asserts work less stable: only very small objects, wrong positioning after the gripper is closed.

With lack of documentation I’ve spent a couple of weeks reverse engineering all of them.

Eventually, simplified to the minimum example, without Python Code, just to check that Surface Gripper works.

Example:

  • 2 cubes and 1 cone
  • 1 D6 joint
  • 1 Surface Gripper object
  • 4 engines: x/y/z rotate + z transitional (copy/pasted from the tutorial: $isaacsim_workdir/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/surface_gripper/surface_gripper.py)
  • the cube representing pneumatic device is fixed with the Fixed Joint.

USD: pneumo.drive_x1.v3.export.usd.gz (5.1 KB)

Problem

image
image1129×1584 70.1 KB

Pressing “Surface Gripper → Surface Gripper → Close” shows “Closing …” and then Open again. The gripper doesn’t work :(

Screenshots or Videos


Finale

I understood, that I’m missing something. So the questions:

  1. Could I test that the Surface Gripper works just in Isaac Sim GUI, without Python code applied (like in the tutorial: Surface Gripper Extension — Isaac Sim Documentation). Or the Python code is mandatory?
  2. The Surface Gripper with one “cone” doesn’t work stable, should I append more “cones”?
  3. In the Robotics Example → Cortex → UR10 Paletizing tutorial there’s a mystic “motion_commander_target” cube. Definitely, not part of the real equipment in the warehouse. Why do we need it? Looks like some kind of “hack” to force the Surface Gripper to work properly.

image
image1920×894 125 KB

I’ll continue my tries, however, more information / documentation / simple examples is highly desired :)

2

I have similar question. I created one D6 joint and 1 Surface Gripper. I set the Body0 of D6 as the tcp, and left the Body1 empty. Unfortunately it does not work. Waiting for precise senario for setting up the surface gripper in GUI.

3

Related blog post:

https://mattmazzola.medium.com/nvidia-isaac-sim-how-to-publish-the-state-of-a-surface-gripper-to-a-ros2-topic-624842b26323

4

Thank you, edsonbffilho for the link. Unfortunately, it’s for the Isaac v4.0.0. Something has changed in the Surface Gripper API in Isaac v5.0.0 - new D6 engines should be used for the gripper emulation.

5

I’ve spent some additional time reverse engineering the default surface gripper tutorial example:

  • $isaacsim_workdir/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/surface_gripper/surface_gripper.py

What’ve been done:

  1. Found the scene file here: $isaacsim_workdir/exts/isaacsim.examples.interactive/data/SurfaceGripper_gantry.usda
  2. Removed everything from the scene except the surface gripper and boxes
  3. Removed Python code which creates the SurfaceGripper meta object
  4. Created the SurfaceGripper meta object by GUI

What’s interesting:

  1. Somehow, It works. No Python code.
    image
    image2158×556 64.3 KB
  2. I’ve tried to remove all cones except one. Giving more force, the gripper still works!
  3. I’ve enabled the Rigid Boxy / Kinematic flag - and it works!
  4. I’ve added the Fixed Joint with no second body (this way it is “fixed to the air”) - and it works!

So, the problem is located somewhere in the D6 Engine configuration. Still don’t know where exactly.

6

Eventually:

  1. disassembled the tutorial scene file $isaacsim_workdir/exts/isaacsim.examples.interactive/data/SurfaceGripper_gantry.usda
  2. replaced the scene with my own models;
  3. left the nine D6 engines untouched;
  4. saved as a separate USD file;
  5. linked to my own project.

Honestly, don’t know what exactly is different. The mystery with the Robotics Example → Cortex → UR10 Paletizing tutorial’s motion_commander_target remains too.

Dear developers, please, make a good documentation for Isaac 5.0.0+. Especially depicting how D6 engine parameters are treated in the “Surface Gripper mode”. The man: 3 objects(2 cubes, 1 cone), one D6 engine, one Surface Gripper, shouldn’t take too much time.

7

Found the problem empirically. In D6Joint all four Local Position {0,1} and Local Rotation {0,1} must be zeros:

image
image1156×1220 66.3 KB

I had 0.8 in Local Position 0 :: Z, which broke Surface Gripper in 5.0.0. Be especially careful.