Add gripper to UR5e

Hi I try to add a gripper (robotiq 2f 85) to UR5e.
Both models are imported from URDF. I follow this post to add gripper to UR5e.

The problem is that I can’t control UR5e with gripper (gripper can’t be controlled, too), but I can control UR5e without gripper. Here is the USD of UR5e and gripper: (3.3 MB)

Besides, the rendering of UR5e is strange (some black area is on the UR5e).


I did a quick test with the provided URDF and the joints can be controlled from the UI. Can you elaborate on how you are trying to control the robot? perhaps I can give more guidance then.

The gripper was imported correctly but because of the nature of the gripper and URDF not supporting cyclical joint loops, the urdf provides multiple joints per gripper finger. See second video:

Hi, I want to use python script to control UR5e through omni.isaac.dynamic_control
I try to write a python script: (2.4 KB)
I put this script to isaac-sim-2020.2.2007-linux-x86_64-release/python_samples and follow Getting Started to set up python environment.
Also, I add omni.isaac.dynamic_control to enabledBuiltin in experiences/isaac-sim-python.json.

When I execute the script, the UR5e shows up. But set_articulation_dof_position_targets doesn’t work for me.
I think the problem may be a warning:
[Warning] [omni.isaac.dynamic_control.plugin] Articulation is not in a physics scene, some functionality is missing, make sure that a physics scene is present and simulation is running
I don’t know how to fix it. Could you please check it for me? Thanks!

Hi @q199493422

it seems your simulation doesn’t have a PhysicsScene. You can create/add it using menu Physics > Create > Physics Scene

You can read more details here: Physics — Omniverse Create documentation

Also, you can create/add it using python code as defined in documentation:

Create A Physics Scene: Robotics Python Manual — Omniverse Robotics documentation

Hi, My ur5e.usd already contained PhysicsScene. However, the warning still exists.
Could you download the script and usd files to check them for me? Thanks!

Hi @q199493422

The articulation must be called while the simulation is running.
To do that, you can subscribe a callback to the editor step event and control the articulation there

I edited your code as shown below and its works for me 😅
I used the UR10 robot for that (because I don’t have the ur5e.usd asset). Also, you can remove some print statements

    def editor_update(self, step):
        print("step", step)
        if not self.ur5e_ar:
            self.ur5e_ar = self._dc.get_articulation("/ur10")

        joint_target = [1.5, 0, 0, 0, 0, 0]
        self._dc.set_articulation_dof_position_targets(self.ur5e_ar, joint_target)
        print("step done", time.time())

    def load_ur5e(self):
        print("load", time.time())
        self._dc = _dynamic_control.acquire_dynamic_control_interface()
        _physxIFace = _physx.acquire_physx_interface()
        self._ar = _dynamic_control.INVALID_HANDLE
        add_ground_plane(self.stage, "/groundPlane", "Z", 1000.0, Gf.Vec3f(0.0), Gf.Vec3f(1.0))
        light_prim = UsdLux.DistantLight.Define(self.stage, Sdf.Path("/World/defaultLight"))

        self.update_sub = self.kit.editor.subscribe_to_update_events(self.editor_update)
        self.ur5e_ar = None

        print("before play", time.time())
        print("after play", time.time())
        # time.sleep(0.5)
        # self.ur5e_ar = self._dc.get_articulation("/ur5e")
        while True:
            print("update", time.time())
            # joint_target = [1.5, 0, 0, 0, 0, 0]

            # self._dc.set_articulation_dof_position_targets(ur5e_ar, joint_target)

By the way, remember to wake up the articulation ( if it is sleeping) before control it:

self._dc.set_articulation_dof_position_targets(self.ur5e_ar, joint_target)

Cite from Physx: Articulations — NVIDIA PhysX SDK 4.0 Documentation

“Like rigid dynamic objects, articulations are also put into a sleep state if their energy falls below a certain threshold for a period of time. In general, all the points in the section Sleeping apply to articulations as well. The main difference is that articulations can only go to sleep if each individual articulation link fulfills the sleep criteria.”

Hi, thanks for your thoroughly explaining!
I modify my code as your suggestion (3.1 KB) .
I use ur10 for testing now, but I still can’t control it.
When I run my code, the step in editor_update keeps showing 0:

step 0.0
step done 1619402965.6744826
update 1619402965.689975
step 0.0
step done 1619402965.6908715
update 1619402965.7059972
step 0.0
step done 1619402965.7069063
update 1619402965.7219727
step 0.0
step done 1619402965.7228785
update 1619402965.7378814
step 0.0
step done 1619402965.7387946

Did I miss any steps that I should do?

I found the problems:
I also need to add physics_update in my code:

def physics_update(self, step):
    print("physics update step:", step, "seconds")

self.physics_sub = omni.physx._physx.acquire_physx_interface().subscribe_physics_step_events(self.physics_update)

and call kit update like:
self.kit.update(1.0 / 60.0)


Great 👌