Advanced Tutorials. 4. Adding a New Manipulator problems

[j.s.p.c]

Hey there,
I am following the New Manipulator tutorial and I have one question and one issue.
https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/tutorial_advanced_adding_new_manipulator.html

1st. The robot i want to add consist in a surface gripper instead of parallel gripper. So in the ‘gripper_control’ I import SurfaceGripper, and I define it as

gripper = SurfaceGripper(
    end_effector_prim_path= "/World/IRB1100/end_facttor",
    direction = "x",

my_denso = my_world.scene.add(SingleManipulator(prim_path="/World/IRB1100", name="irb_1100",
                                                end_effector_prim_name="end_facttor", gripper=gripper))
)

From here, the “apply_action” does not work with the surface gripper but i read the methods in SurfaceGripper and i try to initialize it with

while simulation_app.is_running():
    my_world.step(render=True)
    if my_world.is_playing():
        if my_world.current_time_step_index == 0:
            my_world.reset()
            my_denso.gripper.initialize(root_prim_path= "/World/IRB1100/end_facttor")
        i += 1
        my_denso.gripper.update()
        if i == 2: 
            print("Start")
            my_denso.gripper.close()
        if i == 500:
            my_denso.gripper.open()
            
        if i == 1000:
            

It says [Warning] [omni.isaac.manipulators.grippers.surface_gripper] gripper didn’t close successfully in all the gripper actions.

I try looking at the universal robots as reference but I am not able to manage to do it. I am defining or initializing it wrong?

---

2nd. Following the setting up of the robot, https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/tutorial_advanced_adding_new_manipulator.html#adding-a-follow-target-task, in the point 6, the “ik_solver.py” gives me the following error (when trying with my robot and with cobotta example) when i run the final “follow_target_example.py”:

Cobbota

Traceback (most recent call last):
  File "/home/joanp@mijlweg.visser.eu/Desktop/manipulators/Cobota/Cobotta_scripts_path/follow_target_example.py", line 23, in <module>
    my_controller = KinematicsSolver(my_denso)
  File "/home/joanp@mijlweg.visser.eu/Desktop/manipulators/Cobota/Cobotta_scripts_path/ik_solver.py", line 11, in __init__
    urdf_path="/home/joanp@mijlweg.visser.eu/Desktop/manipulators/Cobota/Robota_urdf/cobotta_pro_900.urdf",
  File "/home/joanp@mijlweg.visser.eu/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/lula/kinematics.py", line 33, in __init__
    self._robot_description = lula.load_robot(robot_description_path, urdf_path)
RuntimeError: bad file
./python.sh: line 40: 126453 Segmentation fault      (core dumped) $python_exe "$@" $args
There was an error running python

My Robot

Traceback (most recent call last):
  File "/home/joanp@mijlweg.visser.eu/Desktop/manipulators/IRB/IRB1100_path/IRB1100_PATH_SCRIPTS/follow_target_example.py", line 24, in <module>
    my_controller = KinematicsSolver(irb1100)
  File "/home/joanp@mijlweg.visser.eu/Desktop/manipulators/IRB/IRB1100_path/IRB1100_PATH_SCRIPTS/ik_solver.py", line 10, in __init__
    urdf_path="/home/joanp@mijlweg.visser.eu/Desktop/manipulators/IRB/IRB1100_path/IRB1100_4_580_2/urdf/IRB1100_4_580_2.urdf")
  File "/home/joanp@mijlweg.visser.eu/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.isaac.motion_generation/omni/isaac/motion_generation/lula/kinematics.py", line 33, in __init__
    self._robot_description = lula.load_robot(robot_description_path, urdf_path)
RuntimeError: bad file
./python.sh: line 40: 129899 Segmentation fault      (core dumped) $python_exe "$@" $args
There was an error running python

It seems to be related to the “urdf_path” but I cannot understand it.

Cobotta ik_solver.py

from omni.isaac.motion_generation import ArticulationKinematicsSolver, LulaKinematicsSolver
from omni.isaac.core.articulations import Articulation
from typing import Optional


class KinematicsSolver(ArticulationKinematicsSolver):
    def __init__(self, robot_articulation: Articulation, end_effector_frame_name: Optional[str] = None) -> None:
        #TODO: change the config path
        self._kinematics = LulaKinematicsSolver(
            robot_description_path="Cobotta_scripts_path/rmpflow/robot_descriptor.yaml",
            urdf_path="/home/joanp@mijlweg.visser.eu/Desktop/manipulators/Cobota/Robota_urdf/cobotta_pro_900.urdf",
            )
        if end_effector_frame_name is None:
            end_effector_frame_name = "onrobot_rg6_base_link"
        ArticulationKinematicsSolver.__init__(self, robot_articulation, self._kinematics, end_effector_frame_name)
        return

My Robot ik_solver.py

from omni.isaac.motion_generation import ArticulationKinematicsSolver, LulaKinematicsSolver
from omni.isaac.core.articulations import Articulation
from typing import Optional


class KinematicsSolver(ArticulationKinematicsSolver):
    def __init__(self, robot_articulation: Articulation, end_effector_frame_name: Optional[str] = None) -> None:
        #TODO: change the config path
        self._kinematics = LulaKinematicsSolver(robot_description_path="IRB1100_path/rmpflow/robot_descriptor.yaml",
                                                urdf_path="/home/joanp@mijlweg.visser.eu/Desktop/manipulators/IRB/IRB1100_path/IRB1100_4_580_2/urdf/IRB1100_4_580_2.urdf")
        if end_effector_frame_name is None:
            end_effector_frame_name = "end_facttor"
        ArticulationKinematicsSolver.__init__(self, robot_articulation, self._kinematics, end_effector_frame_name)
        return

---

Here is the files
manipulators.zip (25.7 MB)

I hope that my doubts are properly explained and thank you so much in advanced for the help!

[BuckBabich]

Starting with your second problem first, I’m not sure why the cobotta example is giving you trouble, but in your robot_descriptor.yaml for the IRB, could you try changing “root_link: world” to “root_link: base_link”? The specified frame must exist in the URDF.

Sorry that it’s failing with a segfault. We’ll be improving the error reporting for this particular mismatch in the next release of Isaac Sim.

[oahmed]

Concerning the first problem. Can you try tuning the surface gripper parameters to be compatible with your asset. The parameters can be set on initialization of the surface gripper. Specifically you can find the parameters below with their corresponding default values.
translate: float = 0,
direction: str = “x”,
grip_threshold: float = 0.01,
force_limit: float = 1.0e6,
torque_limit: float = 1.0e4,
bend_angle: float = np.pi / 24,
kp: float = 1.0e2,
kd: float = 1.0e2,

If this doesn’t work, you can provide us with the robot asset and a minimal repro of the problem so we can guide you better. Thanks!

[j.s.p.c]

Thank you so much. Now both are working.

It was the “root_link” of both robot_descriptor but also in the “ik_solver.py” when defining “robot_description_path” and “urdf_path” both paths have to be full path and starting with an / if not it did not work (at least in my case).

self._kinematics = LulaKinematicsSolver(
            robot_description_path="/home/joanp@mijlweg.visser.eu/Desktop/manipulators/Cobota/Cobotta_scripts_path/rmpflow/robot_descriptor.yaml",
            urdf_path="/home/joanp@mijlweg.visser.eu/Desktop/manipulators/Cobota/Robota_urdf/cobotta_pro_900.urdf"
            )

[j.s.p.c]

Concerning to this, it still does not work :(

Here it is the usd file of the robot: IRB1100_4_580_2.usd (42.1 MB)
And here is the whole folder: IRB1100_path.zip (22.9 MB)

As a quick explanation. The robot already has the gripper attached. I also assembled it in a way the local position of the gripper its in the point of the gripper.

Now in the code, I have added the parameters as you said. I have played with all the possibilities and the same error when defining gripper appear.

gripper = SurfaceGripper(
    end_effector_prim_path= "/World/IRB1100/end_facttor",
    translate = -0.01,
    direction = "y",
    grip_threshold = 0.05,
    force_limit = 2.0e6,
    torque_limit = 3.0e4,
    bend_angle = np.pi / 24,
    kp = 1.8e2,
    kd = 1.7e2,
    )

And consequently, the following error appears:
[241,810ms] [Warning] [omni.isaac.manipulators.grippers.surface_gripper] gripper didn't close successfully

When i set ‘direction = ‘x’’ and ‘translation =0’ o close to 0 then appears the following when trying to close the gripper with irb1100.gripper.forward('close'). But first anyways appears the previous error.

[Warning] [omni.isaac.manipulators.grippers.surface_gripper] Surface Gripper is inside the parent mesh. please move it outside to be able to use the Gripper.

Thank you so much Ahmed.

[oahmed]

Hi Posada,

After looking at your scripts, the warnings you are seeing are expected. When calling gripper.close/open the surface gripper should be positioned close to an object that it is trying to grip. Thats why a warning is printed because a surface is expected to be there and the gripper is not closing on any surface. Therefore, it prints that the gripper didn’t close successfully.

Please let us know if you have more problems.