SO-ARM101 EE Frame Alignment Issue (Fixed Jaw Causes Collision) – Best Strategy in Isaac Sim with Lula IK?

Omniverse Isaac Sim
1

Important: Isaac Sim support

Note: For Isaac Sim support, the community is gradually transitioning from this forum to the Isaac Sim GitHub repository so that questions and issues can be tracked, searched, and resolved more efficiently in one place. Whenever possible, please create a GitHub Discussion or Issue there instead of starting a new forum topic.

Note: For any Isaac Lab topics, please submit your topic to its GitHub repo ( GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim · GitHub ) following the instructions provided on Isaac Lab’s Contributing Guidelines ( Contribution Guidelines — Isaac Lab Documentation ).

Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.

6.0.0
5.1.0
5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 24.04
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: Geforce RTX 5080
  • Driver Version:

Topic Description

Detailed Description

Title: SO-ARM101 EE Frame Alignment Issue (Fixed Jaw Causes Collision) – Best Strategy in Isaac Sim with Lula IK?

Hi everyone,

I’m working with the SO-ARM101 manipulator in Isaac Sim and running into an issue related to the end-effector (EE) frame definition and how it affects grasp execution.

Problem Description:
The EE frame appears to be defined at the tip of the fixed jaw rather than at the center of the gripper (i.e., between the jaws). Because of this:

  • When I solve IK using Lula IK, the solver tries to bring the fixed jaw tip to the target pose.

  • During grasping, this causes the fixed jaw to align with the center of the object, instead of the gripper center.

  • As a result, the fixed jaw collides with the object before a proper grasp can occur.

Current Setup:

  • Simulator: Isaac Sim

  • IK Solver: Lula IK

  • Task: Pick-and-place / grasping

  • Input: Target pose (object pose or grasp pose)

  • EE frame: Default frame attached to fixed jaw tip

Observed Behavior:

  • IK converges successfully to the given target.

  • However, execution leads to collision because the EE reference is not aligned with the intended grasp center.

Questions / Guidance Needed:

  1. Best Practice for EE Frame Definition:

    • Should the EE frame be redefined to the center between the jaws instead of the fixed jaw tip?

    • If yes, what is the recommended way to modify this in Isaac Sim (USD edit vs. runtime transform)?

  2. Offset Strategy:

    • Is it better to:

      • Apply a pre-grasp offset to the target pose before feeding it to Lula IK, OR

      • Redefine the EE frame entirely so that IK naturally aligns with the grasp center?

  3. Grasping Pipeline Consideration:

    • For robust grasping (especially future cases with random objects), what is the preferred approach:

      • IK to a grasp pose defined in gripper-center frame, OR

      • IK to a proxy frame + motion refinement?

  4. Lula IK Constraints:

    • Does Lula IK assume the EE frame is the true tool center point (TCP)?

    • Any recommended way to handle TCP offsets cleanly within Lula?

What I’m Trying to Achieve:
A stable grasping pipeline where:

  • The gripper approaches the object without collision

  • The grasp is centered properly

  • The solution generalizes to arbitrary object shapes (future goal: random objects / perception-driven grasping)

If anyone has dealt with similar EE frame misalignment or has a recommended architecture for grasping in Isaac Sim (especially with Lula IK), I’d really appreciate your guidance.

Thanks in advance!

2

Hi @pathik2595,

This is expected behavior – Lula IK places whatever frame you specify at the target pose, and if that frame is at the fixed jaw tip, that’s where the target will go.

The recommended fix is to add a virtual TCP link in your URDF at the center between the jaws. This is exactly how the Franka (FR3) robot handles it in Isaac Sim – it defines a gripper_center link:

<link name="grasp_center"/>
<joint name="grasp_center_joint" type="fixed">
  <origin rpy="0 0 0" xyz="0 [Y_OFFSET] 0"/>
  <parent link="[your_fixed_jaw_link]"/>
  <child link="grasp_center"/>
</joint>

Where [Y_OFFSET] is half the jaw gap (the distance from the fixed jaw surface to the midpoint between jaws). This is an empty link – no geometry or mass needed.

After re-importing the URDF (or editing the USD to add this fixed joint + Xform prim), pass "grasp_center" as the frame_name to LulaKinematicsSolver.compute_inverse_kinematics(). It will then appear in solver.get_all_frame_names().

Why this approach over runtime offsets:

  • IK solutions are physically meaningful (the TCP IS the grasp point)
  • No need to rotate offsets into the correct frame for different approach angles
  • RMPFlow and other planners can use the same frame definition
  • Generalizes cleanly to perception-driven grasping

If you can’t modify the URDF, you can apply the offset at runtime:

import numpy as np
from scipy.spatial.transform import Rotation

def offset_target_for_ik(target_pos, target_quat_wxyz, tcp_offset_local):
    """Shift target from desired TCP pose to actual EE frame for IK."""
    r = Rotation.from_quat(target_quat_wxyz[[1,2,3,0]])  # to xyzw
    return target_pos - r.apply(tcp_offset_local), target_quat_wxyz

# Measure tcp_offset_local = vector from your current EE frame to grasp center,
# expressed in the EE frame's local coordinate system

To answer your Q4: Lula has no built-in TCP offset parameter. The frame_name argument IS the TCP. The clean way to handle offsets is through the kinematic chain itself (the virtual link approach above).

3

Hi @pathik2595, we wanted to follow up on this thread. We provided guidance recommending you add a virtual grasp_center link in your URDF (similar to the Franka FR3 approach) to properly align the EE frame for Lula IK. Since we haven’t heard back in 14 days, we’ll close this topic. If you’re still experiencing issues or have follow-up questions, please open a new topic and reference this one: SO-ARM101 EE Frame Alignment Issue (Fixed Jaw Causes Collision) – Best Strategy in Isaac Sim with Lula IK?. Thank you!