Lula Kinematics Solver on Custom Made Robot FAIL, tested using Lula Widget

[Hari_Krsna]

Version

|> 4.2.0 |> Ubuntu 22.04 |> RTX 3080 |> 546.80

★ Lula Kinematics Solver on Custom made robot FAIL, testing using Lula Widget

I’m working on a custom robotic arm in Isaac Sim, integrated with the Lula Kinematics Solver to compute inverse kinematics (IK). While I have set up the URDF, USD, and XRDF files correctly with matching articulation paths, joint names, link names, and joint limits, the solver is consistently failing to compute an IK solution.

The robot model is properly loaded, and the IK target is positioned within the robot’s workspace. However, the solver repeatedly fails, returning errors about the inability to find IK solutions and NoneType attribute errors related to joint access.

★ Goal of Project

My immediate goal is to make the end effector of the robotic arm move to a specified point using inverse kinematics. This is a preliminary step for future pick-and-place tasks, where I plan to use a surface gripper to handle objects.

★ Error Messages

Error 1: when chicking follow target

Error 2: When clicking follow Trajectory / Waypoints

★ Video

★ Colliders : No issue

★ USD Stage

★ What I’ve Tried

• Rechecked all names for joints, links, and other components — everything is correct and matching.
• Created the CAD file using a CAD download website. Converted it from STEP > Fusion 360 > added joints > exported as .fbx > imported into Isaac Sim, which was converted to USD.
• Verified colliders are set up correctly, and there are no PhyX errors.

★ Attatched files below (all .txt)

1. mra_articulated_urdf - Robot URDF (made using fusion 360 extension)
2. USD_Prim_Names - Current USD Prim Paths/Parts
3. mra_lulaXRDF - Lula Robot Descprition File

★ What can I do ?

• Any tips on debugging the IK solver or tracing articulation issues?
• What other steps or tools should I try to fix this in Isaac Sim?
• Are there any common mistakes or improvements you suggest for solving this problem?

mra_articulated_urdf.txt (10.0 KB)
mra_lulaXRDF.txt (998 Bytes)
USD_Prim_Names.txt (3.9 KB)

[Hari_Krsna]

Issue Resolved :)

Common Mistakes to Avoid When Using Lula: (My Experience)

1. It’s often best to download a URDF dataset for any robotic arm from an official website. Alternatively, look for Nvidia Isaac Sim URDFs in isaac.omni.motion_generation and inside the motion_configs folder, where you’ll find many URDF files.

2. If your CAD model is prebuilt or you are modifying a robotic arm, it’s always better to convert the CAD model to URDF and then use the URDF importer in Isaac. A common mistake I made was exporting the robotic arm as an FBX file and then importing it using the basic import option. This resulted in the arm being imported in USD but through FBX format but not URDF, and I had to manually add joints, which created numerous issues. When working with kinematics solvers, ensure you have a robust, error-free URDF.

3. The best resource I found for URDFs is [GitHub repository - URDF Datasets](GitHub - Daniella1/urdf_files_dataset).
   I downloaded a Kuka model from there and successfully implemented the Lula Kinematics Solver (worked 100%)

4. Never forget to use the Loaded Scenario Template Extension as it is crucial for Lula and other designs using URDF.
   Without this template, it won’t work.

5. If you have any fixed joints or other joints that are not part of an articulation, go to those joints and
   click Exclude from Articulation.

Long Story Short

1. Follow the tutorial exactly* as described in the documentation: Motion Generation with Lula Kinematics

2. Download the Lula Kinematics Template Zip file, paste it inside the exts/ folder, and your extension will be loaded.

3. Simply replace the URDF, XRDF, and USD files of your robot (when you import a URDF model, a USD file will be generated in the same folder—take a look and use that usd_path in this code and run it. It will work—100% guaranteed.

4. The biggest challenge when implementing Lula algorithms is the URDF file management 🫡 hence be carefull with URDF*