4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):
Operating System
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):
GPU Information
Model: GeForce RTX2080
Driver Version: 535.183.01
Topic Description
Detailed Description
When using the robot assembler (tools > robotics > asset editors > robot assembler) with two official assets (a robot and a gripper from the assets browser), the simulation is very unstable.
Steps to Reproduce
Open a new stage.
Add a UR10e universal robot from the official assets browser (Robots/UniversalRobots/ur10e/ur10e.usd).
Add a 2F140 Robotiq gripper from the official assets browser (Robots/Robotiq/2F-140/Robotiq_2F_140_physics_edit.usd).
Open the robot assembler under tools > robotics > asset editors > robot assembler and assemble the robot and the gripper as follows:
base robot: /World/ur10e
attach robot: /World/Robotiq_2F_140_physics_edit
Base robot attach point: /gripper
Attach robot attach point: /robotiq_base_link
Click on assemble
Play the simulation and observe “exploding” behavior (see video attached) where the arm is moving randomly ?!? and the gripper detaches itself.
Error Messages
Just after the problem arises, multiple “Invalid PhysX transform detected”
Screenshots or Videos
Additional Information
What I’ve Tried
I tried different robots (Yaskawa, Staubli) that were converted to USD from URDF file but same behavior arises when i move the robots’s joints with the physics inspector.
can we actually sort this issue out its been a long standing problem and there have been multiple reported problems for robotiq asset with UR or any robot and its a very typical configuration used in labs and even big OEM’s. I’ve reported the same issue in all versions of Isaac sim. I thought this was fixed in 4.5 finally but no, even the beta assets where we have the option to import usd for ur5e with robotiq as option directly doesn’t work let alone robot assembler.
I followed your steps and didn’t reproduce the issue.
Here is the python code from my “Assembly Summary Frame”:
from isaacsim.robot_setup.assembler import RobotAssembler,AssembledRobot
from isaacsim.core.prims import SingleArticulation
import numpy as np
base_robot_path = "/World/ur10e"
attach_robot_path = "/World/Robotiq_2F_140_physics_edit"
base_robot_mount_frame = "/gripper"
attach_robot_mount_frame = "/robotiq_base_link"
fixed_joint_offset = np.array([0.0,0.0,0.0])
fixed_joint_orient = np.array([0.0,0.7071,0.0,0.7071])
single_robot = False
robot_assembler = RobotAssembler()
assembled_robot = robot_assembler.assemble_articulations(
base_robot_path,
attach_robot_path,
base_robot_mount_frame,
attach_robot_mount_frame,
fixed_joint_offset,
fixed_joint_orient,
mask_all_collisions = True,
single_robot=single_robot
)
# The fixed joint in a assembled robot is editable after the fact:
# offset,orient = assembled_robot.get_fixed_joint_transform()
# assembled_robot.set_fixed_joint_transform(np.array([.3,0,0]),np.array([1,0,0,0]))
# And the assembled robot can be disassembled, after which point the AssembledRobot object will no longer function.
# assembled_robot.disassemble()
# Controlling the resulting assembled robot is different depending on the single_robot flag
if single_robot:
# The robots will be considered to be part of a single Articulation at the base robot path
controllable_single_robot = SingleArticulation(base_robot_path)
else:
# The robots are controlled independently from each other
base_robot = SingleArticulation(base_robot_path)
attach_robot = SingleArticulation(attach_robot_path)
Hi thanks for the response, I have few doubts to clarify to verify I’m not doing something wrong before assembling should I ensure both the assets (Robotiq 2F-85) and UR5e have a valid articulation root layer. I’m creating a default stage, Issac Assets beta (Load UR5e (PhysX enabled) drag and drop under World), Choose Robotiq-2F-85-edit drag and drop in World. So right now both have prim paths (World/UR5e) and (World/Robotiq_2F_85_edit/Robotiq_2F_85). Articulation root layer for robotiq is right now at (Robotiq_2F_85_edit/Robotiq_2F_85) and root_joint for UR5e.
In Robot Assembler if I do Base Robot (World/UR5e) and Attach Robot (World/Robotiq_2F_85_edit/Robotiq_2F_85), base robot attach point /gripper. attach robot attach point /base_link, it doesn’t work please correct me If I’m missing something trivial here. Thanks.
And you may need to do some modifications to make it work stably.
Modifications to Make it Work
The first modification you need is to fix the gripper by adding a PhysicsFixedJoint on it. Because the robot assembler performs robot assembling during playing time, a fixed joint prevents it falling to nowhere. Some other hands, e.g. the Allegro hand has a fixed joint on the root and it has no problem.
When Assembling the robot using the assembler, you may notice that the /gripper xform doesn’t connect to any other part. We can change it to /World/ur10e/wrist_3_link to keep the articulation structure
Finally, you can following your previous steps to make a stable assemble. I believe this bug is specifically related to these two assets, but you may encounter more as due to different asset formats.
I noticed that there is an assembled robotiq140 + ur10e combo that might be helpful.
Hi thanks for the response, I think I have precisely tried whatever you suggested and I feel there is something broken in the asset level for this. My problem with this is that if i do any tuning I’m not able to get stable parameters for the position control to work properly. Can you please look at this flattened usd to help me out with the stable limits and parameters for both robot and robotiq 2F85 together. That would be really helpful for my workflow.
Currently the only articulation root layer is in prim /Robot under which you have /Robotiq_2F_85 and /ur10e, I think I more or less copied the parameters for gripper from the already present ROS2 scenario example, however I feel the joint drive limits should be different for this gripper as it’s 2F85 and not 2F140. I followed the Rigging complicated structure tutorial but that led to a velocity control rather than position one which I could readily use with omnigraph.
In IsaacLab as well this would help in doing AssetCfg as people have issues using robot assembler via code. Thanks in advance.
Hi if you are asking if it’s working and tuned properly no, I’m having trouble tuning the gripper for position control to have stable simulation all the time along with the robot. However I can articulate it without physics if that’s what you are asking. But with physics the solver is struggling / I’m clueless about tuning for mimic joints. but I suspect this is only happening in Robotiq 2F85 gripper.
There is no gripper asset as such apart from this to my knowledge, you can bring in your own asset like Onrobot gripper along with the robot from Onshape and try configuring the stiffness/damping. URDF importer with other grippers could also be a possible workflow. In the examples, there’s only situations where the parallel jaw is already part of the robot prim like cobotta900 and franka.
Yeah, that’s a pity we don’t have a plug&play gripper to attach to a robot arm.. I am still struggling with the gripper to have a stable simulation. Like moving the robot arm only (and NOT controlling the gripper) makes the gripper super instable until it “explodes”.
And as suggested by Nvidia team, i did the above-mentioned changes but without success. :/
Hello, I am looking forward if you could find a solution, because I have already tried to export the builtin Robotiq_2F_85 gripper to URDF and it says that it cannot export closed kinematics, so I am willing to hear if there are news concerning the topic.
Hello, just wanted to chime in and see if this issue (involving the robot assembler tool for a UR arm and a gripper and/or physics simulation post assembly) was resolved to any extent?
I am experiencing similar problems to those described, specifically with a UR5 or UR5e arm with a Robotiq 2F 85 gripper. It seems like related issues have been raised before too, such as here and here.
i tested the robot assembler in Isaac Sim 5.0 and is still not working as intended… crazy how such a necessary tool is still not working with the official assets.
If you have been following that procedure, would it be possible for you to share a short video clip of the issue you are encountering in Isaac Sim 5.0? This would be very valuable for our engineering team to see the exact behavior in the new version.
This issue occurs even when using the short gripper attachment available in the asset list. I encountered the problem while trying to attach it to the UR5e.
I’m currently following the importing a manipulator guide to set up a basic pick-and-place task. While I was able to successfully replicate the tutorial with the cobotta_900 robot, I’m now trying to apply the same process to a new manipulator — specifically the UR5e — which requires attaching a different gripper.
Unfortunately, the Robot Assembler fails at this step. I’ve attached a screenshot below showing how it breaks. Any guidance would be appreciated.
