Hello Guys,
I am using isaac sim 4.0.0. I am trying to load human hand into the robot arm, after merging both urdfs into Isaac sim, its loaded nicely. But when i played simulation, its showing like this and robot is randomly moving.
“2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link1.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link2.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link3.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link4.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link5.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link6.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link7.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/panda_link8.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/hand_base_link.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/index_proximal.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/middle_proximal.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/pinky_proximal.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/ring_proximal.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/thumb_proximal_base.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/index_intermediate.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/middle_intermediate.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/pinky_intermediate.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/ring_intermediate.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/thumb_proximal.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/index_tip.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/middle_tip.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/pinky_tip.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/ring_tip.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/thumb_intermediate.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/thumb_distal.
2024-07-22 13:18:11 [73,052ms] [Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/panda/thumb_tip.”
I found similar issues in forum to solve, but everyone mentioned wrong and infinite positions and tranforms are given. I got that one but i cant able to find where should i make changes for this issue.
I tried to solve by using articulation, only base link is articulated. but didn’t work
please help anyone asap,
cheers!
After doing some work, robot is staying still. But if I want to control the joints from omni graph. randomly rotating and in terminal its showing the warning continously. i tried the articulation (exclude articulation) from remaining joints other than base link, but didn’t work out.
Any ideas to solve this one, much appreciated!!
@AlesBorovicka
@rthaker
@Milad-Rakhsha
@philipp.reist
@gyeongeun1
@gyeoman
Hello Guys, can you please respond and help this one, after keeping articulation only for root and some finger joints (bcoz physxmimic is only on articulation joints), I am getting the following error. before i tried with omnigraph for controlling, now thought of keeping robot still, but after doing articulation, it falling on ground like this
“failed to find internal joint object for PhysxMimicJointAPI at /panda/middle_proximal/middle_intermediate_joint. Please ensure that the prim is a supported joint type and is part of an articulation.”
I am sure that these joints are mentioned correctly as revolute joints and i gave prim ( address of the joint also) and i feel that it is a part of articulation, like floating base joint.
I am stuck here for few days, trying all the possible ways, but nothing is working out.
Please help me in this issue
Hi @dekusmash016 ,
The most likely issue is that the reference joint is not part of an articulation.
Can you either try the articulation inspector or omni pvd to see what the parsing issue may be?
Hi @philipp.reist
Thanks alot for your response. I think reference joint and 1st one is a part of articulation. I tried articulation inspector and got the following results, in terminal some warning,
Hi @philipp.reist
Now, I tried Pvd also in the extension manager.
it is showing the previous error, continously:- for all the links
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/panda_link1.
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/panda_link2.
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/panda_link3.
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/panda_link4.
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/panda_link5.
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/panda_link6.
…
[Warning] [omni.physx.plugin] Invalid PhysX transform detected for /World/hand_dex/thumb_tip.
and then after some time in articulation manager it is showing naN values and then crashed the sim.
How to solve this problem?
Could you confirm with the PVD capture that the reference joint is indeed part of the articulation?
yeah sure @philipp.reist, now i tried only pvd, not articulation manager. After played simulation warnings and crashed.
and one more thing 1st joint is panda_link0, and its joint to panda_link1 is root_joint (its the articulation root), in the terminal it shows from panda_link1 to fingertip. So, we can say that root_joint is considered as reference joint and didn’t show up.
From the warnings it looks like the sim explodes/goes NaN - can you share the OVD capture with me please? You can send it directly to me if you don’t want to post publicly.
Hi @philipp.reist,
I have sent the files directly to you. Can you please let me know after checking.
Thank you
Ok I checked your latest new0 and new1 captures - they look fine and do not explode? So what did you change?
Hi
In “Physics Debug” window, if I use solver type : Force PGS. But when I load recent ovd file for me is showing like this, no robot. after that even if I wanted to control robot either from omnigraph or articulation inspector, it is exploding.
why is it behaving like that
Thanks @philipp.reist
Try finding it in the OmniPVD object tree, then select the articulation and press F. OmniPVD zooms out to show all, so with a large ground plane the objects on it vanish in the viewport
yeah okay, I can able to do that.
Is the problem solved ? Now I tried once again controlling the robot joints with omni graph, but for fingers if i cross some limit, entire robot is randomly rotating. can you please how to solve and what is root cause for this problem. and which is the best method to control the robot in simulation.
Thanks @philipp.reist
How are you controlling the robot? Any chance you are using very stiff joint drives? We found that very stiff drives are not behaving well in the presence of mimic joints. We wrote more information here: Articulations — physx 5.4.1 documentation
1 Like
Good Morning @philipp.reist ,
And regarding question, I’m using omnigraph > articulation position controller. Graph will come, we can change joint angles to move the robot and one more method is articulation inspector, after playing simulation, we can change joint angles in gui to control the robot.
I just go through the documentation and I am thinking the isuue might be because of fingers mimic joint and its joint drive. and also in sim, I saw by default joint drive stiffness is very high for all the joints and I tried to change low for fingers like 10 and same for arm, and both low but still. It didn’t solve the problem.
I wanted to share the urdf I am using, franka arm and inspire hand for reference. just import this urdf into your sim and see, what is the issue.
sodapdf-converted.pdf (12.5 KB)
Thanks @philipp.reist
Hi
In the mean time, I tried adding gripper to the franka arm and tried to control with omnigraph, it working perfectly without any errors. Then I tried to control the hand alone with omnigraph by changing the joint angles, its throwing an errror. Now I am thinking the issue is fingers mimic joint and its joint drive, which I mentioned above. Its not with mimic but with finger mimic joint and joint drive, bcoz in gripper also we have mimic joint and its working fine.
I think we found where the error is, I need some assistance from you solve this one. I tried shunk hand also, same behavior. Can you tell how to control the hand and finger joints in issac sim.
Thanks @philipp.reist
Ok yes, it’s important that the joint position drive gains are not too high for the simulation to remain stable. Please try lower stiffness values for the drives and see if that helps.
How do you mean how to control the hand and finger joints? If you do RL, you should use one of the view classes. It’s important that you use position drives and don’t just set the joint target positions.
1 Like
Hi
Thanks for your message. and Sorry, correct me if I am wrong, “joint position drive gains are not too high” is drive gains means gain tuner, here for few joints, period is showing infinty.
I tried to change the period values, but not changing.
and regarding stiffness, there are 2 stiffness (drive and joint). I think we should focus on drive angular stiffness only, but I already mentioned in last message that I tried keeping stiffness of 10 for all joints and 10 only for finger joints - max for arm joints. But still the behavior is same.
for controlling the hand fingers, now I am controlling it with omnigraph (is it okay for controlling?) . now I want to make sure everything is imported perfectly in sim and setup everything in sim, then I wanted to start RL.
now how to solve this one ??
Thanks @philipp.reist
Hi @philipp.reist
Good morning, I tried adding fixed tendon root axis and fixed tendon axis for proximal joint and distal joint to check is that will work. But it’s not working.
I understand that joint drive stiffness should be low I tried with 10,100,1000. Which one is ideal ? And regarding drive gain how should I change it there, I can’t able to change it in gui.
For controlling hand, now only in Isaac sim. Is omnigraph enough or any method is there for controlling the robot.
What should I do now?
Thanks @philipp.reist
