Isaac Sim, Moveit

[abderrahmen.melliti]

Hey,
I’m running Isaac Sim and Moveit in the same docker on AWS, I followed this tutorial for making the communication between them but I got this error and I don’t know how to fix it.

GitHub - Toni-SM/omni.add_on.ros_control_bridge: ROS Control Bridge for...

Please can someone help me?
[ INFO] [1636322044.476872816, 2770.458737763]: Loading robot model ‘panda’…
[ WARN] [1636322044.476985524, 2770.458737763]: Link panda_leftfinger has visual geometry but no collision geometry. Collision geometry will be left empty. Fix your URDF file by explicitly specifying collision geometry.
[ WARN] [1636322044.477015351, 2770.458737763]: Link panda_rightfinger has visual geometry but no collision geometry. Collision geometry will be left empty. Fix your URDF file by explicitly specifying collision geometry.
[ INFO] [1636322053.945177205, 2779.926480925]: Starting planning scene monitor
[ INFO] [1636322053.946956371, 2779.926480925]: Listening to ‘/move_group/monitored_planning_scene’
[ INFO] [1636322053.962372539, 2779.943207925]: No active joints or end effectors found for group ‘hand’. Make sure that kinematics.yaml is loaded in this node’s namespace.
[ INFO] [1636322053.962999789, 2779.943207925]: No active joints or end effectors found for group ‘hand’. Make sure that kinematics.yaml is loaded in this node’s namespace.
[ INFO] [1636322053.970249581, 2779.943207925]: Constructing new MoveGroup connection for group ‘hand’ in namespace ‘’
[ INFO] [1636322054.944442196, 2780.930092946]: Ready to take commands for planning group hand.
[ INFO] [1636322065.839571803, 2791.819731118]: Constructing new MoveGroup connection for group ‘panda_arm’ in namespace ‘’
[ INFO] [1636322065.858431871, 2791.836459119]: Ready to take commands for planning group panda_arm.
[ INFO] [1636322147.758169262, 2873.734975096]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.
[ INFO] [1636322147.758329891, 2873.734975096]: Planning attempt 1 of at most 1
[ INFO] [1636322147.758640039, 2873.734975096]: Found a contact between ‘panda_link5’ (type ‘Robot link’) and ‘panda_hand’ (type ‘Robot link’), which constitutes a collision. Contact information is not stored.
[ INFO] [1636322147.758668974, 2873.734975096]: Collision checking is considered complete (collision was found and 0 contacts are stored)
[ INFO] [1636322147.758699489, 2873.734975096]: Start state appears to be in collision with respect to group panda_arm
[ WARN] [1636322147.778724573, 2873.751702096]: Unable to find a valid state nearby the start state (using jiggle fraction of 0.050000 and 100 sampling attempts). Passing the original planning request to the planner.
[ INFO] [1636322147.779951341, 2873.751702096]: Planner configuration ‘panda_arm’ will use planner ‘geometric::RRTConnect’. Additional configuration parameters will be set when the planner is constructed.
[ WARN] [1636322147.780566147, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ WARN] [1636322147.780621116, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ WARN] [1636322147.780680531, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ WARN] [1636322147.780738985, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ERROR] [1636322147.780803614, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ERROR] [1636322147.780848273, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ERROR] [1636322147.780915035, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ERROR] [1636322147.780966174, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ WARN] [1636322147.781037485, 2873.751702096]: ParallelPlan::solve(): Unable to find solution by any of the threads in 0.000751 seconds
[ WARN] [1636322147.781183259, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ERROR] [1636322147.781223551, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ WARN] [1636322147.781258607, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ERROR] [1636322147.781307756, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ WARN] [1636322147.781390161, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ WARN] [1636322147.781425168, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ERROR] [1636322147.781460079, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ERROR] [1636322147.781496587, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ WARN] [1636322147.781533987, 2873.751702096]: ParallelPlan::solve(): Unable to find solution by any of the threads in 0.000407 seconds
[ WARN] [1636322147.781646084, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ WARN] [1636322147.781697189, 2873.751702096]: panda_arm/panda_arm: Skipping invalid start state (invalid state)
[ERROR] [1636322147.781743120, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ERROR] [1636322147.781784203, 2873.751702096]: panda_arm/panda_arm: Motion planning start tree could not be initialized!
[ WARN] [1636322147.781823542, 2873.751702096]: ParallelPlan::solve(): Unable to find solution by any of the threads in 0.000235 seconds
[ WARN] [1636322147.789653094, 2873.768433096]: Goal sampling thread never did any work.
[ INFO] [1636322147.789754522, 2873.768433096]: Unable to solve the planning problem
[ INFO] [1636322152.579504850, 2878.552427163]: ABORTED: No motion plan found. No execution attempted

[toni.sm]

Hi @abderrahmen.melliti

The omni.add_on.ros_control_bridge user extension configures Isaac Sim to use the ROS FollowJointTrajectory and GripperCommand actions.

There are a few steps to configure and use such an extension

1. Include dependencies (omni.usd.schema.add_on and omni.add_on.ros_bridge_ui)

2. Publish the joint states of the robot using a RosJointState prim

3. Configure the RosControlFollowJointTrajectory and/or RosControlGripperCommand prims according to the configuration of the ROS controllers made with MoveIt Setup Assistant (ros_controllers.yaml file)

Can you see (rostopic list) the following (or similar) topics only playing the simulator, without launching MoveIt?

/arm_controller/follow_joint_trajectory/cancel
/arm_controller/follow_joint_trajectory/feedback
/arm_controller/follow_joint_trajectory/goal
/arm_controller/follow_joint_trajectory/result
/arm_controller/follow_joint_trajectory/status

Could you please indicate which step is giving you problems or indicate more details about the steps followed (even if not indicated)?

In any case, I added an example (an stage (.usd) and a catkin workspace with the MoveIt configuration) to the repository: Releases · Toni-SM/omni.add_on.ros_control_bridge · GitHub

[abderrahmen.melliti]

sorry for the late i was focusing in other topics, i also face the new problems, before, I unzipped the three plugins omni.usd.schema.add_on and omni.add_on.ros_control_bridge and omni.add_on.ros_bridge_ui in a folder, and every time i want to try to do the communication I copied and put it in the exts folder why I did that because every time I restart the Isaac sim I cannot find them and i don’t know why but I was successful to enable it using window->extention, this time I repeated the same way but in every time I search for the plugin I can not find them, you can see it below in the pictures, for the rest of steps for communication I tried to do the simplest solution I download the example in the link that you provided it to me ‘Releases · Toni-SM/omni.add_on.ros_control_bridge · GitHub’ but I think first I need to solve the problem of the plugin.

[toni.sm]

Hi @abderrahmen.melliti

New Isaac Sim version (2021.2.0) supports user extensions installations from GitHub directly…
Then, you can add the extensions (from the Extensions window) to the Extension Search Path as follow:

• git+https://github.com/Toni-SM/omni.usd.schema.add_on.git?branch=main&dir=exts

• git+https://github.com/Toni-SM/omni.add_on.ros_bridge_ui.git?branch=main&dir=exts

• git+https://github.com/Toni-SM/omni.add_on.ros_control_bridge.git?branch=main&dir=exts

[abderrahmen.melliti]

@toni.sm unfortunately, I don’t have this option I’m using the previous version, I tried again, I put the three plugins in the exts folder but it’s like Isaac sim doesn’t read them and I used the import extension also from window but it doesn’t work. Do you have an idea how to solve it?

[toni.sm]

Hi @abderrahmen.melliti

Did you kill and run Isaac Sim (./isaac-sim.headless.kitremote.sh --allow-root) after copy the extensions to /isaac-sim/exts folder?

[abderrahmen.melliti]

@toni.sm no, actually I have no idea about that.

[abderrahmen.melliti]

Hi @toni.sm
I downloaded the new version and i tried this solution to put the link in the Extention Search Path and also i tried the old one which is to copy the plugins in the exts folder but its failed , could you please help me ?

[ltorabi]

Toni, is on vacation, will be back in a week

[ltorabi]

@toni.sm

[toni.sm]

We have already been in contact… 😅

[09ubberboy90]

@abderrahmen.melliti

[ INFO] [1636322147.758640039, 2873.734975096]: Found a contact between ‘panda_link5’ (type ‘Robot link’) and ‘panda_hand’ (type ‘Robot link’), which constitutes a collision. Contact information is not stored.

Hi, I don’t know if you still have this problem but this error is due to the default position of panda that is wrong. You can either disable self collide (Not sure how to do that in isaac) or you can edit the panda.srdf file that is included in the launch file with this lines

  <disable_collisions link1="panda_link5" link2="panda_hand" reason="Adjacent"/>
  <disable_collisions link1="panda_link5" link2="panda_rightfinger" reason="Adjacent"/>
  <disable_collisions link1="panda_link5" link2="panda_leftfinger" reason="Adjacent"/>

I know that even if everything else was working I would still get failure to move the robot until i fixed that

[abderrahmen.melliti]

Thank you so much I will try it for sure, now I have the problem with the plugins I can not download them and use them.

[09ubberboy90]

I can’t really help you with that as it is working fine for me using github and Isaac 2021.1.2 . What version of Isaac are you using ?

[abderrahmen.melliti]

Thank you, i’m using the version Isaac 2021.1.1 but now its work I was able to put the plugins in the exts folder and i will try your solution but first, i have to simulate the Yaskawa robot with Isaac , thank you again.