# Isaac Sim, Moveit

**URL:** <https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297>\
**Category:** Isaac Sim\
**Tags:** ros\
**Created:** [November 7, 2021, 10:09pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297 "2021-11-07T22:09:05Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [November 7, 2021, 10:09pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/1 "2021-11-07T22:09:05Z")

</div>

Hey,  
I’m running Isaac Sim and Moveit in the same docker on AWS, I followed this tutorial for making the communication between them but I got this error and I don’t know how to fix it.

> **[GitHub - Toni-SM/omni.add\_on.ros\_control\_bridge: ROS Control Bridge for NVIDIA Omniverse Isaac Sim...](https://github.com/Toni-SM/omni.add_on.ros_control_bridge)**
>
> ROS Control Bridge for NVIDIA Omniverse Isaac Sim (Low Level Controllers for MoveIt)

Please can someone help me?  
[INFO] [1636322044.476872816, 2770.458737763]: Loading robot model ‘panda’…  
[WARN] [1636322044.476985524, 2770.458737763]: Link panda\_leftfinger has visual geometry but no collision geometry. Collision geometry will be left empty. Fix your URDF file by explicitly specifying collision geometry.  
[WARN] [1636322044.477015351, 2770.458737763]: Link panda\_rightfinger has visual geometry but no collision geometry. Collision geometry will be left empty. Fix your URDF file by explicitly specifying collision geometry.  
[INFO] [1636322053.945177205, 2779.926480925]: Starting planning scene monitor  
[INFO] [1636322053.946956371, 2779.926480925]: Listening to ‘/move\_group/monitored\_planning\_scene’  
[INFO] [1636322053.962372539, 2779.943207925]: No active joints or end effectors found for group ‘hand’. Make sure that kinematics.yaml is loaded in this node’s namespace.  
[INFO] [1636322053.962999789, 2779.943207925]: No active joints or end effectors found for group ‘hand’. Make sure that kinematics.yaml is loaded in this node’s namespace.  
[INFO] [1636322053.970249581, 2779.943207925]: Constructing new MoveGroup connection for group ‘hand’ in namespace ‘’  
[INFO] [1636322054.944442196, 2780.930092946]: Ready to take commands for planning group hand.  
[INFO] [1636322065.839571803, 2791.819731118]: Constructing new MoveGroup connection for group ‘panda\_arm’ in namespace ‘’  
[INFO] [1636322065.858431871, 2791.836459119]: Ready to take commands for planning group panda\_arm.  
[INFO] [1636322147.758169262, 2873.734975096]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.  
[INFO] [1636322147.758329891, 2873.734975096]: Planning attempt 1 of at most 1  
[INFO] [1636322147.758640039, 2873.734975096]: Found a contact between ‘panda\_link5’ (type ‘Robot link’) and ‘panda\_hand’ (type ‘Robot link’), which constitutes a collision. Contact information is not stored.  
[INFO] [1636322147.758668974, 2873.734975096]: Collision checking is considered complete (collision was found and 0 contacts are stored)  
[INFO] [1636322147.758699489, 2873.734975096]: Start state appears to be in collision with respect to group panda\_arm  
[WARN] [1636322147.778724573, 2873.751702096]: Unable to find a valid state nearby the start state (using jiggle fraction of 0.050000 and 100 sampling attempts). Passing the original planning request to the planner.  
[INFO] [1636322147.779951341, 2873.751702096]: Planner configuration ‘panda\_arm’ will use planner ‘geometric::RRTConnect’. Additional configuration parameters will be set when the planner is constructed.  
[WARN] [1636322147.780566147, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[WARN] [1636322147.780621116, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[WARN] [1636322147.780680531, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[WARN] [1636322147.780738985, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[ERROR] [1636322147.780803614, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[ERROR] [1636322147.780848273, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[ERROR] [1636322147.780915035, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[ERROR] [1636322147.780966174, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[WARN] [1636322147.781037485, 2873.751702096]: ParallelPlan::solve(): Unable to find solution by any of the threads in 0.000751 seconds  
[WARN] [1636322147.781183259, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[ERROR] [1636322147.781223551, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[WARN] [1636322147.781258607, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[ERROR] [1636322147.781307756, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[WARN] [1636322147.781390161, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[WARN] [1636322147.781425168, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[ERROR] [1636322147.781460079, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[ERROR] [1636322147.781496587, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[WARN] [1636322147.781533987, 2873.751702096]: ParallelPlan::solve(): Unable to find solution by any of the threads in 0.000407 seconds  
[WARN] [1636322147.781646084, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[WARN] [1636322147.781697189, 2873.751702096]: panda\_arm/panda\_arm: Skipping invalid start state (invalid state)  
[ERROR] [1636322147.781743120, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[ERROR] [1636322147.781784203, 2873.751702096]: panda\_arm/panda\_arm: Motion planning start tree could not be initialized!  
[WARN] [1636322147.781823542, 2873.751702096]: ParallelPlan::solve(): Unable to find solution by any of the threads in 0.000235 seconds  
[WARN] [1636322147.789653094, 2873.768433096]: Goal sampling thread never did any work.  
[INFO] [1636322147.789754522, 2873.768433096]: Unable to solve the planning problem  
[INFO] [1636322152.579504850, 2878.552427163]: ABORTED: No motion plan found. No execution attempted

---

<div class="post-metadata">

**Author:** ![toni.sm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/toni.sm/32/27559_2.png) [@toni.sm](https://forums.developer.nvidia.com/u/toni.sm)\
**Post date:** [November 8, 2021, 11:05am UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/2 "2021-11-08T11:05:45Z")

</div>

Hi @abderrahmen.melliti

The [omni.add\_on.ros\_control\_bridge](https://forums.developer.nvidia.com/t/external-extensions-ros-control-bridge-low-level-controllers-for-moveit/182340) **user extension** configures Isaac Sim to use the ROS FollowJointTrajectory and GripperCommand actions.

There are a few steps to configure and use such an extension

1. Include dependencies ([omni.usd.schema.add\_on](https://forums.developer.nvidia.com/t/external-extensions-usd-add-on-schemas/182263) and [omni.add\_on.ros\_bridge\_ui](https://forums.developer.nvidia.com/t/external-extensions-ros-bridge-ui-add-on/182264))

2. Publish the joint states of the robot using a `RosJointState` prim

3. Configure the `RosControlFollowJointTrajectory` and/or `RosControlGripperCommand` prims according to the configuration of the ROS controllers made with MoveIt Setup Assistant (`ros_controllers.yaml` file)

Can you see (`rostopic list`) the following (or similar) topics only playing the simulator, without launching MoveIt?

```
/arm_controller/follow_joint_trajectory/cancel
/arm_controller/follow_joint_trajectory/feedback
/arm_controller/follow_joint_trajectory/goal
/arm_controller/follow_joint_trajectory/result
/arm_controller/follow_joint_trajectory/status

```

Could you please indicate which step is giving you problems or indicate more details about the steps followed (even if not indicated)?

In any case, I added an example (an stage (.usd) and a catkin workspace with the MoveIt configuration) to the repository: [Releases · Toni-SM/omni.add\_on.ros\_control\_bridge · GitHub](https://github.com/Toni-SM/omni.add_on.ros_control_bridge/releases)

---

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [November 27, 2021, 6:52pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/3 "2021-11-27T18:52:56Z")

</div>

sorry for the late i was focusing in other topics, i also face the new problems, before, I unzipped the three plugins omni.usd.schema.add\_on and omni.add\_on.ros\_control\_bridge and omni.add\_on.ros\_bridge\_ui in a folder, and every time i want to try to do the communication I copied and put it in the exts folder why I did that because every time I restart the Isaac sim I cannot find them and i don’t know why but I was successful to enable it using window-\>extention, this time I repeated the same way but in every time I search for the plugin I can not find them, you can see it below in the pictures, for the rest of steps for communication I tried to do the simplest solution I download the example in the link that you provided it to me ‘[Releases · Toni-SM/omni.add\_on.ros\_control\_bridge · GitHub](https://github.com/Toni-SM/omni.add_on.ros_control_bridge/releases)’ but I think first I need to solve the problem of the plugin.

 ![Screenshot from 2021-11-27 17-36-26](https://global.discourse-cdn.com/nvidia/original/3X/a/3/a3bdd079e4df9f713a83d77603a9329774eb439c.jpeg)  
 ![Screenshot from 2021-11-27 17-38-41](https://global.discourse-cdn.com/nvidia/original/3X/6/6/66628d4d155ccc0c343a3df5f9af95e9da896707.jpeg)  
 ![Screenshot from 2021-11-27 17-39-05](https://global.discourse-cdn.com/nvidia/original/3X/9/e/9e0cc4b6760d4534d09852071ee49aa15542a06b.jpeg)

---

<div class="post-metadata">

**Author:** ![toni.sm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/toni.sm/32/27559_2.png) [@toni.sm](https://forums.developer.nvidia.com/u/toni.sm)\
**Post date:** [November 28, 2021, 4:57pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/4 "2021-11-28T16:57:58Z")

</div>

Hi @abderrahmen.melliti

New Isaac Sim version (2021.2.0) supports user extensions installations from GitHub directly…  
Then, you can add the extensions (from the Extensions window) to the Extension Search Path as follow:

- `git+https://github.com/Toni-SM/omni.usd.schema.add_on.git?branch=main&dir=exts`

- `git+https://github.com/Toni-SM/omni.add_on.ros_bridge_ui.git?branch=main&dir=exts`

- `git+https://github.com/Toni-SM/omni.add_on.ros_control_bridge.git?branch=main&dir=exts`

 ![Screenshot from 2021-11-28 17-47-10](https://global.discourse-cdn.com/nvidia/original/3X/e/4/e4767188e37144eac429fa61baa7e793594414c3.jpeg)

---

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [November 28, 2021, 5:49pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/5 "2021-11-28T17:49:24Z")

</div>

@toni.sm unfortunately, I don’t have this option I’m using the previous version, I tried again, I put the three plugins in the exts folder but it’s like Isaac sim doesn’t read them and I used the import extension also from window but it doesn’t work. Do you have an idea how to solve it?

---

<div class="post-metadata">

**Author:** ![toni.sm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/toni.sm/32/27559_2.png) [@toni.sm](https://forums.developer.nvidia.com/u/toni.sm)\
**Post date:** [November 28, 2021, 6:12pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/6 "2021-11-28T18:12:52Z")

</div>

Hi @abderrahmen.melliti

Did you kill and run Isaac Sim (`./isaac-sim.headless.kitremote.sh --allow-root`) after copy the extensions to `/isaac-sim/exts` folder?

---

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [November 28, 2021, 6:15pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/7 "2021-11-28T18:15:44Z")

</div>

@toni.sm no, actually I have no idea about that.

---

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [December 5, 2021, 6:06pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/8 "2021-12-05T18:06:45Z")

</div>

Hi @toni.sm  
I downloaded the new version and i tried this solution to put the link in the Extention Search Path and also i tried the old one which is to copy the plugins in the exts folder but its failed , could you please help me ?

 ![image](https://global.discourse-cdn.com/nvidia/original/3X/2/0/20f43363bb0f7dd857c6125dcecfeb836a442389.png)  
 ![image](https://global.discourse-cdn.com/nvidia/original/3X/f/8/f8a269b057b56bf14ba7b2cef491a8ddb7f72acc.png)

---

<div class="post-metadata">

**Author:** ![ltorabi](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/ltorabi/32/14043_2.png) [@ltorabi](https://forums.developer.nvidia.com/u/ltorabi)\
**Post date:** [December 6, 2021, 4:58pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/9 "2021-12-06T16:58:32Z")

</div>

Toni, is on vacation, will be back in a week

---

<div class="post-metadata">

**Author:** ![ltorabi](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/ltorabi/32/14043_2.png) [@ltorabi](https://forums.developer.nvidia.com/u/ltorabi)\
**Post date:** [December 20, 2021, 7:56pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/10 "2021-12-20T19:56:25Z")

</div>

@toni.sm

---

<div class="post-metadata">

**Author:** ![toni.sm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/toni.sm/32/27559_2.png) [@toni.sm](https://forums.developer.nvidia.com/u/toni.sm)\
**Post date:** [December 20, 2021, 7:58pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/11 "2021-12-20T19:58:44Z")

</div>

We have already been in contact… 😅

---

<div class="post-metadata">

**Author:** ![09ubberboy90](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@09ubberboy90](https://forums.developer.nvidia.com/u/09ubberboy90)\
**Post date:** [January 11, 2022, 3:07pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/12 "2022-01-11T15:07:14Z")

</div>

> [@abderrahmen.melliti](#):
>
> [INFO] [1636322147.758640039, 2873.734975096]: Found a contact between ‘panda\_link5’ (type ‘Robot link’) and ‘panda\_hand’ (type ‘Robot link’), which constitutes a collision. Contact information is not stored.

Hi, I don’t know if you still have this problem but this error is due to the default position of panda that is wrong. You can either disable self collide (Not sure how to do that in isaac) or you can edit the panda.srdf file that is included in the launch file with this lines

```auto
  <disable_collisions link1="panda_link5" link2="panda_hand" reason="Adjacent"/>
  <disable_collisions link1="panda_link5" link2="panda_rightfinger" reason="Adjacent"/>
  <disable_collisions link1="panda_link5" link2="panda_leftfinger" reason="Adjacent"/>

```

I know that even if everything else was working I would still get failure to move the robot until i fixed that

---

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [January 11, 2022, 6:32pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/13 "2022-01-11T18:32:19Z")

</div>

Thank you so much I will try it for sure, now I have the problem with the plugins I can not download them and use them.

---

<div class="post-metadata">

**Author:** ![09ubberboy90](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@09ubberboy90](https://forums.developer.nvidia.com/u/09ubberboy90)\
**Post date:** [January 12, 2022, 4:34pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/14 "2022-01-12T16:34:49Z")

</div>

I can’t really help you with that as it is working fine for me using github and Isaac 2021.1.2 . What version of Isaac are you using ?

---

<div class="post-metadata">

**Author:** ![abderrahmen.melliti](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@abderrahmen.melliti](https://forums.developer.nvidia.com/u/abderrahmen.melliti)\
**Post date:** [January 15, 2022, 9:04pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/15 "2022-01-15T21:04:14Z")

</div>

Thank you, i’m using the version Isaac 2021.1.1 but now its work I was able to put the plugins in the exts folder and i will try your solution but first, i have to simulate the Yaskawa robot with Isaac , thank you again.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [April 5, 2024, 3:36pm UTC](https://forums.developer.nvidia.com/t/isaac-sim-moveit/194297/16 "2024-04-05T15:36:41Z")

</div>

Closed on April 5, 2024, 3:36pm UTC


