We are considering testing Isaac Sim for path planning with obstacle avoidance for a 7 axis robotic arm in a constrained environment. We would need some clarification reading our understanding of the current capabilities.
Our robot evolves inside a box that is is relatively small. It is so small that the robot cannot fit into it in its zero angular position. Our pick points can be everywhere in the box. As a consequence, the paths are very narrow. The robot often collide with the environment. Most of time, it is intermediate parts that collides. Also, we have self collisions between robot parts. We also often push the robot into its joints limits.
Currently, we perform collision detection with PyBullet. If the tool collides, we push it out of collision using the collision direction. Then, if one intermediate part of the robot collides, we recompute and inverse kinematics using a rest pose that push the robot out of collision. If the robot self collide we unfold it and if it collide with environment we fold it. We need to iterate to find a equilibrium between these opposite constraints.
This works, but this can take a while to converge, what is not compatible with the production constraints of time.
When we ask for direction we always have the same answer: try ROS and Moveit.
This is a big rewamp of the project because we don’t use ROS and everything is in Python, while Moveit 2 Python API for ROS does not seem to be mature enough for this need, what mean that we need to switch the planner to C++.
We have seen the Motion Generation page of Isaac documentation and RMPFlow. It seems to be easier to deploy than Moveit. Also, we are very interested into the realistic simulation of Isaac Sim because we work a lot with computer vision.
What is less clear for us is the degree of maturity of the path planning with collisions avoidance of RMPFlow for 7 axis robotic arms.
Our project is tile constrained. If we gain time by using a solution that is easier to code in Python but then we face limitations that create the need for lots of custom code, it is not a gain.
We have seen that RMPFlow includes an algorithm for obstacle avoidance. It mentions the collisions between the robot parts and the environment, but not the self collisions between the different robot parts. Is it included?
One approach would be to ask RMPFlow to generate a trajectory that avoids obstacles and then, if it fails, to rollback to our custom algorithm, replacing PyBullet obstacle detection by the one of Isaac Sim.
For that we need a detailed report on collisions between robot parts and the environments and between robot parts themselves. This report has to include the extreme points of penetration, so that we can compute a vector to find the direction that gets us out of collision.
Is it something that Isaac Sim can provide?
Thanks a lot for the clarifications!
Ho, and some less important side questions…
It may also be interesting to test some OMPL algorithms. We can do that using the Python API of OMPL. Unless there are already some integrations?
Also, our robot has strange joints angular limits. Some limits are not fixed. They depend upon a linear combination of the axis angles. Currently we cannot manage this with PyBullet. The only solution we found is to reject inverse kinematics out of limits after calculus and to push back into the limits with a rest pose that unfold the robot. Unless there are better options with Isaac?