Sim2Real of RMPflowcontrol on my robot

Omniverse Isaac Sim
1

Hi.

I am using IsaacSim to customize my own robot manipulation (Doosan M1013).

Ive been trying to use ‘RMPflowcontorl’ solving IK and moving my robot end-effector.

After that, how can I control my ‘real’ robot using RMPflowcontrol in the virtual world(Omniveres) ?

Is there any easy method to control the real robot using RMP, otherwise should I make a specific interface to communicate with my robot?

Could you tell me about the Sim2Real method of robots that IsaacSIm developers mainly use?

Thank you !

2

Hi Swimpark,

WE will have some new tools for this in our next release, in April.

Please stay tuned.

kindly,
Liila

4

Hi @ltorabi ,

I have a similar problem. I have built an Isaac Sim simulation with a Universal robot and the RMPflow motion policy. This works quite well so far. Now I am not sure what approaches there are to transfer the RMPflow motion policy to a real universal robot, for example in Python or as ROS block…

Can you help me further?

Many thanks for your support

5

Hi @robin-vetsch , that is great to hear.
Let me assign someone to help.

6

We don’t have anything out-of-the-box like that for the UR line at the moment like we do for Franka. What you’d need to do is implement a controller which accepts the RMPflow command stream and translates that to desired velocities for the UR velocity control interface. See this discussion for a discussion of how to use the velocity controller:

This would likely involve translating the desired positions to velocity commands using a law similar to

    qd_c += alpha (q_d - q) 
    qd_c = beta qd_c + (1-beta) qd_d

alpha and beta are both between 0 and 1. The first line nudges the velocity command in the direction of the current position error, while the second blends the desired command toward the desired velocity as a regularizer. You might also need to blend toward zero (regularize back toward zero).

In the end, you’ll likely have to play with the parameters, or the form of the controller. These are well studied problems, so there are some good referenced out there on how to do this. It’d help to find a controls expert to consult with.

We’re working to add more low-level controllers to enable easier connection to physical robots, but currently we only have the Franka as an example. Take a look at that to see what it’s doing:
https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/tutorial_cortex_6_ros_synchronization.html#controlling-physical-robots
You’d need to implement a version doing something similar for the UR using the UR’s control interface:

1 Like