Isaac GYM, How to perform Sim2Real

Autonomous Machines Robotics - Isaac Isaac Gym
1

Hi~ Anybody has done Sim2Real works by utilizing isaac gym(isaac gym environments examples). I would appreciate it, if some can provide me some help about the pytorch neuron nerwork transplanting and isaac gym framework communicating with real robot.

3 Likes
2

I am doing Sim2real too.But I still can’t deal with it,can I have your email,we can discuss for it.

3

please check your message, i’v sent you my email :)

4

Can I have your email too? I’m also quite care about this problem, and we can exchange our idea.

7

Does anyone success?

8

Hi,

I am grappling with the same issue. Simulation to Real robot RL deployment seems to be well guarded secret. One solution I am attempting is logging the position velocity and torque values from the play scripts of the example files and putting that onto the robot in a loop.

I don’t think this is ideal but something to start with.

Sanket

9

Hi @slokhande and others,

Lots of domain randomization is important for Sim2Real transfer. You can read more about what we have done in our own research on this here: https://dextreme.org/

Actually hooking up the Sim model with your real robot is going to be very dependent on the way you have your robot control system set up. Simply playing back a log of position or torque commands isn’t likely to be enough. You will need to feed observations from your real robot sensors back into the policy in your control loop.

Take care,
-Gav

1 Like
10

Any success on this @Kumiko98 @slokhande @user38580 @CaptainMaoli @780927810 ?

Any new resources found are appreciated!

Thanks

11

I’m working on my sim to real tutorials right now. I’ve gotten an inverted pendulum trained in isaac gym running on an STM32 last month. Will drop an update when I get things together in a more presentable way. In the mean time feel free to reach out with any questions about the process.

1 Like
12

Isaac.Orbit is better for implementing sim2real!
My experience is that using RL for real life applications is very challenging! especially if you want to use it for replacing a classic controller in a dynamically unstable system. And Nvidia has made Isaac Sim so complicated which makes Real2Sim and Sim2Real even more difficult to implement. For implementing a Sim2Real you first need a good real2sim step which needs expensive lab equipment like 6 Axis force sensors and …
At this stage I will stick to RL for higher level decision makings rather than using it for controlling the robot and replacing lower level controllers. It’s just not worth the time!

1 Like