Simulation to Real Project Signal Solutions

Robotics & Edge Computing Robotics - Isaac Isaac Gym
1

Hi I’m trying to do Sim2Real Project by myself.
The delay between simulation and hardware seems to be a little severe.
Do you have any idea about these problems?

I’m using Python & threading method to share Angle value,
And also using Arduino for hardware. [pwm output]

2

I think Isaac Gym is not a good real-time system, it is only good for simulation and training. After you get the trained neural network model, it is better to transfer to a real system with ROS/ROS2/Isaac SDK

3

based on my observation, Gym on GPU is fast enough, If you use the headless mode or turn off the viewer (using v key) simulation would be much faster and I’m guessing you can get real time simulation, the problem should come from outputting the data, as probably data need to be transferred from GPU to CPU which creates a latency, I’m guessing if somehow you could get the data directly from GPU it would solve that problem too. If you want to use your pc and your GPU has audio output, you might be able to get something out of it, it would be funny to see such a thing :)
A more crazier method would be using a HDMI decoder!

Jetson family is designed for sim2real applications, using TensorRT you can also make the NN much more efficient and faster for Sim2Real applications.

4

I like your Ideas so I tried it several time

  1. I tried ‘v’ key, but it doesn’t work (I don’t know why)

  2. I tired headless mode by
    python myfile.py --graphics_device_id=-1
    or --nographics
    then terminal shows ‘Reading …’ messages. it doesn’t work TT

I should buy Jetson Nano once I make some money. :)
Always thank you

5

Thanks for your reply!
I’ve tried only Isaac sim & gym
So I need to learn more about SDK thanks!