I was wondering if it is possible to use Isaac Gym for something like exploration/navigation RL problems, where a robot or a fleet or robots try to maximize their coverage of an environment the minimum time, or learn to navigate around obstacles in some kind of environment. The environment is randomized for every episode of training. For the most part, I see people only using Isaac Gym for cases that require 3D RL problems (like manipulators, object control, walking, flying, etc.). Is Isaac Gym not intended for this use case, or has anyone tried this out before?
what you train depends on your observations and your reward function, you can insert anything to your observation space, from joint values to the location of each robot in the environment. Every time you reset one environment you can have a robot appears randomly somewhere in its environment. It’s very doable I think.
Hi @Abi64, thanks for your reply. It seems like some people who have already developed with IsaacGym say its not feasible to reset environments per episode (see Section 2.1 of this paper: [2109.11978v3] Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning). I’ll need to figure out a way to generate several different environments at once, but each environment has boundaries such that robots cannot interact with robots in another environment. I haven’t done too much experimentation with IsaacGym yet so I’m not sure if this is possible.
First thing first move to Isaac sim or orbit, you have more options there and have access to gym as well. in gym there is an option to define which assets can collide:
Collision filter bitmask - shapes A and B only collide if (filterA & filterB) == 0.
Robots can move through each other without colliding if you define them like this. I think it’s one of the gym examples as I remember.
You can reset an environment anytime during the simulation while other environments are running.
Don’t expect to setup your environment and get the results very soon, it would take 1-3 months at least to understand gym, sim and orbit! NVIDIA has made these products for high level software engineers not robotics engineers, so nothing would be easy!