Unable to train multi environment robot

[toni.sm]

Hi @LYS_00

After looking at the code, I have the following comments:

• The VecEnvBase instance (from the omni.isaac.gym extension) used to create the task inherits from gym.Env class.

• The OpenAI Gym gym.Env class can only define 1 environment.

• It is possible to define multi-environments in Gym via the gym.vector.make API. This API will instantiate the classes gym.vector.SyncVectorEnv or gym.vector.AsyncVectorEnv according to the asynchronous parameter. However, it is not recommended to use this API with Isaac Sim’s original example (1 environment) because it will instantiate as many SImulationApps as the number of environments defined using the num_envs parameter.

---

The recommended solution is to move to the RL Framework defined in the OmniIsaacGymEnvs for working with multiples environments in Isaac Sim.

---

Nevertheless, you can always create a custom wrapper (CustomGymWrapper) to work with multiple environments even if the task inherits from gym.Env as shown in the attached code. By the way, the code uses the skrl-v0.9.0 (unreleased) which introduces a small breaking change in the way models’ .compute(...) methods are called to support RNN, GRU, and LSTM :)

cartpole_train.py (6.7 KB)

Also, modify the cartpole_task.py script and remove the .item() statement in calculate_metrics and is_done methods: return reward and return resets