How to change action and observation space in RLTask?

[mkiln]

I’ve written a custom RLTask using the Cartpole example. However, I do not know how to change the observation and action spaces. By default they are Box spaces of floating numbers. It is possible to clip those values in the task.cfg file, however I can’t figure out how to use other spaces.

In my specific environment I need to use a Box of [-1.0, 1.0] values for observations and a Discrete action space of [-5, 5].

Is there a way to change the default action and observation spaces defined in the RLTask?

[mkiln]

After debugging the errors which lead to inspecting the source files provided in the Isaac Gym implementation, I have finally found the solution!

Mind that this solution is for a custom task class which derives from the RLTask class:

1. Inside the initializer of your custom task, before calling RLTask.__init__(self, name, env), you should specify the action and observation spaces: self.action_space, self.observation_space.

NOTE: As per the a2c_common.py file, on line 893, you need to use a gym.spaces.Discrete or a gym.spaces.Tuple as an action space. It will not work with any other gym space and it will not work if you use gymnasium.spaces. It NEEDS to be gym.spaces.

2. Inside your ...PPO.yaml config file, the algorithm name should be a2c_discrete and the model name should be discrete_a2c. Inside the network config, the space should be discrete. It can be continuous, discrete and multi_discrete.

NOTE: As per the torch_runner.py and model_builder.py files, the available algorithms, models and networks you can use in the config file are:

• Algorithms: a2c_continuous, a2c_discrete and sac.

• Models: discrete_a2c, multi_discrete_a2c, continuous_a2c, continuous_a2c_logstd, soft_actor_critic, central_value.

• Networks: actor_critic, resnet_actor_critic, rnd_curiosity, soft_actor_critic.

I do not know if this is covered somehow in the documentation but I could not find it anywhere. If someone knows the links to the documentation that explain this, please share.