Train an agent with discrete action


I want to train an agent with discrete action space, but I don’t know how to implement.
It seems that all samples in IsaacGym have continuous action space, so I am confused.
Is there any good example codes?

Thank you very much.


After trying to do the same unsuccessfully, I started debugging the errors which lead to inspecting the source files provided in the Isaac Gym implementation. Finally found the solution!

Mind that this solution is for a custom task class which derives from the RLTask class:

  1. Inside the initializer of your custom task, before calling RLTask.__init__(self, name, env), you should specify the action and observation spaces: self.action_space, self.observation_space.

NOTE: As per the file, on line 893, you need to use a gym.spaces.Discrete or a gym.spaces.Tuple as an action space. It will not work with any other gym space and it will not work if you use gymnasium.spaces. It NEEDS to be gym.spaces.

  1. Inside your ...PPO.yaml config file, the algorithm name should be a2c_discrete and the model name should be discrete_a2c. Inside the network config, the space should be discrete. It can be continuous, discrete and multi_discrete.

NOTE: As per the and files, the available algorithms, models and networks you can use in the config file are:

  • Algorithms: a2c_continuous, a2c_discrete and sac.

  • Models: discrete_a2c, multi_discrete_a2c, continuous_a2c, continuous_a2c_logstd, soft_actor_critic, central_value.

  • Networks: actor_critic, resnet_actor_critic, rnd_curiosity, soft_actor_critic.

I do not know if this is covered somehow in the documentation but I could not find it anywhere. If someone knows the links to the documentation that explain this, please share.

1 Like