Trouble with reinforcement learning task in Isaac Lab

Omniverse Isaac Sim
1

Hello,

Recently I managed to train neural networks to balance a double pendulum (including the swing up) using a naive and very simple evolutionary algorithm. I now want to compare training speed and results with more modern and robust RL algorithms such as the ones provided with Isaac Lab on the exact same task. I modified the cartpole example to turn it into a double pendulum and modified the reward terms accordingly.

However after a few hours of training no solution was found (using rl_games and skrl with 8192 envs).
I then tried to use a easier setup by using a very low gravity and high rotational damping on the poles but it was also unsuccessful.

I am currently unsure if the problem comes from

  • the reward function
  • the setup
  • the task in itself being too challenging
  • or if I just need to increase the number of envs or training time

Any insight would be very helpful!

Thanks,
Pezzza

3

Hi @pezzzas.work ,

Thank you for your post.
Could you please share your modification/patch about the cartpole so we can try it on our end?

Thanks,
-Todd

4

Thank you for your quick response.

I created a fork of the repo and put my changes in a branch called “double_pendulum_older” that you can checkout here https://github.com/johnBuffer/IsaacLab/tree/double_pendulum_older

I run the training using this command (replacing skrl with rl_games depending on the algorithm to test):

isaaclab.bat -p source\standalone\workflows\skrl\train.py --task Isaac-Cartpole-v0 --num_envs 4096 --headless

If you need any additional information don’t hesitate to ask!

5

Hi @pezzzas.work ,

The good news, we’ll have double pendulum task in the next release of isaac lab.
You can compare the implementation then.

Thanks,
-Todd

6

Hi Todd,

A few questions regarding this information:

  • Do you know if this includes the swing up?
  • Do you know when this next release will be available?

Thanks,
Jean

7

Hi @pezzzas.work ,

Do you know if this includes the swing up?

No, it doesn’t include swing up.

Do you know when this next release will be available?

It’s about the 2nd of September.

Thanks,
-Todd

8

Hi Todd,

Thank you for your response. I will try the new release when it will be out and see if I can manage to add the swing up then.

Thanks,
Jean

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.