Customize loggers to view individual reward parameter

Hi everyone,

Summary: I have been trying to tune the reward function (both parameters and scales) for better training results. My training is a legged robot based on the Anymal Terrain example. I am wondering if there is a way to log and visualize individual rewards on each parameter throughout the epochs. So far I can only see mean reward using tensorboard or WandB. Original And it doesn’t tell me how the reward scale is changing the model. Is there a good place to start editing? In addition, can I save simulation values like velocity, force, etc. for the agents?

Attempt so far: I have located that the training data, i.e. loss, mean reward, and time, are logged in amp_contineous. Can I just define an extra function to save individual reward terms that are used to calculate the mean reward? Do I have to edit anything else?

I have been really new to RL and IsaacGym, and I am asking this question because it seems like a reasonable way by identifying individual terms in the reward function in order to create useable reward functions. I would really appreciate it if you could also share how to scientifically tune rewards if this is not the way to go.

Thank you in advance.

Cheers,

It’s not Isaacgym related, but you can checkout the issues of rl-games: Issues · Denys88/rl_games · GitHub
The issue might be related: Logging in environments · Issue #105 · Denys88/rl_games · GitHub

1 Like

Hi @lyichen

I encourage you to try the skrl library…
There you will find an easy way to record and visualize the metrics you want…

https://skrl.readthedocs.io/en/latest/intro/data.html#tracking-custom-metrics-scales

1 Like