skrl is an open-source modular library for Reinforcement Learning written in Python (using PyTorch) and designed with a focus on readability, simplicity, and transparency of algorithm implementation. In addition to supporting the OpenAI Gym and DeepMind environment interfaces, it allows loading and configuring NVIDIA Isaac Gym and NVIDIA Omniverse Isaac Gym environments, enabling agents’ simultaneous training by scopes (subsets of environments among all available environments), which may or may not share resources, in the same run
Please, visit the documentation for usage details and examples
The current version 0.6.0 is now available (it is under active development. Bug detection and/or correction, feature requests and everything else are more than welcome: Open a new issue on GitHub! ). Please refresh your browser (Ctrl + Shift + R or Ctrl + F5) if the documentation is not displayed correctly
This new version has focused on supporting the training and evaluation of reinforcement learning algorithms in NVIDIA Omniverse Isaac Gym
skrl version 0.7.0 is now available (it is under active development. Bug detection and/or correction, feature requests and everything else are more than welcome: Open a new issue on GitHub! ). Please refresh your browser (Ctrl + Shift + R or Ctrl + F5) if the documentation is not displayed correctly
Added
A2C agent
Isaac Gym (preview 4) environment loader
Wrap an Isaac Gym (preview 4) environment
Support for OpenAI Gym vectorized environments
Running standard scaler for input preprocessing
Installation from PyPI (pip install skrl)
Now, with the implementation of the standard scaler (adapted from rl_games), better performance is achieved…
The idea is to add more algorithms to the library gradually :)
Regarding environments, the development is focused on adding algorithms and functionalities to work with the relevant environments and interfaces in the RL field (such as Omniverse Isaac Gym, Isaac Gym, OpenAI Gym, or DeepMind, for example). Creating or providing environments as part of the library is not on my roadmap at the moment.
skrl version 0.8.0 is now available (it is under active development. Bug detection and/or correction, feature requests, and everything else is more than welcome: Open a new issue on GitHub! ). Please refresh your browser (Ctrl + Shift + R or Ctrl + F5) if the documentation is not displayed correctly
Added
AMP agent for physics-based character animation
Manual trainer
Gaussian model mixin
Support for creating shared models
Parameter role to model methods
Wrapper compatibility with the new OpenAI Gym environment API (by @JohannLange)
Internal library colored logger
Migrate checkpoints/models from other RL libraries to skrl models/agents
Configuration parameter store_separately to agent configuration dict
Set random seed and configure deterministic behavior for reproducibility
Benchmark results for Isaac Gym and Omniverse Isaac Gym on the GitHub discussion page
Franka Emika real-world example
Changed
Models implementation as Python mixin [breaking change]
Multivariate Gaussian model (GaussianModel until 0.7.0) to MultivariateGaussianMixin
Trainer’s cfg parameter position and default values
Show training/evaluation display progress using tqdm (by @JohannLange)
Update Isaac Gym and Omniverse Isaac Gym examples
Fixed
Missing recursive arguments during model weights initialization
Tensor dimension when computing preprocessor parallel variance
Models’ clip tensors dtype to float32
Removed
Parameter inference from model methods
Configuration parameter checkpoint_policy_only from agent configuration dict
As a showcase for the basic Franka Emika real-world example, a simulated version of the environment for Isaac Gym and Omniverse Isaac Gym are provided to support advanced implementations :)
skrl version 0.9.0 is now available (it is under active development. Bug detection and/or correction, feature requests, and everything else is more than welcome: Open a new issue on GitHub! ). Please refresh your browser (Ctrl + Shift + R or Ctrl + F5) if the documentation is not displayed correctly
Added
Support for Farama Gymnasium interface
Wrapper for robosuite environments
Weights & Biases integration (by @juhannc)
Set the running mode (training or evaluation) of the agents
Allow clipping of the gradient norm for DDPG, TD3, and SAC agents
Initialize model biases
Add RNN (RNN, LSTM, GRU, and any other variant) support for A2C, DDPG, PPO, SAC, TD3, and TRPO agents
Allow disabling training/evaluation progressbar
Farama Shimmy and robosuite examples
KUKA LBR iiwa real-world example
Changed
Forward model inputs as a Python dictionary [breaking change]
Returns a Python dictionary with extra output values in model calls [breaking change]
Adopt the implementation of terminated and truncated over done for all environments
Fixed
Omniverse Isaac Gym simulation speed for the Franka Emika real-world example
Call agents’ method record_transition instead of the parent method
to allow storing samples in memories during the evaluation
Move TRPO policy optimization out of the value optimization loop
Access to the categorical model distribution
Call reset only once for Gym/Gymnasium vectorized environments
Removed
Deprecated method start in trainers
As a showcase, a basic real-world example is provided where a KUKA LBR iiwa robot is controlled through a direct API and via ROS/ROS2. In addition, its simulated version, in Omniverse Isaac Gym, is provided to support advanced implementations :)
skrl version 0.10.0 is now available (it is under active development. Bug detection and/or correction, feature requests, and everything else is more than welcome: Open a new issue on GitHub! ). Please refresh your browser (Ctrl + Shift + R or Ctrl + F5) if the documentation is not displayed correctly
This unexpected new version has focused on supporting the training and evaluation of reinforcement learning algorithms in NVIDIA Isaac Orbit
Dear developer,
I am currently using the multi-env framework with the skrl algorithm library. However, it’s strange that the performance of skrl is slightly worse than that of stable baselines3. I’m not sure if this is due to my configuration. Additionally, when clip_action=False, the actions outputted by skrl sometimes exceed the action space that I’ve defined. If clip_action=True, the actions will be clipped to the boundary values of -1 or 1. Could you please help me clarify these issues? Thank you very much.
skrl version 1.0.0-rc.1 is now available (it is under active development. Bug detection and/or correction, feature requests, and everything else is more than welcome: Open a new issue on GitHub! ). Please refresh your browser (Ctrl + Shift + R or Ctrl + F5) if the documentation is not displayed correctly.
Among the major features of this new version are:
JAX support
Multi-agent training (the beginning).
Comprehensive documentation with new structure and theme
Visit the library documentation to start training your favorite task in Isaac Gym preview, Isaac Orbit or Omniverse Isaac Gym using PyTorch or JAX!
For questions and others, please open a new discussion in the skrl repository.
That way we leave this topic for announcements :)
The execution times used to construct this graph are the complete execution of the script
(including environment loading and, in the case of JAX, the jit compilation of XLA, for example).
