# PPO Implementation

**URL:** <https://forums.developer.nvidia.com/t/ppo-implementation/202561>\
**Category:** Isaac Gym\
**Created:** [February 6, 2022, 12:13am UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561 "2022-02-06T00:13:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![noshaba](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@noshaba](https://forums.developer.nvidia.com/u/noshaba)\
**Post date:** [February 6, 2022, 12:13am UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561/1 "2022-02-06T00:13:52Z")

</div>

Hi,

Where can I find the PPO implementation used in Isaac Gym?

Best,  
Noshaba

---

<div class="post-metadata">

**Author:** ![780927810](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@780927810](https://forums.developer.nvidia.com/u/780927810)\
**Post date:** [February 13, 2022, 2:58pm UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561/2 "2022-02-13T14:58:49Z")

</div>

There are several PPO implementations in the following website.

> **[GitHub - isaac-sim/IsaacGymEnvs: Isaac Gym Reinforcement Learning Environments](https://github.com/isaac-sim/IsaacGymEnvs)**
>
> Isaac Gym Reinforcement Learning Environments

---

<div class="post-metadata">

**Author:** ![noshaba](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@noshaba](https://forums.developer.nvidia.com/u/noshaba)\
**Post date:** [February 15, 2022, 4:15pm UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561/3 "2022-02-15T16:15:00Z")

</div>

Where in the repo do you see a PPO implementation? Could you point me to a specific file?

---

<div class="post-metadata">

**Author:** ![erwin.coumans](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/erwin.coumans/32/24866_2.png) [@erwin.coumans](https://forums.developer.nvidia.com/u/erwin.coumans)\
**Post date:** [February 15, 2022, 10:04pm UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561/4 "2022-02-15T22:04:29Z")

</div>

Isaac Gym uses rl\_games PPO, see one of the algorithms here:

> **[rl\_games/rl\_games/algos\_torch at master · Denys88/rl\_games](https://github.com/Denys88/rl_games/tree/master/rl_games/algos_torch)**
>
> RL implementations. Contribute to Denys88/rl\_games development by creating an account on GitHub.

another PPO is implemented here: [rsl\_rl/rsl\_rl/algorithms at main · leggedrobotics/rsl\_rl · GitHub](https://github.com/leggedrobotics/rsl_rl/tree/master/rsl_rl/algorithms)  
this is used in legged\_gym, build on top of Isaac Gym

---

<div class="post-metadata">

**Author:** ![vmakoviychuk](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vmakoviychuk/32/54577_2.png) [@vmakoviychuk](https://forums.developer.nvidia.com/u/vmakoviychuk)\
**Post date:** [February 20, 2022, 9:49pm UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561/5 "2022-02-20T21:49:42Z")

</div>

Hi @noshaba,

As @erwin.coumans posted we use rl-games: [GitHub - Denys88/rl\_games: RL implementations](https://github.com/Denys88/rl_games) with all of our training environments in IsaacGymEnvs as well as in the Isaac Gym paper: [[2108.10470] Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://arxiv.org/abs/2108.10470) It referenced in the default setup.py installation script. In addition to the PPO it has high-performance vectorized zero-copy SAC implementation, and support of multi-agent and self-play scenarios.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [March 10, 2022, 11:07am UTC](https://forums.developer.nvidia.com/t/ppo-implementation/202561/6 "2022-03-10T11:07:14Z")

</div>

Closed on March 10, 2022, 11:07am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.