Use SAC under Isaac Gym (v. 2023.0.1b)

elia.tosin · November 23, 2023, 9:17am

Hello,

Recently I have updated Isaac Sim to version 2023.1.0-hotfix.1 (as seen in this screen from the omniverse launcher)

I also upgraded OmniIsaacGymEnvs by cloning the repo tagged with 2023.1.0b.

Now I’m trying to run locally the training with the task AntSAC as done in the examples.

The script I’m running in the omniisaacgymenvs folder inside the cloned repository is the following (as done here):

PYTHON_PATH scripts/rlgames_train.py task=AntSAC train=AntSAC

For the other task (Ant/Humanoid/Anymal) it works fine but when I try to that task (that uses the SAC instead of PPO, same problem for HumanoidSAC) it gives me an error saying like I missed some configuration in the command. The error rises after the first step in the simulation, after loading Isaac Sim.

[2023-11-23 10:27:52] Running RL reset
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: Error executing job with overrides: ['task=HumanoidSAC', 'train=HumanoidSAC']

Error executing job with overrides: ['task=HumanoidSAC', 'train=HumanoidSAC']
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: Traceback (most recent call last):

Traceback (most recent call last):
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/Documents/OmniIsaacGymEnvs23/omniisaacgymenvs/scripts/rlgames_train.py", line 150, in <module>

  File "/home/eliatosin/Documents/OmniIsaacGymEnvs23/omniisaacgymenvs/scripts/rlgames_train.py", line 150, in <module>
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: parse_hydra_configs()
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: 

    parse_hydra_configs()
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/main.py", line 94, in decorated_main

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/main.py", line 94, in decorated_main
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: _run_hydra(
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: 

    _run_hydra(
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: _run_app(
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: 

    _run_app(
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: run_and_report(
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: 

    run_and_report(
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 223, in run_and_report

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: raise ex
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: 

    raise ex
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,592ms] [Error] [omni.kit.app._impl] [py stderr]: return func()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    return func()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: lambda: hydra.run(
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    lambda: hydra.run(
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: _ = ret.return_value
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    _ = ret.return_value
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: raise self._return_value
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    raise self._return_value
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: ret.return_value = task_function(task_cfg)
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    ret.return_value = task_function(task_cfg)
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/Documents/OmniIsaacGymEnvs23/omniisaacgymenvs/scripts/rlgames_train.py", line 142, in parse_hydra_configs

  File "/home/eliatosin/Documents/OmniIsaacGymEnvs23/omniisaacgymenvs/scripts/rlgames_train.py", line 142, in parse_hydra_configs
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: rlg_trainer.run()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    rlg_trainer.run()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/Documents/OmniIsaacGymEnvs23/omniisaacgymenvs/scripts/rlgames_train.py", line 74, in run

  File "/home/eliatosin/Documents/OmniIsaacGymEnvs23/omniisaacgymenvs/scripts/rlgames_train.py", line 74, in run
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: runner.run(
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    runner.run(
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/torch_runner.py", line 133, in run

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/torch_runner.py", line 133, in run
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: self.run_train(args)
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    self.run_train(args)
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/torch_runner.py", line 116, in run_train

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/torch_runner.py", line 116, in run_train
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: agent.train()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    agent.train()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/algos_torch/sac_agent.py", line 541, in train

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/algos_torch/sac_agent.py", line 541, in train
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: step_time, play_time, update_time, epoch_total_time, actor_losses, entropies, alphas, alpha_losses, critic1_losses, critic2_losses = self.train_epoch()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    step_time, play_time, update_time, epoch_total_time, actor_losses, entropies, alphas, alpha_losses, critic1_losses, critic2_losses = self.train_epoch()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/algos_torch/sac_agent.py", line 529, in train_epoch

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/algos_torch/sac_agent.py", line 529, in train_epoch
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: return self.play_steps(random_exploration)
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    return self.play_steps(random_exploration)
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:   File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/algos_torch/sac_agent.py", line 497, in play_steps

  File "/home/eliatosin/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/kit/python/lib/python3.10/site-packages/rl_games/algos_torch/sac_agent.py", line 497, in play_steps
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]:     
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: self.obs = next_obs.clone()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

    self.obs = next_obs.clone()
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: AttributeError
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: : 
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 'dict' object has no attribute 'clone'
2023-11-23 09:27:58 [19,593ms] [Error] [omni.kit.app._impl] [py stderr]: 

AttributeError: 'dict' object has no attribute 'clone'

(the folder /home/eliatosin/Documents/OmniIsaacGymEnvs23 is actually the cloned repository OmniIsaacGymEnvs. I renamed it to know the version.

Thank you

psh9002 · November 23, 2023, 12:47pm

I recommend you to try with skrl.

I think it is more easy to use than default usage on the documentation.

bwlan616 · April 28, 2024, 7:31am

It seems to be a problem of rl_games.
There’s an issue about it, and I solved the problem by editing the file “kit\python\Lib\site-packages\rl_games\algos_torch\sac_agent.py” in IsaacSim package as this PR did.

Topic		Replies	Views
Poor performance of Soft Actor Critic (SAC) in OmniverseIsaacGym Isaac Sim	1	776	June 1, 2024
Errors when running Cartpole RL example in Isaac Sim 2023.1.1 Isaac Sim	4	329	June 27, 2024
OmniIsaacGymEnv porevious version (Isaac Sim 2022.2.0) Isaac Sim	6	667	June 10, 2023
Error occurring when trying reinforcement learning examples in Isaac gym Isaac Sim	15	1550	March 8, 2023
I'm using OmniIgaacGym with Isaac Sim. I have an error Isaac Sim	0	198	May 8, 2024
Isaacsim 4.5 crash Isaac Sim isaac-sim-v4-5-0	7	245	June 23, 2025
IsaacGymEnv task run issue Isaac Gym	2	719	June 29, 2023
Attempt Isaac gym in isaac sim, run the cartpole example, i met some problem Isaac Gym cuda , ubuntu , python	0	489	July 26, 2022
SKRL Documentation Jetbot example for Isaac sim doesn't work Isaac Sim rl	13	1036	September 4, 2023
Cannot import `omni` related modules Isaac Sim	17	8015	April 23, 2023

Use SAC under Isaac Gym (v. 2023.0.1b)

Related topics