Jetbot Lane Following reinforcement learning sample ERROR_OUT_OF_DEVICE_MEMORY

Omniverse Isaac Sim
,
1

I am trying to run the JetBot Lane Following reinforcement learning sample from here: https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/rl_samples.html

I believe everything should be set up properly, imported the fbx files into the Nucleus server, sourced setenv.sh, set up conda env etc. but I get the following errors when running python jetbot_train.py:

This error multiple times:

2020-12-23 11:53:20 [18,079ms] [Warning] [omni.usd] inputs:diffuse_color_constant already exists but type is mismatched. Removing old type (SdfAssetPath) and creating new one (GfVec3f)
2020-12-23 11:53:20 [18,079ms] [Error] [omni.client.plugin]  Main: usd_plugin: Type mismatch for </World/Env/Tiles/Tile_22/Looks/White_material/White_material.inputs:diffuse_color_constant>: expected 'SdfAssetPath', got 'GfVec3f'

And finally this error, where I am a bit puzzled that it says ERROR_OUT_OF_DEVICE_MEMORY:

2020-12-23 11:53:23 [21,170ms] [Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_DEVICE_MEMORY
2020-12-23 11:53:23 [21,170ms] [Error] [carb.graphics-vulkan.plugin] vkAllocateMemory failed.
2020-12-23 11:53:23 [21,170ms] [Error] [rtx.resourcemanager.plugin] Texture creation failed for the device: 0.
Fatal Python error: Segmentation fault

Current thread 0x00007f80b812c340 (most recent call first):
  File "/home/robotlab/Downloads/isaac-sim-2020.2.2007-linux-x86_64-release/_build/linux-x86_64/release/exts/omni.isaac.synthetic_utils/omni/isaac/synthetic_utils/scripts/omnikit.py", line 195 in update
  File "/home/robotlab/Downloads/isaac-sim-2020.2.2007-linux-x86_64-release/python_samples/jetbot/jetbot_env.py", line 112 in step
  File "/home/robotlab/Downloads/isaac-sim-2020.2.2007-linux-x86_64-release/python_samples/jetbot/jetbot_env.py", line 158 in reset
  File "/home/robotlab/anaconda3/envs/isaac-sim/lib/python3.6/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 59 in reset
  File "/home/robotlab/anaconda3/envs/isaac-sim/lib/python3.6/site-packages/stable_baselines3/common/vec_env/vec_transpose.py", line 60 in reset
  File "/home/robotlab/anaconda3/envs/isaac-sim/lib/python3.6/site-packages/stable_baselines3/common/base_class.py", line 477 in _setup_learn
  File "/home/robotlab/anaconda3/envs/isaac-sim/lib/python3.6/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 208 in learn
  File "/home/robotlab/anaconda3/envs/isaac-sim/lib/python3.6/site-packages/stable_baselines3/ppo/ppo.py", line 264 in learn
  File "jetbot_train.py", line 49 in train
  File "jetbot_train.py", line 83 in <module>
Segmentation fault (core dumped)

Isaac Sim opens briefly, loads the assets and the errors occur. Any advice or ideas how to get it running?

System:
Isaac Sim 2020.2.2007
Ubuntu 18.04
Quadro RTX3000, driver version 455.38
i7-10875H CPU @ 2.30GHz × 16
32 GB RAM

2

The jetracer sample works btw, but still not the jetbot.

3

RTX3000 has 6GB of memory i believe. It’s best to use at least 8GB card like RTX 2080, as recommended in the Isaac Sim docs.

Meanwhile, could you try reducing the PPO’s batch_size to 64 or 32 first and run again? Then reducing the images’ resolution, etc.

    model = PPO(
        "CnnPolicy",
        env,
        verbose=1,
        tensorboard_log=args.tensorboard_dir,
        policy_kwargs=policy_kwargs,
        device="cuda",
        n_steps=args.step_freq,
        batch_size=2048,
        n_epochs=50,
        learning_rate=0.0001,
    )

CUSTOM_CONFIG = {
“width”: 224,
“height”: 224,

Remember to match the lower resolution numbers with the network’s architecture
self.observation_space = spaces.Box(low=0, high=255, shape=(224, 224, 6), dtype=np.uint8)