Error in creating camera sensor handle while headless

toni.sm · January 17, 2023, 8:35pm

The problem comes from vec_task.py line 89

NVIDIA-Omniverse/IsaacGymEnvs/blob/dee7c56765e14f6f4344c4d2e91d7a9eb3bfa619/isaacgymenvs/tasks/base/vec_task.py#L89


      
              else:
                  print("GPU Pipeline can only be used with GPU simulation. Forcing CPU Pipeline.")
                  config["sim"]["use_gpu_pipeline"] = False
          
          
self.rl_device = rl_device
          
          
# Rendering
          # if training in a headless mode
          self.headless = headless
          
          
enable_camera_sensors = config.get("enableCameraSensors", False)
          self.graphics_device_id = graphics_device_id
          if enable_camera_sensors == False and self.headless == True:
              self.graphics_device_id = -1
          
          
self.num_environments = config["env"]["numEnvs"]
          self.num_agents = config["env"].get("numAgents", 1)  # used for multi-agent environments
          self.num_observations = config["env"]["numObservations"]
          self.num_states = config["env"].get("numStates", 0)
          self.num_actions = config["env"]["numActions"]

The configuration for enabling camera sensors (enableCameraSensors) in Isaac Gym environments is defined at task.env.enableCameraSensors however, the configuration in the script is looking at task.enableCameraSensors.

Because this last sub-key does not exist, enable_camera_sensors = config.get("enableCameraSensors", False) is always returning False. Then, when headless is True, the final graphics_device_id is -1

To create and set the sub-key, when running the script, use +task.enableCameraSensors=True

Example:

python train.py task=Cartpole headless=True +task.enableCameraSensors=True

Or edit the vec_task.py line 89

enable_camera_sensors = config["env"].get("enableCameraSensors", False)

and use the original configuration sub-key:

Example:

python train.py task=Cartpole headless=True task.env.enableCameraSensors=True