Why it returns -1 when I tried to create_camera_sensor?

zdxiaohuihui · June 18, 2022, 11:24am

I use self.gtm.create_camera_sensor(env_handle, self.cam_props) to create a camera sensor but it returns -1, so I can’t set the camera pos and get the image. I have checked the env_handle and it’s normal. I am wondering what could be the potential reason because it seems I just add the sensor as the example shows…

When I try to attact camera to body, it shows could not find camera with handle -1 in environment 0

Here is my code

local_offset = gymapi.Vec3(0.0, 0.0, 0.05)
cam_pitch = np.random.uniform(25, 35)
local_rotation = gymapi.Quat.from_axis_angle(gymapi.Vec3(0, 1, 0), np.deg2rad(cam_pitch))
camera_base_handle = self.gym.find_actor_rigid_body_handle(env_handle, robot_handle, “base”)
cam_handle = self.gym.create_camera_sensor(env_handle, self.cam_props)
self.camera_handles.append(cam_handle)
self.gym.attach_camera_to_body(
cam_handle, env_handle, camera_base_handle, gymapi.Transform(local_offset, local_rotation),
gymapi.FOLLOW_TRANSFORM
)

toni.sm · June 18, 2022, 4:09pm

Hi @zdxiaohuihui

Make sure that the graphics_device is correctly configured (and it has a value different that -1) in gym.create_sim(compute_device, graphics_device, physics_engine, sim_params)

zdxiaohuihui · June 18, 2022, 5:35pm

Ah that works. I set the graphics_device to be -1 when training because it’s headless. Thank you so much!
But I want to use the camera data during training while not rendering the environment as it will make the training quite slow. Would this be possible?

toni.sm · June 18, 2022, 7:34pm

Glad to hear it’s working…

As for the configuration, it is possible to configure the cameras (by setting the graphics_device) without creating/enabling the viewer.

For a reference, you can check the isaacgymenvs/tasks/base/vec_task.py file…
Particularly…

The configuration of the graphics_device

github.com

NVIDIA-Omniverse/IsaacGymEnvs/blob/9656bac7e59b96382d2c5040b90d2ea5c227d56d/isaacgymenvs/tasks/base/vec_task.py#L79-L82


      
          enable_camera_sensors = config.get("enableCameraSensors", False)
          self.graphics_device_id = graphics_device_id
          if enable_camera_sensors == False and self.headless == True:
              self.graphics_device_id = -1

The creation of the viewer

github.com

NVIDIA-Omniverse/IsaacGymEnvs/blob/9656bac7e59b96382d2c5040b90d2ea5c227d56d/isaacgymenvs/tasks/base/vec_task.py#L204-L231


      
          def set_viewer(self):
              """Create the viewer."""
          
          
    # todo: read from config
              self.enable_viewer_sync = True
              self.viewer = None
          
          
    # if running with a viewer, set up keyboard shortcuts and camera
              if self.headless == False:
                  # subscribe to keyboard shortcuts
                  self.viewer = self.gym.create_viewer(
                      self.sim, gymapi.CameraProperties())
                  self.gym.subscribe_viewer_keyboard_event(
                      self.viewer, gymapi.KEY_ESCAPE, "QUIT")
                  self.gym.subscribe_viewer_keyboard_event(
                      self.viewer, gymapi.KEY_V, "toggle_viewer_sync")
          
          
        # set the camera position based on up axis
                  sim_params = self.gym.get_sim_params(self.sim)
                  if sim_params.up_axis == gymapi.UP_AXIS_Z:

This file has been truncated. show original

And, the rendering

github.com

NVIDIA-Omniverse/IsaacGymEnvs/blob/9656bac7e59b96382d2c5040b90d2ea5c227d56d/isaacgymenvs/tasks/base/vec_task.py#L386-L414


      
          def render(self):
              """Draw the frame to the viewer, and check for keyboard events."""
              if self.viewer:
                  # check for window closed
                  if self.gym.query_viewer_has_closed(self.viewer):
                      sys.exit()
          
          
        # check for keyboard events
                  for evt in self.gym.query_viewer_action_events(self.viewer):
                      if evt.action == "QUIT" and evt.value > 0:
                          sys.exit()
                      elif evt.action == "toggle_viewer_sync" and evt.value > 0:
                          self.enable_viewer_sync = not self.enable_viewer_sync
          
          
        # fetch results
                  if self.device != 'cpu':
                      self.gym.fetch_results(self.sim, True)
          
          
        # step graphics
                  if self.enable_viewer_sync:

This file has been truncated. show original

As you can identify, the graphics_device is set to -1 only if you don’t want to use camera sensors and want to run in headless mode… While the viewer is only created and used if the headless flag is set to False

zdxiaohuihui · June 18, 2022, 8:06pm

So I can train without viewer by setting graphics_device_id not to be -1 while run in headless mode, but anyway I need to render if I want to use camera sensors. I see. Thank you so much for your kind reply!

Topic		Replies	Views
Error in creating camera sensor handle while headless Isaac Gym camera , gym	2	921	January 17, 2023
Usage of Camera API in headless mode Isaac Gym camera , headless	6	2607	December 11, 2023
Camera example and headless mode Isaac Gym camera , headless	11	6520	July 12, 2023
Error using camera in headless mode Isaac Gym camera , headless	0	1064	February 3, 2022
Rlgpu frame headless mode's cameras are different with graphical mode Isaac Gym	0	10	July 25, 2024
Gym.render_all_camera_sensors(sim) always causes segmentation fault (simple script to reproduce) Isaac Gym	2	933	September 23, 2024
Get camera image from gym in headless mode Isaac Sim	5	1651	June 22, 2023
Omniverse ISAAC Gym - Access camera stream Isaac Gym	10	3234	October 26, 2023
Create camera sensor fail on buffer Isaac Gym camera	5	2761	October 12, 2021
cudaExternamMemoryGetMappedBuffer failed when creating camera IsaacGym Isaac Gym	0	32	March 1, 2025

Why it returns -1 when I tried to create_camera_sensor?

Related topics