Usage of Camera API in headless mode

Hello! I’m currently working on a use case that requires using the Camera API of IsaacGym. However, the use case also requires the code to be run in headless mode. However, when I try to create a camera sensor, I get the following error:


[Error] [carb.gym.plugin] SetCameraLocation: Error: could not find camera with handle -1 in environment 0

For reference, I’m using IsaacGym Preview 2 on Ubuntu 18.04.5 LTS. Here is the code snippet that I’m talking about - any advice is preferred!


cam_props = gymapi.CameraProperties()

cam_props.width = self.cam_width

cam_props.height = self.cam_height

cam_props.enable_tensors = True

cam_handle = self.gym.create_camera_sensor(env, cam_props) # This is the line that has the error

print(f"Cam Handle: {cam_handle}") # Prints -1

self.gym.set_camera_location(cam_handle, env, gymapi.Vec3(1.2, 0, 2.5), gymapi.Vec3(0, 0, 1.2))

self.cams.append(cam_handle)

# obtain camera tensor

cam_tensor = self.gym.get_camera_image_gpu_tensor(self.sim, env, cam_handle, gymapi.IMAGE_COLOR)

# wrap camera tensor in a pytorch tensor

torch_cam_tensor = gymtorch.wrap_tensor(cam_tensor)

self.cam_tensors_np.append(torch_cam_tensor)

print("Got camera tensor with shape", cam_tensor.shape)

print("  Torch camera tensor device:", torch_cam_tensor.device)

print("  Torch camera tensor shape:", torch_cam_tensor.shape)

Hi, your code looks very similar to what we have in our interop_torch.py example, which should work in headless mode. Can you confirm whether you are able to run that example in headless mode? You can use python interop_torch.py --headless.

Thanks for the reply! I tried running that command and unfortunately got the below error:

Importing module 'gym_37' (<path-to-directory>/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_37.so)
Setting GYM_USD_PLUG_INFO_PATH to <path-to-directory>/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.8.1+cu102
Device count 1
<path-to-directory>isaacgym/python/isaacgym/_bindings/src/gymtorch
Using<path-to-home>/varunl/.cache/torch_extensions as PyTorch extensions root...
Emitting ninja build file <path-to-home>/varunl/.cache/torch_extensions/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Segmentation fault

Is there anything that I’m doing immediately wrong?

For the record, the crash appears to be happening when sim = gym.create_sim(args.compute_device_id, args.graphics_device_id, args.physics_engine, sim_params) is called. Here, both the compute and graphics device IDs are 0, the physics engine is PhysX, and the sim params are as default. The code is running on a Tesla P100-PCIE-12GB which has been verified to work with cuda by using torch.cuda.is_available()