Usage of Camera API in headless mode

Hello! I’m currently working on a use case that requires using the Camera API of IsaacGym. However, the use case also requires the code to be run in headless mode. However, when I try to create a camera sensor, I get the following error:

[Error] [carb.gym.plugin] SetCameraLocation: Error: could not find camera with handle -1 in environment 0

For reference, I’m using IsaacGym Preview 2 on Ubuntu 18.04.5 LTS. Here is the code snippet that I’m talking about - any advice is preferred!

cam_props = gymapi.CameraProperties()

cam_props.width = self.cam_width

cam_props.height = self.cam_height

cam_props.enable_tensors = True

cam_handle = self.gym.create_camera_sensor(env, cam_props) # This is the line that has the error

print(f"Cam Handle: {cam_handle}") # Prints -1

self.gym.set_camera_location(cam_handle, env, gymapi.Vec3(1.2, 0, 2.5), gymapi.Vec3(0, 0, 1.2))


# obtain camera tensor

cam_tensor = self.gym.get_camera_image_gpu_tensor(self.sim, env, cam_handle, gymapi.IMAGE_COLOR)

# wrap camera tensor in a pytorch tensor

torch_cam_tensor = gymtorch.wrap_tensor(cam_tensor)


print("Got camera tensor with shape", cam_tensor.shape)

print("  Torch camera tensor device:", torch_cam_tensor.device)

print("  Torch camera tensor shape:", torch_cam_tensor.shape)

Hi, your code looks very similar to what we have in our example, which should work in headless mode. Can you confirm whether you are able to run that example in headless mode? You can use python --headless.

Thanks for the reply! I tried running that command and unfortunately got the below error:

Importing module 'gym_37' (<path-to-directory>/isaacgym/python/isaacgym/_bindings/linux-x86_64/
Setting GYM_USD_PLUG_INFO_PATH to <path-to-directory>/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.8.1+cu102
Device count 1
Using<path-to-home>/varunl/.cache/torch_extensions as PyTorch extensions root...
Emitting ninja build file <path-to-home>/varunl/.cache/torch_extensions/gymtorch/
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Segmentation fault

Is there anything that I’m doing immediately wrong?

For the record, the crash appears to be happening when sim = gym.create_sim(args.compute_device_id, args.graphics_device_id, args.physics_engine, sim_params) is called. Here, both the compute and graphics device IDs are 0, the physics engine is PhysX, and the sim params are as default. The code is running on a Tesla P100-PCIE-12GB which has been verified to work with cuda by using torch.cuda.is_available()