Usage of Camera API in headless mode

Hello! I’m currently working on a use case that requires using the Camera API of IsaacGym. However, the use case also requires the code to be run in headless mode. However, when I try to create a camera sensor, I get the following error:


[Error] [carb.gym.plugin] SetCameraLocation: Error: could not find camera with handle -1 in environment 0

For reference, I’m using IsaacGym Preview 2 on Ubuntu 18.04.5 LTS. Here is the code snippet that I’m talking about - any advice is preferred!


cam_props = gymapi.CameraProperties()

cam_props.width = self.cam_width

cam_props.height = self.cam_height

cam_props.enable_tensors = True

cam_handle = self.gym.create_camera_sensor(env, cam_props) # This is the line that has the error

print(f"Cam Handle: {cam_handle}") # Prints -1

self.gym.set_camera_location(cam_handle, env, gymapi.Vec3(1.2, 0, 2.5), gymapi.Vec3(0, 0, 1.2))

self.cams.append(cam_handle)

# obtain camera tensor

cam_tensor = self.gym.get_camera_image_gpu_tensor(self.sim, env, cam_handle, gymapi.IMAGE_COLOR)

# wrap camera tensor in a pytorch tensor

torch_cam_tensor = gymtorch.wrap_tensor(cam_tensor)

self.cam_tensors_np.append(torch_cam_tensor)

print("Got camera tensor with shape", cam_tensor.shape)

print("  Torch camera tensor device:", torch_cam_tensor.device)

print("  Torch camera tensor shape:", torch_cam_tensor.shape)

1 Like

Hi, your code looks very similar to what we have in our interop_torch.py example, which should work in headless mode. Can you confirm whether you are able to run that example in headless mode? You can use python interop_torch.py --headless.

Thanks for the reply! I tried running that command and unfortunately got the below error:

Importing module 'gym_37' (<path-to-directory>/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_37.so)
Setting GYM_USD_PLUG_INFO_PATH to <path-to-directory>/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.8.1+cu102
Device count 1
<path-to-directory>isaacgym/python/isaacgym/_bindings/src/gymtorch
Using<path-to-home>/varunl/.cache/torch_extensions as PyTorch extensions root...
Emitting ninja build file <path-to-home>/varunl/.cache/torch_extensions/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Segmentation fault

Is there anything that I’m doing immediately wrong?

For the record, the crash appears to be happening when sim = gym.create_sim(args.compute_device_id, args.graphics_device_id, args.physics_engine, sim_params) is called. Here, both the compute and graphics device IDs are 0, the physics engine is PhysX, and the sim params are as default. The code is running on a Tesla P100-PCIE-12GB which has been verified to work with cuda by using torch.cuda.is_available()

Hello,

i have the same issue with one of our machines, which has two NVIDIA Quadro P5000. The interop_torch.py example causes segmentation faults, as soon as create_sim() is called. Curiously enough, the example works fine with a much smaller machine (laptop with a NVIDIA T500). I hope you can shed some light into the problem.

Hi there,

Having more than one GPU in the machine could the cause of the issue. Does it work for you if you force it to just have 1 GPU available with set CUDA_VISIBLE_DEVICES?

Hi, I have encountered the same problem. I run isaacgym preview 4 on a remote server with ubuntu 18.04 and driver version 470.57.02

I have set CUDA_VISIBLE_DEVICES to 0, but interpo_torch.py still crashed with segmentation fault.

Hello, I am running into this problem right now. When I run create_sim with graphics_device_id=-1 then I the error: SetCameraLocation: Error: could not find camera with handle -1 in environment 0 , if I run it with any other ID set then I get a segmentation fault.
I also realized that if I set cam_props.enable_tensors=False then I don’t get a segmentation fault anymore.
Oh also changing CUDA_VISIBLE_DEVICES didn’t really make any difference.
Is there any solution to this problem, any help is very much appreciated!