GPU resource capacity issues when acquiring point cloud data in Isaac Sim


I am working on training an RL model in the Omniverse Isaac Sim Gym Environment (OGIE) with a point cloud as input. When training with point clouds in my training environment, it seems to be using too much GPU computational resources. I have created 4 environments and about 2GB GPU memory is being used. I am using an RTX 3090 GPU. The point clouds are being acquired from 3 cameras per environment using replicator.

There seems to be a definite problem when comparing the two.

  1. When my colleague used Isaac Gym to train with point clouds, it was not a problem to create more than 100 environments (using more than 2 cameras in each environment).
  2. When I trained with OIGE using a given coordinate of an object attached to the robot rather than point cloud input, I was able to train even after creating 2048 environments.

When using point clouds in OIGE, how can I use GPU memory more efficiently?

The code that I used is the same as my previous question. (link)

Is there any solution to this?

Hi @psh9002 - Devs are busy with GTC related activities. I will request them to get back to you as soon as they can.

1 Like

Does anyone solve this problem?