Why does the gym environment occupy some amount of GPU memory on each visible GPU?

Robotics & Edge Computing Robotics - Isaac Isaac Gym
1

Hi,

If I use ISAAC GYM on one GPU, it seems that it will spawn something on all the other available gpus and consumes some memory on other GPUs. In this case, the available GPU memory on other gpus are reduced. Is there a way to circumvent this issue?

For example:

image
image1436×323 44.6 KB

I am running the job on GPU 0, but some amount of GPU memory is also consumed on the other GPUs.

2

Hi @xt02348,

At the launch of a task in Isaac Gym, we introduce a CUDA context on each available GPU device to ensure we have a valid CUDA context on all of the devices. This could be the reason why you are seeing GPU memory usage for all of your GPUs. If you’d like to limit the usage to a single GPU, you can use export CUDA_VISIBLE_DEVICES to specify the GPU to use.

Thanks,
Kelly