Isaac Gym: Error while running train.py file

Hello everyone,

I have problems with running python train.py files in the isaacgymenvs folder in Ubuntu 20.04. I will try to explain this as clear as possible.

When running nvidia-smi I obtain the following information

I have downloaded the Isaac Gym Preview 4 and I used the create_conda_env_rl_rlgpu.sh to setup the virtual env. After doing this I reinstalled Torch and the Cuda Toolkit as following:
conda install pytorch==1.8.1 torchvision==0.9.1 torchaudio==0.8.1 cudatoolkit=10.2 -c pytorch

Meaning that I have a bit older version of Torch and Cuda toolkit, but this works on my driver and GPU. I tested this virtual env with a python script which can be seen in the figure below with the output also displayed

This output tells me that a torch tensor can be made and can be placed on my GPU. This meant that I thought that the installation worked and I could train via my GPU.

When I cd to the folder: /IsaacGym/isaacgym/python/examples and run the command: python 1080_balls_of_solitude.py IsaacGym works and I see the simulation.

But, when I cd to the correct folder: /IsaacGym/IsaacGymEnvs/isaacgymenvs, and run the python train.py task=Cartpole I obtain the following error: [Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: …/…/…/source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 991

I think this error has something do to with the CUDA kernel not being correctly compiled within the Isaac Gym install. I want it to target sm_50, but I am not sure if it is doing that.

Maybe I am completely wrong and I am doing something else totally wrong, but if there is someone with an expertise in this, please let me know and help me with the problem. Also if you need more information, please let me know.

@l.l.a.v.bree i am probably do equipped to provide the correct answer, but something feels like it’s related to the GPU and the driver version you are using per your nvidia smi. but i’ll defer to the mods/devs.