The problem
When I run isaacgym/python/examples/apply_forces.py, I get the following errors:
[Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 926
[Error] [carb.gym.plugin] Failed to fill rigid body state tensor
What I’ve tried
Downgrading Cuda (I was running Cuda 11.5 before). Downgraded to 11.3. No change.
Removing the cached build folder under ~/.cache/torch_extensions/py38_cu113/gymtorch/ and rebuilding the extension. No change.
Creating a new Conda environment with only python=3.8, torch=1.10.0+cu113 and isaacgym, then removing the extension folder to force rebuilding. No change.
Compiling pytorch from source (took about 2 hours), in a brand-new Conda environment, then removing the extension folder to force rebuilding. No change.
EDIT: The following is running fine on my other machine: Ubuntu 20.04.3, Nvidia RTX 2080 Ti, same pytorch version (1.10.3+cu113), driver 495.29.05, Cuda 11.5.
Is there anything else I can try?
Any help would be appreciated. My next approach would be to nuke the OS and install Ubuntu 20.04 LTS but I was hoping to maybe not have to do that just for IsaacGym.
My friend is running it fine on a GTX 1080 Ti card, so maybe that’s not the issue? Maybe it’s the compute capability? Titan X has cc 5.2 and GTX 1080 and anything newer has cc 6.1 (source: https://developer.nvidia.com/cuda-gpus#compute)
good point
However, I would try with cuda 10.2
technical requirements for Gym seem underdocumented :
I was not able to find exact GPU hardware/ Cuda version list supported
But e.g. in Jetson ecosystem Isaac Sim only supports cuda 10.2
However, I can see a similar issue here
@fgolemo
It appears it also may be a pytorch version issue that doesn’t support specific SM
you may try pytorch docker container maybe
or build pytorch from source
or you may try this
The error above is in the Gym PhysX CUDA code, rather than PyTorch - I’m afraid that we only support Pascal GPUs and higher. Your TITAN X and GTX 750 cards are both Maxwell class cards.
@_av - there is a short requirements list on the download page, but it looks like we don’t include required GPU version. Only software needs are provided. I will request an update to that area.