Hi! I’m actually find some problem running Isaac Gym. I got a nvidia 2070, windows 11 (so there is no problem running graphics application), but when I start an example In python i got:
*** Warning: failed to preload CUDA lib
*** Warning: failed to preload PhysX libs
Importing module ‘gym_38’ (/home/enne/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/enne/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
WARNING: Forcing CPU pipeline.
Not connected to PVD
/buildAgent/work/f3416cf82e3cf1ba/source/physx/src/gpu/PxPhysXGpuModuleLoader.cpp (147) : internal error : libcuda.so!
[Warning] [carb.gym.plugin] Failed to create a PhysX CUDA Context Manager. Falling back to CPU.
Physics Engine: PhysX
Physics Device: cpu
GPU Pipeline: disabled
No GPU devices found.
[Error] [carb.gym.plugin] Failed to create Nvf device in createNvfGraphics. Please make sure Vulkan is correctly installed.
*** Failed to create sim
If i run nvidia-smi in got correctly my graphic card, with a Driver Version: 510.10 and CUDA Version: 11.6.
Is there any incompatibility or some steps are missing?
python3 joint_monkey.py
Importing module ‘gym_37’ (/home/kaykay/Downloads/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_37.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/kaykay/Downloads/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
WARNING: Forcing CPU pipeline.
[Error] [carb.gym.plugin] Sim CUDA device 0 can’t be set, the total number of available devices is -1
Not connected to PVD
/buildAgent/work/f3416cf82e3cf1ba/source/cudamanager/src/CudaContextManager.cpp (404) : warning : cuInit failed
[Warning] [carb.gym.plugin] Failed to create a valid PhysX CUDA Context Manager. Falling back to CPU.
Physics Engine: PhysX
Physics Device: cpu
GPU Pipeline: disabled
[Error] [carb.gym.plugin] Gym cuda error: no CUDA-capable device is detected: …/…/…/source/plugins/carb/gym/impl/Gym/GymCuda.h: 110
[Error] [carb.gym.plugin] Failed to create primary CUDA context
[Warning] [carb.gym.plugin] Failed to create primary CUDA context on graphics device
No GPU devices found.
[Error] [carb.gym.plugin] Failed to create Nvf device in createNvfGraphics. Please make sure Vulkan is correctly installed.
*** Failed to create sim
nvidia-smi gives this:
nvidia-smi
Sun Oct 24 17:54:21 2021
±----------------------------------------------------------------------------+
| NVIDIA-SMI 510.00 Driver Version: 510.06 CUDA Version: 11.6 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce … On | 00000000:01:00.0 Off | N/A |
| N/A 54C P8 4W / N/A | 220MiB / 6144MiB | N/A Default |
| | | N/A |
±------------------------------±---------------------±---------------------+
±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------+
I was hoping that that would not be a problem with wsl2 (since it is basically a virtual machine, with GPU passthrough and GUI - in my case of Ubuntu 20.04). As far I understood there is some Vulkan problem in wsl2 and maybe this is the motivation of this problem
We have not tested Isaac Gym in a virtual machine or wsl2 so I can’t say for sure what issues may arise. Vulkan is generally required for rendering, if your use case doesn’t require rendering, you could try running one of the examples in headless mode and see if that works for you. From the error messages posted in this thread, it mostly looks like the PhysX backend was not able to find the correct CUDA binaries, or it couldn’t find any available GPU devices, so it’s possible that these things are not being mapped correctly in the virtual environments.
I am getting the same error. I’m using Ubuntu 18.04 on an ec2 g4dn.2xlarge instance.
$ python3.8 joint_monkey.py
Importing module 'gym_38' (/home/ubuntu/.local/lib/python3.8/site-packages/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/ubuntu/.local/lib/python3.8/site-packages/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
WARNING: Forcing CPU pipeline.
[Error] [carb.gym.plugin] Sim CUDA device 0 can't be set, the total number of available devices is -1
Not connected to PVD
/buildAgent/work/45f70df4210b2e3e/source/cudamanager/src/CudaContextManager.cpp (404) : warning : cuInit failed
[Warning] [carb.gym.plugin] Failed to create a valid PhysX CUDA Context Manager. Falling back to CPU.
Physics Engine: PhysX
Physics Device: cpu
GPU Pipeline: disabled
[Error] [carb.gym.plugin] Gym cuda error: no CUDA-capable device is detected: ../../../source/plugins/carb/gym/impl/Gym/GymCuda.h: 110
[Error] [carb.gym.plugin] Failed to create primary CUDA context
[Warning] [carb.gym.plugin] Failed to create primary CUDA context on graphics device
$ nvidia-smi
Wed Apr 6 15:58:12 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.103.01 Driver Version: 470.103.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:1E.0 Off | 0 |
| N/A 49C P0 28W / 70W | 0MiB / 15109MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
torch has access to cuda, so I’m not sure what’s going wrong
$ python3.8
Python 3.8.12 (default, Oct 12 2021, 13:49:34)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True