Error when initializing Franka robot on cuda device

Hi,
I’m encountering an issue with initializing a Franka robot. When I initialize the world with device=“cpu” and backend=“numpy”, everything functions correctly (commented out). However, when I switch to GPU with device=“cuda:0” and backend=“torch”, I encounter errors shown below. Here is a simplified code sample to reproduce the error:

from omni.isaac.kit import SimulationApp


# Simple example showing how to start and stop the helper
simulation_app = SimulationApp({"headless": False})

### Perform any omniverse imports here after the helper loads ###
from omni.isaac.core.world import World
from omni.isaac.core.objects.ground_plane import GroundPlane
from omni.isaac.franka import Franka
import omni
import numpy as np

stage = omni.usd.get_context().get_stage()

world = World(stage_units_in_meters=1.0, 
        rendering_dt=1.0/60.0, backend="torch", device="cuda:0",)


#world = World(stage_units_in_meters=1.0, 
#        rendering_dt=1.0/60.0, backend="numpy", device="cpu",)
GroundPlane(prim_path="/World/groundPlane", size=10, color=np.array([0.5, 0.5, 0.5]))

world.reset()
omni.timeline.get_timeline_interface().play()

# spawn robot
franka = Franka(prim_path="/World/Franka", name="franka")
simulation_app.update()
franka.initialize()

while(True):
    simulation_app.update()  # Render a single frame

Running this code I get the following error messages:

2024-02-16 09:16:04 [10,949ms] [Error] [omni.physx.plugin] PhysX error: PxArticulationReducedCoordinate::updateKinematic(): it is illegal to call this method if PxSceneFlag::eENABLE_DIRECT_GPU_API is enabled!, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/physx/src/NpArticulationReducedCoordinate.cpp, LINE 691
2024-02-16 09:16:04 [10,949ms] [Warning] [omni.physx.plugin] Detected an articulation at /World/Franka with more than 4 velocity iterations being added to a TGS scene.The related behavior changed recently, please consult the changelog. This warning will only print once.
2024-02-16 09:16:04 [11,051ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuEventRecord failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 53
2024-02-16 09:16:04 [11,052ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,052ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuStreamWaitEvent failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 59
2024-02-16 09:16:04 [11,052ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,052ms] [Error] [omni.physx.plugin] PhysX error: memcpy failed fail!
  700, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpunarrowphase/src/PxgNarrowphaseCore.cpp, LINE 1447
2024-02-16 09:16:04 [11,053ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,053ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuEventRecord failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 53
2024-02-16 09:16:04 [11,053ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,054ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuStreamWaitEvent failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 59
2024-02-16 09:16:04 [11,054ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,054ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuEventRecord failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 53
2024-02-16 09:16:04 [11,054ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,054ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuStreamWaitEvent failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 59
2024-02-16 09:16:04 [11,055ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,055ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuEventRecord failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 75
2024-02-16 09:16:04 [11,055ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,055ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuStreamWaitEvent failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 81
2024-02-16 09:16:04 [11,055ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,056ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuEventRecord failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 75
2024-02-16 09:16:04 [11,056ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,056ms] [Error] [omni.physx.plugin] PhysX error: SynchronizeStreams cuStreamWaitEvent failed with error 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpucommon/include/PxgCudaUtils.h, LINE 81
2024-02-16 09:16:04 [11,056ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,056ms] [Error] [omni.physx.plugin] PhysX error: GPU solveContactParallel fail to launch kernel!!
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpusolver/src/PxgTGSCudaSolverCore.cpp, LINE 1463
2024-02-16 09:16:04 [11,056ms] [Error] [omni.physx.plugin] Cuda context manager error, simulation will be stopped and new cuda context manager will be created.
2024-02-16 09:16:04 [11,057ms] [Error] [omni.physx.plugin] PhysX error: GPU initializeSolverVelocitiesTGS fail to launch kernel!! 700
, FILE /buildAgent/work/eb2f45c4acc808a0/physx/source/gpusolver/src/PxgTGSCudaSolverCore.cpp, LINE 1500

Thank you in advance for your answer.

@mabraun i am just another user passing by, do you mind pointing to the Isaac Sim version you are using? and help the mods/devs with troubleshooting by providing more context, i.e. uploading the .log in its entirety?

you can find the latest .log file in directory mentioned in section 1:

https://docs.omniverse.nvidia.com/isaacsim/latest/installation/install_faq.html#common-path-locations

Hi,
of course, I use the latest Isaac Sim Version 2023.1.1 but I had the same issue when using 2023.1.0-hotfix. You can find the latest log file attached.
kit_20240215_155945.log (1.2 MB)