I meet a bug when using Isaac Sim version 4.2.0.2. I installed it using pip.
I have a simple script to load the Franka robot into the scene and control the joint position, joint velocity and joint effort using the function apply_action under the Articulation class. The script worked fine when using CPU, However, when I changed to GPU mode, it output errors, especially when I specified the value of joint_velocities and joint_efforts.
Here is the code I use:
from isaacsim import SimulationApp
simulation_app = SimulationApp({"headless": True})
import torch
import numpy as np
from omni.isaac.core import World
from omni.isaac.franka import Franka
from omni.isaac.core.utils.types import ArticulationAction
def main():
device = 'cuda'
world = World(device='cuda', backend='torch')
world.scene.add_default_ground_plane()
franka = world.scene.add(Franka(prim_path="/World/Fancy_Franka", name="fancy_franka"))
world.reset()
while simulation_app.is_running():
joint_positions = torch.tensor(np.random.uniform(0.0, 1.0, size=9)).to(device)
joint_velocities = torch.tensor(np.random.uniform(0.0, 1.0, size=9)).to(device)
joint_efforts = torch.tensor(np.random.uniform(0.0, 1.0, size=9)).to(device)
action = ArticulationAction(joint_positions, joint_velocities, joint_efforts)
franka.apply_action(action)
world.step()
simulation_app.close()
if __name__ == "__main__":
main()
When I run the code, I get this error: TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
Potential Solution
In file isaacsim/exts/omni.isaac.core/omni/isaac/core/controllers/articulation_controller.py
At line 75, change from if joint_velocities[0][i] is None or np.isnan(joint_velocities[0][i]): to if joint_velocities[0][i] is None or np.isnan(self._articulation_view._backend_utils.to_numpy(joint_velocities[0][i])):
At line 84, change from if joint_efforts[0][i] is None or np.isnan(joint_efforts[0][i]): to joint_efforts[0][i] is None or np.isnan(self._articulation_view._backend_utils.to_numpy(joint_efforts[0][i])):
After this, my problem was solved.
If this is not a bug, can anyone help me solve this properly?