Hello,
I have been playing with the teleoperation example in isaac orbit (teleop_se3_agent.py) and used this example to integrate the teleoperation into my own Cartesian impedance controller. So far, I can move the robot with the spacemouse but I have issues with the gripper command as I don’t see clearly how to handle the input bool signal. The code provided from the teleoperation excample is the following:
def pre_process_actions(delta_pose: torch.Tensor, gripper_command: bool) -> torch.Tensor:
"""Pre-process actions for the environment."""
# compute actions based on environment
if "Reach" in args_cli.task:
# note: reach is the only one that uses a different action space
# compute actions
return delta_pose
else:
# resolve gripper command
gripper_vel = torch.zeros(delta_pose.shape[0], 1, device=delta_pose.device)
gripper_vel[:] = -1.0 if gripper_command else 1.0
# compute actions
return torch.concat([delta_pose, gripper_vel], dim=1)
I see that the gripper is commanded using a velocity command but I don’t see how is handled this command by the controller in the example. I see that the example uses DifferentialIKControllerto control the robot but I dont see how handles the velocity command for the robot gripper as the environment is wrapped into a gym, and the command is saved into a action variable that is stepped by the simulation in the main loop.
# simulate environment
while simulation_app.is_running():
# run everything in inference mode
with torch.inference_mode():
# get keyboard command
delta_pose, gripper_command = teleop_interface.advance()
delta_pose = delta_pose.astype("float32")
print(gripper_command)
# convert to torch
delta_pose = torch.tensor(delta_pose, device=env.unwrapped.device).repeat(env.unwrapped.num_envs, 1)
# pre-process actions
actions = pre_process_actions(delta_pose, gripper_command)
# apply actions
env.step(actions)
My own environment is a basic envs.BaseEnv, and all I need is to somehow open/close the gripper without entering into any actions or gym stuff.
Can someone give me any insight in how to overcome this problem?