Hi, I am using Isaac Sim, 2022.1.1. I tried to run the tutorial example but I found that it does not work properly. Also the size of the cube is not the same as what we see in the tutorial video.
Please let me know what I am doing wrong.
Thanks!
Hi, I am using Isaac Sim, 2022.1.1. I tried to run the tutorial example but I found that it does not work properly. Also the size of the cube is not the same as what we see in the tutorial video.
Hi Leila,
Thanks for reporting this issue. The problem here is the default size of the cube which will be fixed in the next release and the default gripper position. This can be solved by specifying the size of the cube to be 1 as well as explicitly setting the gripper positions to be opened on reset and on load.
Here is the modified script:
from omni.isaac.examples.base_sample import BaseSample
from omni.isaac.franka import Franka
from omni.isaac.core.objects import DynamicCuboid
from omni.isaac.franka.controllers import PickPlaceController
from omni.isaac.core.tasks import BaseTask
import numpy as np
class FrankaPlaying(BaseTask):
# NOTE: we only cover here a subset of the task functions that are available,
# checkout the base class for all the available functions to override.
# ex: calculate_metrics, is_done..etc.
def __init__(self, name):
super().__init__(name=name, offset=None)
self._goal_position = np.array([-0.3, -0.3, 0.0515 / 2.0])
self._task_achieved = False
return
# Here we setup all the assets that we care about in this task.
def set_up_scene(self, scene):
super().set_up_scene(scene)
scene.add_default_ground_plane()
self._cube = scene.add(
DynamicCuboid(
prim_path="/World/random_cube",
name="fancy_cube",
position=np.array([0.3, 0.3, 0.3]),
scale=np.array([0.0515, 0.0515, 0.0515]),
color=np.array([0, 0, 1.0]),
size=1 #TODO: change here
)
)
self._franka = scene.add(Franka(prim_path="/World/Fancy_Franka", name="fancy_franka"))
return
# Information exposed to solve the task is returned from the task through get_observations
def get_observations(self):
cube_position, _ = self._cube.get_world_pose()
current_joint_positions = self._franka.get_joint_positions()
observations = {
self._franka.name: {"joint_positions": current_joint_positions},
self._cube.name: {"position": cube_position, "goal_position": self._goal_position},
}
return observations
# Called before each physics step,
# for instance we can check here if the task was accomplished by
# changing the color of the cube once its accomplished
def pre_step(self, control_index, simulation_time):
cube_position, _ = self._cube.get_world_pose()
if not self._task_achieved and np.mean(np.abs(self._goal_position - cube_position)) < 0.02:
# Visual Materials are applied by default to the cube
# in this case the cube has a visual material of type
# PreviewSurface, we can set its color once the target is reached.
self._cube.get_applied_visual_material().set_color(color=np.array([0, 1.0, 0]))
self._task_achieved = True
return
class HelloWorld(BaseSample):
def __init__(self) -> None:
super().__init__()
return
def setup_scene(self):
world = self.get_world()
# We add the task to the world here
world.add_task(FrankaPlaying(name="my_first_task"))
return
async def setup_post_load(self):
self._world = self.get_world()
# The world already called the setup_scene from the task (with first reset of the world)
# so we can retrieve the task objects
self._franka = self._world.scene.get_object("fancy_franka")
self._controller = PickPlaceController(
name="pick_place_controller", gripper=self._franka.gripper, robot_articulation=self._franka
)
self._world.add_physics_callback("sim_step", callback_fn=self.physics_step)]
#TODO: change here
self._franka.gripper.set_joint_positions(self._franka.gripper.joint_opened_positions)
await self._world.play_async()
return
async def setup_post_reset(self):
self._controller.reset()
await self._world.play_async()
#TODO: change here
self._franka.gripper.set_joint_positions(self._franka.gripper.joint_opened_positions)
return
def physics_step(self, step_size):
# Gets all the tasks observations
current_observations = self._world.get_observations()
actions = self._controller.forward(
picking_position=current_observations["fancy_cube"]["position"],
placing_position=current_observations["fancy_cube"]["goal_position"],
current_joint_positions=current_observations["fancy_franka"]["joint_positions"],
)
self._franka.apply_action(actions)
if self._controller.is_done():
self._world.pause()
return
Let us know if you have more questions.
Thank you @oahmed for your answer.
The problem for opening the gripper was solved by the provided code, however, it seems that the expected behavior of the robot for picking up the cube is different from what we see; The robot does not correctly recognize the size/scale/boundary of the cube and its gripper goes through the cube and cannot grab it. So the robot is not able to pick the cube up.
I have these warnings :
[Warning] [omni.client.plugin] Tick: authentication: Discovery(wss://localhost/omni/discovery): Error creating Api/Connection search: Not connected
[Warning] [carb.flatcache.plugin] Type tag does not have a corresponding USD type
[Warning] [gpu.foundation.plugin] Requesting texture to use texture streaming, but the service isn’t available.The texture will be created as a regular resource.
I would appreciate it if you give the solution to this problem too.
I found that the problem was due to the assets that I used to import Franka, and fixed by loading ““/Isaac/Robots/Franka/franka.usd” file instead of "/Isaac/Robots/Franka/franka_alt_fingers.usd” in the line 59 of the following direction:
.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.isaac.franka/omni/isaac/franka/franka.py
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.