How to drive a Franka with an offset

Omniverse Isaac Sim
1

Hello. When I use Franka in this case and combine it with the example code to drive the robotic arm, Franka can never find the exact position of the Cube. How should I modify it?I attached my code

image
image805×837 74.3 KB

# Copyright (c) 2020-2023, NVIDIA CORPORATION. All rights reserved.
#
# NVIDIA CORPORATION and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA CORPORATION is strictly prohibited.
#

from omni.isaac.examples.base_sample import BaseSample
# This extension has franka related tasks and controllers as well
from omni.isaac.franka import Franka
from omni.isaac.core.objects import DynamicCuboid
from omni.isaac.franka.controllers import PickPlaceController
import numpy as np
from omni.isaac.core.utils.stage import add_reference_to_stage

i=0
class ManipulatorRobot(BaseSample):
    def __init__(self) -> None:
        super().__init__()
        return

    def setup_scene(self):
        world = self.get_world()
        world.scene.add_default_ground_plane()
        # Robot specific class that provides extra functionalities
        # such as having gripper and end_effector instances.
        add_reference_to_stage(usd_path="D:/software/omniverse/content/isaac_sim-assets-1-2023.1.1/Assets/Isaac/2023.1.1/Isaac/Robots/Clearpath/RidgebackFranka/ridgeback_franka.usd",prim_path="/World")
        franka = world.scene.add(Franka(prim_path="/World", name="fancy_franka"))
        # add a cube for franka to pick up
        world.scene.add(
            DynamicCuboid(
                prim_path="/World/random_cube",
                name="fancy_cube",
                position=np.array([0.3, 0.3, 0.3]),
                scale=np.array([0.0515, 0.0515, 0.0515]),
                color=np.array([0, 0, 1.0]),
            ))
        return

    async def setup_post_load(self):
        self._world = self.get_world()
        self._franka = self._world.scene.get_object("fancy_franka")
        self._fancy_cube = self._world.scene.get_object("fancy_cube")
        # Initialize a pick and place controller
        self._controller = PickPlaceController(
            name="pick_place_controller",
            gripper=self._franka.gripper,
            robot_articulation=self._franka,
        )
        self._world.add_physics_callback("sim_step", callback_fn=self.physics_step)
        # World has pause, stop, play..etc
        # Note: if async version exists, use it in any async function is this workflow
        self._franka.gripper.set_joint_positions(self._franka.gripper.joint_opened_positions)
        #await self._world.play_async()
        return
    
    def physics_step(self, step_size):
        cube_position, _ = self._fancy_cube.get_world_pose()
        goal_position = np.array([-0.5, 0.8, 0])
        current_joint_positions = self._franka.get_joint_positions()
        actions = self._controller.forward(
            picking_position=cube_position,
            placing_position=goal_position,
            current_joint_positions=current_joint_positions,
        )
        print("cube_position "+str(cube_position))
        print("current_joint_positions "+str(current_joint_positions))
        self._franka.apply_action(actions)
        # Only for the pick and place controller, indicating if the state
        # machine reached the final state.
        if self._controller.is_done():
            self._world.pause()
        return
        
    # This function is called after Reset button is pressed
    # Resetting anything in the world should happen here
    async def setup_post_reset(self):
        self._controller.reset()
        self._franka.gripper.set_joint_positions(self._franka.gripper.joint_opened_positions)
        await self._world.play_async()
        return
2

I’m wondering if controlling Franka requires that there must be only Franka in the XForm it is in, and no other joints.
Is this so that I can use Franka class and PickPlaceController and forward normally?

3

Hi @Gourd - Are you following this document?
https://docs.omniverse.nvidia.com/isaacsim/latest/core_api_tutorials/tutorial_core_adding_manipulator.html

4

@rthaker Yes, I just changed the default Franka robot in the document to the robot in the asset/Isaac/Robots/Clearpath/RidgebackFranka/ridgeback_franka.usd
It contains a Franka robot and a car body

5

Hi @Gourd - And you are still having issues?

6

@rthaker Thanks for your attention.
I found that the problem came from franka’s rootjoint.
I think you need to improve this USD or api. Because I didn’t change the asset, this error occurred while using the api.
The only solution I can think of is to enable rootjoint to it and cancel it when I want the car to move.
Do you have any other way to modify USD or other api

8

@rthaker I also want to know if this API can be applied to other degrees of freedom robotic arms or other types of robotic arms.
How should I design to implement this idea