Franka Arm Moves Downward Before Picking/Placing When Placed on a Table

Omniverse Isaac Sim
1

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: RTX 4070
  • Driver Version: 535.183.01

Topic Description

Detailed Description

I am performing a simple pick-and-place task using the Franka robot in Isaac Sim with the PickPlaceController. When the Franka is placed on the ground, it works perfectly: it avoids collisions, picks up objects (cubes and spheres), and places them in their respective positions.

However, when I place the Franka on a table (elevated surface), the robot behaves unexpectedly. Even though the picking and placing positions are above the table, the arm first moves downward, collides with the table, and then moves up to the target position. This behavior does not occur when the robot is on the ground.

Screenshots or Videos

Screenshot from 2025-02-21 18-44-43
Screenshot from 2025-02-21 18-44-431920×1200 209 KB

Screenshot from 2025-02-23 10-22-29
Screenshot from 2025-02-23 10-22-291920×1200 187 KB

Screenshot from 2025-02-23 10-22-45
Screenshot from 2025-02-23 10-22-451920×1200 186 KB

  1. Picking position

    • The actual picking position is correctly defined (as seen in the final photo).
    • Despite this, the robot arm descends downward before moving to the picking position.

  2. Collision with Table:

    • When a table is added, the arm collides with it as it moves downward before reaching the picking position.

  3. Behavior on Ground:

    • The robot works perfectly when placed on the ground, along with the objects and bins.
    • In that setup, the arm avoids collisions and moves directly to the picking/placing positions.

  4. Rigid Bodies:

    • In the current setup, the bins and objects are not locked, so they tend to move downward due to gravity (as they are rigid bodies).

Additional Information

What I’ve Tried

  • Disabled gravity for the robot.
  • Removed the table from the scene to eliminate collisions and check how the robot behaves.
  • Locked the positions of the bins and objects to prevent them from moving.
  • I have verified that the table is properly set as a collision object.
  • The issue persists even if I remove the table from the scene, suggesting that the arm’s default behavior is to move downward first.
  • I suspect the PickPlaceController might be assuming the ground as the default reference frame.

Any help or suggestions would be greatly appreciated!

2

hey, maybe take a look at this solution where the pickplacecontroller has to be modified slightly

3

Hey @jean7474 The PickPlaceController seems to be abstract in the latest version of isaac sim (4.5.0). How did you navigate to that file ?

4

you should find it in the installation folder of isaac sim under (for isaac sim 4.5)

$HOME/isaacsim/exts/isaacsim.robot.manipulators/isaacsim/robot/manipulators/controllers/pick_place_controller.py
5

Hey @JeanLesur. Your solution did work, thanks a lot for your valuable feedback :)

1 Like
6

great :)
honestly i don’t understand why it was not fixed in 4.5.

7

I guess this problem didn’t reach the attention of the Nvidia people yet, though someone from the mods team replied to you.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

9

It appears you’ve encountered a limitation with the PickPlaceController example extension. As it stands, this is a demonstration of basic pick and place functionality and might not cover all scenarios, particularly those involving changes in the robot’s base elevation. The example may need to be adapted to account for different base heights or collision objects in the environment.

Thank you to @JeanLesur for your contribution to the community.

1 Like