PickPlaceController Fails with Ridgeback Franka

Omniverse Isaac Sim
,
1

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: NVIDIA RTX 3500 Ada Generation
  • Driver Version: 570.124.06

Topic Description

Detailed Description

I’m using the standard PickPlaceController from isaacsim.robot.manipulators.examples.franka.controllers with a Franka Emika Panda arm mounted on a Ridgeback mobile base in Isaac Sim 4.5. The controller fails to properly direct the arm to target objects when the Franka is mounted on the Ridgeback.

The issue appears to be that the PickPlaceController._get_target_hs() method doesn’t properly account for the actual Z-position of objects when calculating target heights during pick and place operations. When the Franka is mounted on the Ridgeback, the entire arm is elevated from ground level, causing the controller to target incorrect positions.

The controller works correctly when the Franka is used standalone (not mounted), but fails when mounted on the elevated Ridgeback platform.

Error Messages

No direct error messages, but the controller behaves incorrectly.

Screenshots or Videos

Additional Information

What I’ve Tried

Created a subclass of PickPlaceController that correctly implements the _get_target_hs(picking_z, placing_z) method to properly account for object Z-positions

Created a custom MountedPickPlaceController that extends the standard controller to properly handle Z-positions:

Related Issues

2

Example applications in Isaac Sim like this are intentionally simplified and may require adaptation for different hardware setups or mounting heights.

Have you tried the solution provided in the related topic you found?

3

Yes, I did in the video I sent. I noticed the end-effector (EE) was going above the cube, then drifted away. I thought it was because of the EE orientation in the video, so I found a way to change the orientation so that the EE points downwards. Unfortunately, it did not help. Every time the EE gets above the cube, it drifts in another direction instead of going downward to pick.

I even placed a visual marker to be sure my pick and place positions are correct.