Collision issue in scenario with PickPlaceController

Omniverse Isaac Sim
,
1

Note: For any Isaac Lab topics, please submit your topic to its GitHub repo (GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim) following the instructions provided on Isaac Lab’s Contributing Guidelines (Contribution Guidelines — Isaac Lab Documentation).

Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.

Isaac Sim Version

4.2.0
4.1.0
4.0.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: NVIDIA GeForce RTX 2080
  • Driver Version: 535.183.01

Topic Description

Detailed Description

I am building a basic demo with the pick and place controller. I have a UR10 robotic arm and the object to pick up is a DynamicCuboid. To perform the task, i am using the PickPlaceController from the universal_robots.controllers package and I based my code on the pickplace task.

However, i have a weird error while placing the object to pick on a table. See the videos below.
1st video without table: as expected, the object falls on the floor and the robot pick it.
2nd video with table: the object falls on the table but the robot tries to reach the position in which the object would be without the table. The robot is thus colliding with the table.

Weirdly, the position of the cube as retrieved by the controller is the right one when i compare it with the GUI.

My workflow is “Extension” and I chose to use a Loaded Scenario Template from the documentation.

Steps to Reproduce

  1. Unzip the scenario code test_scenario.zip (24.6 KB)
  2. Run the code as an extension.
  3. Uncomment lines 171-178 in ui_builder.py, reload the scenario and play it

Error Messages

No messages

Screenshots or Videos


Additional Information

What I’ve Tried

  • changed the table with some other assets
  • moved the robot in different positions
  • played with the colliders options of the physics API (convex hull, triangle, etc)
  • checked the “enable CCD” option

Related Issues

similar issue despite i am not having a negative sign in the scale

Additional Context

2

Could you explain how the controller obtains the cube’s position in your code?

4

i am using this object_pos, _ = self._object.get_world_pose() where self._object is the DynamicCuboid I added to the scene.
I also tried to use local coordinates with the appropriate transforms (so that the object is express in the robot’s coordinate system) but the behavior is also the same).

I looks like there is a discrepancy between the physics engine and the stage

5

Hey,
Just to let you know that i found my issue.
It comes from the manipulators_controllers.PickPlaceController which does not consider the Z-position of the object to pick&place…
So I had to slightly modify this controller and it is now working as expected!

For anyone interested here i had to change the method _get_target_hs() by passing it the picking-z-position and change how the return h parameter is calculated.

6

Hey @jean7474 can you help us with what code changes you have done on _get_target_hs() method, it will be a great help if you can

7

@jean7474 is height and z picking position not same, what i am guessing it is for it is picking from top of the object right ??