Unitiliazed robot Articulation error when simulating robot and clothing in the same scene

Continuing the discussion from How can I make deformable body in python?:

I am also experiencing the same issue, and I have the same fix in GUI. I am using the franka pickandPlaceController for manipulating a Franka robot, which is 99% based on example 4.5 in 4. Adding a Manipulator Robot — Omniverse Robotics documentation (nvidia.com)) .
I have modified my scene to try to experiment with cloth manipulation, but I am experiencing some errors when the robot interacts with the cloth.

Before fixing the GPU not enabled error as described in the linked issues the robot simply moves through the cloth and, completing all 9 steps of the pickandplace routine on the small cube.

If I fix it however, the cloth is properly simulated but the robot fails to pick up the cube. In fact it fails to start the picking sequence at all. It simply returns to this idle position as soon as the simulation starts.

It seems to be some form of problem when the cloth interacts with the robot path planning, as it seems that the Articulation fails and gets unitiliazed.

Healthy log:

initializing handles for /World/Fancy_Franka
Event finished Move end_effector above the cube center at the 'end_effector_initial_height'.
Event finished Lower end_effector down to encircle the target cube
Event finished Wait for Robot's inertia to settle.
Event finished close grip
Event finished Move end_effector up again, keeping the grip tight (lifting the block)
Event finished Smoothly move the end_effector toward the goal xy, keeping the height constant.
Event finished Move end_effector vertically toward goal height at the 'end_effector_initial_height'.
Event finished loosen the grip
Event finished Move end_effector vertically up again at the 'end_effector_initial_height'
Event finished Move end_effector towards the old xy position

Unhealthy log with cloth simulated under robotic arm:

initializing handles for /World/Fancy_Franka
2023-07-12 07:03:54 [34,944,840ms] [Warning] [omni.isaac.core.articulations.articulation_view] Physics Simulation View is not created yet in order to use get_joint_positions
2023-07-12 07:03:54 [34,944,840ms] [Warning] [omni.isaac.core.articulations.articulation_subset] Attempting to access an uninitialized robot Articulation.
2023-07-12 07:03:54 [34,944,840ms] [Warning] [omni.isaac.core.articulations.articulation_subset] Attempting to access an uninitialized robot Articulation.
2023-07-12 07:03:54 [34,944,840ms] [Warning] [omni.isaac.core.articulations.articulation_subset] Attempting to access an uninitialized robot Articulation.
2023-07-12 07:03:54 [34,944,841ms] [Warning] [omni.isaac.core.articulations.articulation_subset] Attempting to access an uninitialized robot Articulation.
2023-07-12 07:03:54 [34,944,841ms] [Error] [omni.isaac.motion_generation.articulation_motion_policy] Attempted to compute an action, but the robot Articulation has not been initialized.  Cannot get joint positions or velocities.
2023-07-12 07:03:54 [34,944,841ms] [Error] [carb.physx.python] AttributeError: 'NoneType' object has no attribute 'astype'

  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.motion_generation\omni\isaac\motion_generation\lula\motion_policies.py(511): _evaluate_acceleration
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.motion_generation\omni\isaac\motion_generation\lula\motion_policies.py(504): _euler_integration
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.motion_generation\omni\isaac\motion_generation\lula\motion_policies.py(491): _update_robot_joint_states
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.motion_generation\omni\isaac\motion_generation\lula\motion_policies.py(146): compute_joint_targets
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.motion_generation\omni\isaac\motion_generation\articulation_motion_policy.py(102): get_next_articulation_action
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.motion_generation\omni\isaac\motion_generation\motion_policy_controller.py(52): forward
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.manipulators\omni\isaac\manipulators\controllers\pick_place_controller.py(158): forward
  c:\users\eirik\appdata\local\ov\pkg\isaac_sim-2022.2.1\exts\omni.isaac.examples\omni\isaac\examples\user_examples\hello_world.py(130): physics_step

The log message in the unhealthy log gets produces continously as long as the simulation is stepping using physics_step.

Interrestingly I am able to fix all my issues by moving the cloth out of the way of the robot, then the robot is able to execute the pickandplace routine while the cloth is simulated to the side.

Source code:
hello_world.py (11.1 KB)

1 Like

Fixed! Sortof… I was able to make the robot move as expected by using the reset button in the world control widget that comes with the Hello World sample series.