A weird collision between my URDF and the box


Hi all, currently I’m conducting RL research on dexterous hand grasping in IsaacGymEnvs (link) and I have an issue in the collision between URDF and box.

My code and the video are all attached here Question.zip (417.2 KB), including:

  • 1_WrongResult.mp4: video demo of the wrong result
  • 2_CorrectResult.mp4: video demo of the correct result
  • BugsENVPPO.yaml: train config file following IsaacGymEnvs format
  • BugsENV.yaml: task config file following IsaacGymEnvs format
  • bugs.py: task python file following IsaacGymEnvs format
  • mano/: the folder that contains my URDF file


Wrong Result Correct Result

After setting the pose of the hand handle (a URDF file) and object handle (created by the IsaacGym API create_box) together, the object handle collides with the hand handle and pops away, resulting in a wrong initial pose.

However, I have tried to render collision meshes, but it seems that there’s no collision between the hand handle and the object handle (left figure).

  • 1_WrongResult.mp4 is the wrong result after setting both hand and object pose.
  • Commands for reproducing this result: python train.py task=BugsENV task.env.loadHand=True

To confirm the collision comes from the hand handle and the object handle, I only create the object handle and set its pose, which is correct (right figure).

  • 2_CorrectResult.mp4 is the correct result if we only set the object pose.
  • Commands for reproducing this result: python train.py task=BugsENV task.env.loadHand=False

To solve this issue, currently, I set max_depenetration_velocity = 1 which is a very small value compared to other tasks (mostly 100 or 1000). However, this results in hand object penetration so my grasp policy performs badly.

This bug is causing a significant issue for me, please let me know if there is any information that may be helpful in diagnosing the problem. Thanks a lot!!!

Detail Description

The following describes the detail of my implementation in bugs.py.

  • In function _create_envs

    • The hand handle is generated from Handover-Sim (link), which has 51-DoF. The first 6-DoF is the base translation and rotation (in Euler XYZ), and the other 45-DoF joint rotation
    • The object handle is generated from the IsaacGym API create_box
  • In function _read_train_data

    • The pose of the hand handle is [0, 0, 0.3, 0, …0] (translation (dim=3) + Euler (dim=3) + Euler (dim=45))
    • The pose of the object handle is [0, 0, 0, 0, 0, 0, 1] (translation (dim=3) + quaternion (dim=4))
  • In function reset_idx

    • Set the root state of the hand handle and object handle with IsaacGym API set_actor_root_state_tensor_indexed
    • Set the DoF state, and the DoF target of the hand handle with IsaacGym API, set_dof_state_tensor_indexed, set_dof_position_target_tensor_indexed
  • Some physx parameters in the config file

    	num_threads: ${....num_threads}
    	solver_type: ${....solver_type}
    	use_gpu: ${contains:"cuda",${....sim_device}} # set to False to run on CPU
    	num_position_iterations: 4
    	num_velocity_iterations: 1
    	contact_offset: 0.005
    	rest_offset: 0.0
    	bounce_threshold_velocity: 0.2
    	max_depenetration_velocity: 1000.0
    	default_buffer_size_multiplier: 5.0
    	max_gpu_contact_pairs: 2097152 # 2 * 1024 * 1024
    	num_subscenes: ${....num_subscenes}
    	contact_collection: 1

Hi, sorry for the late reply.

My hand asset is generated from HandoverSim (https://github.com/NVlabs/handover-sim).
Following the installation steps, after finishing step 5 (compile assets: run ./handover/data/compile_assets.sh), the URDF file will be generated.
Thank you so much for your assistance.

Best regards