Introduction
Hi all, currently I’m conducting RL research on dexterous hand grasping in IsaacGymEnvs (link) and I have an issue in the collision between URDF and box.
My code and the video are all attached here Question.zip (417.2 KB), including:
-
1_WrongResult.mp4: video demo of the wrong result -
2_CorrectResult.mp4: video demo of the correct result -
BugsENVPPO.yaml: train config file following IsaacGymEnvs format -
BugsENV.yaml: task config file following IsaacGymEnvs format -
bugs.py: task python file following IsaacGymEnvs format -
mano/: the folder that contains my URDF file
Issue
After setting the pose of the hand handle (a URDF file) and object handle (created by the IsaacGym API create_box) together, the object handle collides with the hand handle and pops away, resulting in a wrong initial pose.
However, I have tried to render collision meshes, but it seems that there’s no collision between the hand handle and the object handle (left figure).
-
1_WrongResult.mp4is the wrong result after setting both hand and object pose. - Commands for reproducing this result:
python train.py task=BugsENV task.env.loadHand=True
To confirm the collision comes from the hand handle and the object handle, I only create the object handle and set its pose, which is correct (right figure).
-
2_CorrectResult.mp4is the correct result if we only set the object pose. - Commands for reproducing this result:
python train.py task=BugsENV task.env.loadHand=False
To solve this issue, currently, I set max_depenetration_velocity = 1 which is a very small value compared to other tasks (mostly 100 or 1000). However, this results in hand object penetration so my grasp policy performs badly.
This bug is causing a significant issue for me, please let me know if there is any information that may be helpful in diagnosing the problem. Thanks a lot!!!
Detail Description
The following describes the detail of my implementation in bugs.py.
-
In function
_create_envs- The hand handle is generated from Handover-Sim (link), which has 51-DoF. The first 6-DoF is the base translation and rotation (in Euler XYZ), and the other 45-DoF joint rotation
- The object handle is generated from the IsaacGym API
create_box
-
In function
_read_train_data- The pose of the hand handle is [0, 0, 0.3, 0, …0] (translation (dim=3) + Euler (dim=3) + Euler (dim=45))
- The pose of the object handle is [0, 0, 0, 0, 0, 0, 1] (translation (dim=3) + quaternion (dim=4))
-
In function
reset_idx- Set the root state of the hand handle and object handle with IsaacGym API
set_actor_root_state_tensor_indexed - Set the DoF state, and the DoF target of the hand handle with IsaacGym API,
set_dof_state_tensor_indexed,set_dof_position_target_tensor_indexed
- Set the root state of the hand handle and object handle with IsaacGym API
-
Some
physxparameters in the config filephysx: num_threads: ${....num_threads} solver_type: ${....solver_type} use_gpu: ${contains:"cuda",${....sim_device}} # set to False to run on CPU num_position_iterations: 4 num_velocity_iterations: 1 contact_offset: 0.005 rest_offset: 0.0 bounce_threshold_velocity: 0.2 max_depenetration_velocity: 1000.0 default_buffer_size_multiplier: 5.0 max_gpu_contact_pairs: 2097152 # 2 * 1024 * 1024 num_subscenes: ${....num_subscenes} contact_collection: 1