Unexpected behavior of `set_dof_position_target_tensor` and `set_dof_state_tensor` when using GPU

Hello, everyone. I skimmed the forums and didn’t find anybody asking about this problem before.

What I’m trying to do is the following: I have an actor that is a quadruped robot (Laikago) and I have a set of reference motions that I want it to reproduce. The reference motions consists of a sequence of PD targets for each joint of the robot, e.g. the robot has 12 joints, so a 24 frame reference motion would have the shape (24x12). At each timestep I’m setting the PD targets of each joint to the current frame.

Now, here comes the problem. There are many different ways to go about setting the PD targets or even directly setting the state of the joints. This is what I have tried:

  1. Not using the Tensor API – With just one actor, directly calling set_actor_dof_position_targets. This works fine.
  2. Using Tensor API on CPU to set PD targets – Setting use_gpu_pipeline to False and set_dof_position_target_tensor. This works fine.
  3. Using Tensor API on GPU to set PD targets – Setting use_gpu_pipeline to True and set_dof_position_target_tensor. This produces weird behavior.
  4. Using Tensor API on CPU to set the DOF states – use_gpu_pipeline = False and using set_dof_state_tensor. This works fine.
  5. Using Tensor API on GPU to set the DOF states – use_gpu_pipeline=True and using set_dof_state_tensor. This produces weird behavior

The only differences between 2 vs 3 and 4 vs 5 is changing the value of use_gpu_pipeline and .cuda() on the tensors passed to set_actor_dof_position_targets and set_dof_state_tensor. For my application I’d rather use PD targets (as in 2 and 3), but I also checked the behavior of directly setting the DOF states and also found that it produced strange behavior.

Please check the reference video for observing the behavior (it’s a bit hard to describe it in words, haha): https://www.youtube.com/watch?v=FSJDoSJwkSE&

I’d be very happy if somebody could provide me some light on this matter. Thanks.

I have investigated the matter a bit further and found that the problem comes from calling both set_actor_root_state_tensor and set_dof_position_target_tensor when using the GPU. Calling both of them when running on CPU works fine. My observations are the following:

  1. ** On CPU **: Any combinations work. Everything runs fine;
  2. ** On GPU **: Calling ONLY set_dof_position_target_tensor moves the joints as expected;
  3. ** On GPU **: Calling BOTH set_actor_root_state_tensor and set_actor_root_state_tensor makes the joints move strangely, however;
  4. If calling set_actor_root_state_tensor with a fixed pose (always same position and orientation) then calling set_dof_position_target_tensor works as expected. This really puzzles me.

Another thing I have noticed is that on the GUI widget on the simulation viewer, the position values and PD targets don’t change when running on the GPU. You can see that on the left side of the video below.

One more question that I have: Is the function set_actor_dof_position_targets not supported when running on GPU? (As I understand this function is not part of the Tensor API, but I also get strange joint behavior when running on GPU while it’s completely fine on CPU).

I have uploaded a new video to supplement this reply that shows the behaviors I described above:
https://www.youtube.com/watch?v=HP7OUALKRg0&

I have also attached a .zip file containing the source code for the whole project, should you want to try to reproduce these behaviors. Everything relevant is in laikago_motion_playback.py.

laikago_motion_playback.zip (1.4 MB)

Thank you!

Hi @guichristmann,

Sorry for the late response here. We’ve been heads down with GTC and the new release.

This looks like it could be a bug on the GPU PhysX side, or possibly something related to friction parameters. I’ve asked the team to have a more detailed look. Thanks for providing the source for review - we can test whether the problem still shows up in the new version.

Take care,
-Gav