Bug: Isaac Sim does not update world poses which makes imitation policies nearly impossible to implement

[whoishoa]

We are running GitHub - NVIDIA-Omniverse/OmniIsaacGymEnvs: Reinforcement Learning Environments for Omniverse Isaac Gym and when we try to get the current world position of an XForm or Mesh, it’s not being updated in the simulation and we are always getting a frozen value. We can’t train any imitation policies because of this.

Steps to reproduce:

1. Launch OmniIsaacGymEnvs Humanoid task.
2. Select the pelvis prim on the GUI: World > Envs > env_0 > Humanoid > pelvis
3. Press ‘f’ to center on the pelvis
4. Note how the controls to adjust the position of the pelvis doesn’t change positions when the simulation is running. This illustrates why none of our programmatic code works for retrieving an XForm position when we use:

keypoint = XFormPrim("/World/envs/env_" + str(idx) + "/Humanoid/" + body_prim + "/keypoint")
  pos, _ = keypoint.get_world_pose()
  return pos

or

torso_keypoint = stage.GetPrimAtPath("/World/envs/env_" + str(idx) + "/Humanoid/" + body_prim + "/keypoint")
  matrix: Gf.Matrix4d = omni.usd.get_world_transform_matrix(torso_keypoint)
  translate: Gf.Vec3d = matrix.ExtractTranslation()
  return translate

It always returns the same position.

When we run the simulation with a fresh stage with only a single instance, our code works at tracking specific XForms or joints. The Isaac GUI is updated also reflect the changing position as well.

We tested this bug on 2023.1 and 2022.2.1 on three different machines.

[sujitvasanth]

This is not universal as i use getworldposes() in my rl and does work for my custom usd robot and have debugged previously …
the humanoid task uses some specific custom views so that may be an edge case where things are going wrong

e.g.
self._robots = ArticulationView(prim_paths_expr=“/World/envs/./biped", name=“biped_view”)
self._lfoot = RigidPrimView(prim_paths_expr="/World/envs/./biped/LeftFoot”, name=“lfoot_view”, reset_xform_properties=False, track_contact_forces=True, prepare_contact_sensors=False)
self._rfoot = RigidPrimView(prim_paths_expr=“/World/envs/./biped/RightFoot", name=“rfoot_view”, reset_xform_properties=False, track_contact_forces=True, prepare_contact_sensors=False)
self._body = RigidPrimView(prim_paths_expr="/World/envs/./biped/Body”, name=“body_view”, reset_xform_properties=False, track_contact_forces=False, prepare_contact_sensors=False)

and in my later code
xpos = torch.stack([self._lfoot.get_world_poses()[0][:, 0], self._rfoot.get_world_poses()[0][:, 0],
self._body.get_world_poses()[0][:, 0]], dim=1)

Maybe naive of me but perhaps the way you are referencing by trying to pull individual environments rather than the whole tensor … then you can extract the environment you want

sincerely

Sujit Vasanth

[kellyg]

Hi there, for performance reasons, the OmniIsaacGymEnvs setup avoids syncing updates to USD. Updates are instead passed through fabric, from which states can be collected using the physics tensor APIs provided in the omni.isaac.core RigidPrimView and ArticulationView classes. If you would like to use USD APIs during simulation, you will need to set the Simulation Settings to USD as the Simulation Output, as shown in the screenshot, then use the CPU pipeline when launching an environment with command line argument pipeline=cpu.