Hey there.
In exts/omni.isaac.synthetic_utils/omni/isaac/synthetic_utils/syntheticdata.py:get_pose(self) you are getting the world transform matrix, but that works only for static objects.
For animated ones you need the time from the timeline :)
Hi there,
I tested get_pose and the underlying get_world_transform_matrix functions on animated and static objects (see attached usd stage) and it seems to work as intended. Can you let me know if the issue persist?
Repro:
-
load stage: get_pose_from_anim.usd (78.0 KB)
-
hit play to run the animation
-
run script in the Script Editor window:
import omni from omni.isaac.synthetic_utils import SyntheticDataHelper stage = omni.usd.get_context().get_stage() cone_prim = stage.GetPrimAtPath("/World/Cone") sphere_prim = stage.GetPrimAtPath("/World/Sphere") sdh = SyntheticDataHelper() poses = sdh.get_pose() print(f"poses={poses}") cone_loc = omni.usd.utils.get_world_transform_matrix(cone_prim).ExtractTranslation() print(f"cone_loc={cone_loc}") sphere_loc = omni.usd.utils.get_world_transform_matrix(sphere_prim).ExtractTranslation() print(f"sphere_loc={sphere_loc}") -
terminal output:
poses=[('/World/Cone', 2, 'Cone', array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]])), ('/World/Sphere', 1, 'Sphere', array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]))] cone_loc=(0, 0, 0) sphere_loc=(0, 0, 0) poses=[('/World/Cone', 2, 'Cone', array([[1. , 0. , 0. , 0. ], [0. , 1. , 0. , 0. ], [0. , 0. , 1. , 0. ], [0. , 0. , 5.11514892, 1. ]])), ('/World/Sphere', 1, 'Sphere', array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]))] cone_loc=(0, 0, 5.11514891677613) sphere_loc=(0, 0, 0)
Hi @ahaidu
I tried this with a script in Isaac 2021, and for the ones animated with keyframes it did not save the correct data at all.
I’ll try again tomorrow with the new simulation, but I won’t be able to reproduce the full pipeline in a short time.
I read the code and found it equal, so thought that the issue might have been propagated.
I’ll let you know.
Hey, I quickly checked. and it seems to work when run from the interface.
However, when I run that in the past from the script it didn’t.
I used this custom recorder link, which is essentially your recorder with few modifications to be able to call it manually.
The main script I used is this, but fairly is a bit complicated for a test like this. I might be missing something though about the recorder or about the default timecode.
Can you tell me what is that Usd.TimeCode.Default()?
Here is the official USD documentation on Usd.TimeCode.Default():
Here is an isaac sim snippet using specific timecodes:
If the issue persists could you share a small repro script in the latest isaac sim for us to take a look.
I was able to reproduce with the following
import omni
from omni.isaac.synthetic_utils import SyntheticDataHelper
from pxr import Gf, UsdGeom, Usd
stage = omni.usd.get_context().get_stage()
cube_prim = stage.GetPrimAtPath("/Cube")
omni.kit.commands.execute('ChangePropertyCommand',
prop_path="/Cube"+ '.xformOp:translate',
value=Gf.Vec3d(0,0,0),
prev=Gf.Vec3d(0, 0, 0),
type_to_create_if_not_exist=UsdGeom.XformOp.TypeTranslate,
timecode=Usd.TimeCode(0))
omni.kit.commands.execute('ChangePropertyCommand',
prop_path="/Cube"+ '.xformOp:translate',
value=Gf.Vec3d(100,100,100),
prev=Gf.Vec3d(0, 0, 0),
type_to_create_if_not_exist=UsdGeom.XformOp.TypeTranslate,
timecode=Usd.TimeCode(100))
sdh = SyntheticDataHelper()
poses = sdh.get_pose()
print(f"poses={poses}") # initial loc
cube_loc = omni.usd.utils.get_world_transform_matrix(cube_prim).ExtractTranslation()
print(f"cube_loc={cube_loc}") # initial loc
cube_loc = omni.usd.utils.get_world_transform_matrix(cube_prim, Usd.TimeCode(100)).ExtractTranslation()
print(f"cube_loc={cube_loc}") #100,100,100
both in headless and in normal modes in the latest 2022.2.1
I get that that commands.execute is probably not the best way to do this, but still it seems reasonable.
In this case, get_poses returns always the initial loading location, as well as get_world_transform_matrix. However, get_world_transform_matrix(prim, Usd.TimeCode(x) return the correct value! This, irrespective if the sim is playing or not.
Are you starting the animation/simulation anywhere in your code?
Yes. After setting the positions(before call sdh). Let me take a video.
Here you can see that the segmentation works (otherwise get_pose is empty), and that it retains only the initial position (prev video the cube spawned in 0,0,0).
There seems to be a difference between setting time sampled animations using TimeCode() and animation key frames using the timeline:
- Animation — Omniverse Create documentation
- Animation Timeline and key frames — Omniverse Create documentation
It does no seem to be synthetic data related.
Yes, the difference lies there.
However, if get_pose in get_world... uses the current time code the problem is solved.
e.g.
def get_obj_poses(time):
"""Get pose of all objects with a semantic label.
"""
stage = omni.usd.get_context().get_stage()
mappings = helpers.get_instance_mappings()
pose = []
for m in mappings:
prim_path = m[1]
prim = stage.GetPrimAtPath(prim_path)
prim_tf = omni.usd.get_world_transform_matrix(prim, time)
pose.append((str(prim_path), m[2], str(m[3]), np.array(prim_tf)))
return pose
This would account for all situations.
Also, if I record synthetic data using the ‘bugged’ way/method to animate assets, I get wrong data (from get_pose), which seems counter-intuitive since it’s a perfectly working way to set animation in both translation orientation and scale.
Moreover, with the timeline it seems that I’m not able to set the orientation animation, unfortunately.
Finally, I noticed this also in the orientation component when importing USDs that have Xform->SkelRoot the SkelRoot has a pre-fixed rotation and I add a rotation to the Xform.
All the childs of the SkelRoot see only the Xform and not the overall rotation. I can share the USD privately if you are interested. I’ve attached the screenshots
Some updates:
The difference between time-samples animation and keyframes is that keyframed animated attributes should not be retrieved with TimeCode. Basically, one should use the default value of TimeCode.Default() to retrieve the animated attribute.
Furthermore, by default, keyframed animation’s default interpolation type is Bezier interpolation, while the timesamples animation’s interpolation is linear interpolation.
Regarding the setting rotations using the keyframes method:
-
only euler angles are supported (matrix or quaternion keyframes are currently not supported)
-
using the UI this can be set up by:
- using the property window’s context menu (right click on corresponding attribute)
- or animation timeline at the bottom to add those transform keys
-
Animation Timeline and key frames — Omniverse Code documentation
Still, people will get wrong data by calling get_pose().
Now, that this is on purpose, because of the way one animates the asset, or any other reason, I don’t think it’s the core point, at least for me.
I get all the insights that you are sharing, and I’m understanding the difference.
Still, if one has a helper/function/method that gives you the pose, should give you the correct pose of the object in that instant in time. Or, at the very least, signal that this pose may be incorrect.
Right now, if I simply use the code and save the data, it is not “correct” from my point of view because the pose is not the real pose of the object at that point in time in the simulation.
The correct position and orientation is what I would expect. Now, you may consider that correct because that’s how the keyframe works. But I’ve a different viewpoint in this case since I’m using get_pose to generate groundtruth data.
Right now, if I diff the result get_pose with any predicted position or orientation the result will be wrong given that I animated my objects in that way.
BTW, at this point, it’s not a problem for me. I’ll be aware of this in the future, and will take the time to post-process the data accordingly.
Regarding using the UI: a big important part in data generation and testing is the capability of scripting. I guess that’s the very reason behind the choice of allowing that for the public.
Using the TimeCode in the get_pose method works for both instances of time samples and keyframes. I don’t see a problem there.
About interpolation: never had a problem with that.
About euler angles: never had a problem with that either.
About orientation: I’ve only said that this does not work (right click on orientation). It works with a right click on rotation
Let me know if you want to check the human asset which shows similar problems. I’ll send it to you.
Thanks for the support.
1 Like
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.