USDSkel not in sync when the scene is played

Omniverse Isaac Sim
, ,
1

Isaac Sim Version

X 5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
X Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: RTX A5000
  • Driver Version: 572.60

Topic Description

USDSkel out of sync with Mesh only when scene is “played” or during rendering.

Detailed Description

I am trying to track different parts of a humanoid mesh during animation, to do this I want to use the mesh’s respective skeleton/rig/armature to drive the movement of my precomputed bounding boxes for different regions. So I would be extracting the translation and rotation of the bones at each time code for alignment. The problem is when I step through the timeline manually the skeleton is perfectly in sync with the mesh. But when I press play or render the skeleton is out of sync with the mesh. It seems that the skeleton is at least 1 frame ahead of the mesh when out of sync but even when I keyframe the last frame to match the mesh it doesn’t align.

Steps to Reproduce

  1. Sample FBX for testing (https://free3d.com/3d-model/nathan-animated-003-walking-644277.html). Same problem with USD format but this model is free and used as an example.

  2. Import FBX and enable extension “omni.anim.skelJoint” for skel vizualisation.

  3. Scrub through timeline manually for synced skeleton. Press play to see out of sync skeleton.

Error Messages

No Error Messages

Screenshots or Videos

Additional Information

What I’ve Tried

The only difference I can think of between manually scrubbing through the timeline and playing the scene is that simulation is being implemented when the timeline is playing by itself. So I have checked all time domain properties but they seem to align to the 24fps rate that I intend to render at. I also disabled asynchronous rendering as I thought this might separate the rendering of the image and updating of the skeleton to independent processes and to see if that fixed the problem but to no avail.

image
image966×512 43.2 KB

Related Issues

I haven’t seen any issues related to this problem.

Additional Context

N/A

2

I would like to know as well. Any help here?

3

@sam.duignan @josiah7 if you disable FSD via Preferences > Rendering > Fabric Scene Delegate, you should get the same result as scrubbing through the timeline:


not sure if what is happening under the hood or whether it’s intended FSD behavior in order to optimize scene rendering, so i’ll defer to the devs to elaborate more on this.

fun fact, in Isaac Sim 4.5, it’s desync’d regardless of the FSD setting 🧐

1 Like
4

Thank you @Simplychenable!

I was able to sync the skeleton when disabling this feature. For any future users that want to know the carb path to this setting as it took me some digging through the source code:

import carb.settings

s = carb.settings.get_settings()
s.set("/app/useFabricSceneDelegate", False)

Another fun fact, if you set this as false and then reload the scene (which is what is suggested upon disabling) both "manually “scrubbing” and playing/rendering is all out of sync 🧐

1 Like
5

so sounds like it’s a bug as it will only work if you were to change the setting without reloading the scene? i wonder what the actual behavior should be with Skel…

6

@Simplychenable
After we disable the FSD setting our viewport syncs the mesh and skeleton but when I query the pose of the skeleton at render time the returned value is the out of sync one still … so for this use case the FSD isn’t enough to align meshes to the bones/joints.

7

just to get a bit more information, can you specify which renderer you are using?

9

Hi @sam.duignan, @josiah7, and @Simplychenable,

Thanks for the thorough debugging here – @Simplychenable’s FSD diagnosis was spot-on.

Root Cause

This is a Fabric Scene Delegate (FSD) pipeline ordering issue. During real-time playback, the skeleton debug visualization (omni.anim.skelJoint) reads joint transforms directly from USD at frame N, while Hydra mesh skinning reads from the Fabric cache which is still on frame N-1. Manual scrubbing forces a synchronous compose, so both sources stay in sync.

This also explains why disabling FSD without reloading “fixes” it (Hydra falls back to USD reads), but reloading re-introduces a different timing path.

Workarounds

As @sam.duignan found, disabling FSD fixes the viewport desync but does not fix programmatic skeleton queries at render time – and reloading the scene after toggling FSD off can re-break it. So that’s not a reliable solution, especially for the bounding-box tracking use case described here.

1. For programmatic skeleton tracking, offset your query by -1 frame to align with the rendered mesh:

render_aligned_frame = current_frame - 1
xform_cache = UsdGeom.XformCache(Usd.TimeCode(render_aligned_frame))
joint_xforms = skel_query.ComputeJointWorldTransforms(xform_cache)

2. For offline/synthetic data capture, use Replicator’s synchronous step mode which forces a full evaluation before the frame is captured:

import omni.replicator.core as rep
rep.orchestrator.step()  # fully synchronous -- no frame lag

We’ve confirmed this is still present in Isaac Sim 6.0.0 (FSD still default-on, same frame history count, same pipeline architecture). Flagging internally.