Skeleton based object deformation with physics?

Omniverse Isaac Sim
1

Hi all, I’m building a scene with a large number of deformable plants. The sim would be too heavy to make them all deformable so I want to generate a skeleton and rig/skin the plants similar to how SpeedTree works. I found all the omniverse docs on Skeletal Animation but can’t find any info on integrating bones with physics. Ideally I’d want to assign a primitive collider to each bone, joint constraints etc then have the plant interact with the physics system. To be clear I’m not asking for skinned colliders as AFAIK that’s not supported. Is this possible? Thanks!

3

Any update on this?

4

Hi @geordie_GR, thank you for posting this issue and sorry for the long delay on replying. In short, to enable integration of skeleton with physics you need to make sure that colliders are instantiated and tracking each skeleton joint.

I am not the expert on this, but @rdenomme has extensive experience.

Here is a more detailed explanation on how to setup up skeleton using Action Graphs to enable collision interaction with physics:

First, it is useful to instantiate one xform per skeleton joint you would like to have a collider. It’s handy to put them underneath the skelroot with the following properties:

  • They have a resetXformOpStack so that their transform represents a world transform and not a local transform relative to their parent. They also then need a transform op applied.
  • Their local transform is equal to the world transform of their associated skeleton at a fixed point in time
  • Put geometry - meshes or primitives with UsdPhysics.CollisionAPI under each xform and move these (i.e. move their transform, not that of their parent xform) to match the pose of your skeleton.

This can be done via script, and example of which is given below. This script is written as a action-graph script node and you can apply it to specific skeletons by using the ‘GetGraphTarget’ node to feed in the prim_path parameter and apply the action graph to the desired skelroots.If you just want the colliders to move with the animation, enable the anim.graph.bundle extension, and create a new action graph. Use the ‘GetAnimationGraphJointTransform’ to get the transform of the joint and plug that into the ‘WritePrimAttribute’ node to write this transform to the associated xform by writing to the attribute named ‘xformOp:transform’, which is why we added the transform op when setting up the xforms. I haven’t experimented with fabric yet, but you could also try writing to ‘omni:fabric:localMatrix’ for better performance.

# This script is executed the first time the script node computes, or the next time
# it computes after this script is modified or the 'Reset' button is pressed.
#
# The following callback functions may be defined in this script:
#  setup(db): Called immediately after this script is executed
#  compute(db): Called every time the node computes (should always be defined)
#  cleanup(db): Called when the node is deleted or the reset button is pressed
# Available variables:
#  db: og.Database The node interface - attributes are exposed in a namespace like db.inputs.foo and db.outputs.bar.
#          Use db.log_error, db.log_warning to report problems in the compute function.
#  og: The omni.graph.core module
import omni.usd, carb
import omni.graph.core as og
import pxr.UsdGeom, pxr.Sdf, pxr.UsdSkel, pxr.UsdPhysics
import usdrt.UsdGeom
 
def setup(db: og.Database):
  carb.log_info("[SkelCollider] setup")
  carb.log_info("[SkelCollider] skelroot_path: " + str(db.inputs.skelroot_path))
  carb.log_info("[SkelCollider] joints: " + str(db.inputs.joints[0]))
  stage = omni.usd.get_context().get_stage()
  skelcollider_parent_path = pxr.Sdf.Path(db.inputs.skelroot_path).AppendChild("SkelCollider")
  if not pxr.UsdGeom.Xform.Get(stage, skelcollider_parent_path):
    parent_xform = pxr.UsdGeom.Xform.Define(stage, skelcollider_parent_path)
    parent_xform.SetResetXformStack(True)
  
  
  for joint in db.inputs.joints:
    collider_xform_path = skelcollider_parent_path.AppendChild(joint)
    collider_xform = None
    if pxr.UsdGeom.Xform.Get(stage, collider_xform_path):
      continue
    
    collider_xform = pxr.UsdGeom.Xform.Define(stage, collider_xform_path)
    xform_op = collider_xform.AddTransformOp()
    xform_op.Set(pxr.Gf.Matrix4d(1.0))
    collider_geom = pxr.UsdGeom.Capsule.Define(stage, collider_xform_path.AppendChild("collider"))
    pxr.UsdPhysics.CollisionAPI.Apply(collider_geom.GetPrim())
 
 
 
def cleanup(db: og.Database):
  pass
 
def compute(db: og.Database):
  
  return True

This should create the following ActionGraph

image
image2181×1020 141 KB

You will need to run the graph to move the xforms to the location of their associated skel joints and then you can move the individual colliders under the xforms. Once you have this set up (i.e. you have your animation rigged). Unfortunately, there is no action graph node for setting the transforms of the animation skeleton at this time (and it is a little complicated to implement as it requires a forward kinematic calculation).

Another option to consider is to use the USD Skeletons API to move the xforms together with the skeleton joints. Please refer to this link for examples on how to use it: Universal Scene Description: API Introduction

5

Thanks for looking into this!! Unfortunately this wont work since graph functionality isn’t great in Isaac Lab but it’s a good start.

6

I will close the topic for now, but if there are any further issues you need help with feel free to open a new issue.