Please do! Make sure to include the replicator.
We have now figure out how to work around the crashes. New issue was observed:
Related issue: Variants on Bones result with rendering of ghosted meshes
import omni
from pxr import UsdGeom, Gf, UsdSkel
def get_stage():
return omni.usd.get_context().get_stage()
def create_xform_prim(scene_path):
return UsdGeom.Xform.Define(get_stage(), scene_path).GetPrim()
def find_first_of_type(prim, type):
if prim.GetTypeName() == type:
return prim
for child in prim.GetChildren():
material = find_first_of_type(child, type)
if material is not None:
return material
return None
def find_joint_index(skeleton, joint_name):
joints = skeleton.GetJointsAttr().Get()
for i, name in enumerate(joints):
if name.endswith(joint_name):
return i
return -1
human_prim = get_stage().GetPrimAtPath('/World/biped_demo')
human_mesh = find_first_of_type(human_prim, 'Mesh')
human_skeleton = UsdSkel.Skeleton(find_first_of_type(human_prim, 'Skeleton'))
cellphone_prim = create_xform_prim('/cellphone')
cellphone_prim.GetReferences().AddReference('/home/USERNAME/Downloads/Rock_Small_01.usd')
cellphone_prim.SetInstanceable(False)
cellphone_mesh = find_first_of_type(cellphone_prim, 'Mesh')
cellphone_xform = Gf.Matrix4f([[1,0,0,0],[0,1,0,0],[0,0,1,0],[-83,136,0,1]])
len_human_mesh_points = len(human_mesh.GetAttribute('points').Get())
cellphone_mesh_points = list(cellphone_mesh.GetAttribute('points').Get())
for i in range(len(cellphone_mesh_points)):
cellphone_mesh_points[i] = cellphone_xform.Transform(cellphone_mesh_points[i])
human_mesh.GetAttribute('points').Set(list(human_mesh.GetAttribute('points').Get()) + cellphone_mesh_points)
cellphone_mesh_normals = list(cellphone_mesh.GetAttribute('normals').Get())
for i in range(len(cellphone_mesh_normals)):
cellphone_mesh_normals[i] = cellphone_xform.Transform(cellphone_mesh_normals[i])
human_mesh.GetAttribute('normals').Set(list(human_mesh.GetAttribute('normals').Get()) + cellphone_mesh_normals)
human_mesh.GetAttribute('faceVertexCounts').Set(list(human_mesh.GetAttribute('faceVertexCounts').Get()) + list(cellphone_mesh.GetAttribute('faceVertexCounts').Get()))
cellphone_mesh_faceVertexIndices = list(cellphone_mesh.GetAttribute('faceVertexIndices').Get())
for i in range(len(cellphone_mesh_faceVertexIndices)):
cellphone_mesh_faceVertexIndices[i] += len_human_mesh_points
human_mesh.GetAttribute('faceVertexIndices').Set(list(human_mesh.GetAttribute('faceVertexIndices').Get()) + cellphone_mesh_faceVertexIndices)
index = find_joint_index(human_skeleton, 'R_Wrist')
len_cellphone_mesh_faceVertexIndices = len(cellphone_mesh.GetAttribute('faceVertexCounts').Get())
element_size = 8
group = [index] + [0] * (element_size - 1)
human_mesh.GetAttribute('primvars:skel:jointIndices').Set(list(human_mesh.GetAttribute('primvars:skel:jointIndices').Get()) + group * len_cellphone_mesh_faceVertexIndices)
group = [1] + [0] * (element_size - 1)
human_mesh.GetAttribute('primvars:skel:jointWeights').Set(list(human_mesh.GetAttribute('primvars:skel:jointWeights').Get()) + group * len_cellphone_mesh_faceVertexIndices)
human_mesh.GetAttribute('primvars:st').Set(list(human_mesh.GetAttribute('primvars:st').Get()) + list(cellphone_mesh.GetAttribute('primvars:st').Get()))
len_human_mesh_st = len(human_mesh.GetAttribute('primvars:st').Get())
cellphone_mesh_st_indices = list(cellphone_mesh.GetAttribute('primvars:st:indices').Get())
human_mesh.GetAttribute('primvars:st:indices').Set(list(human_mesh.GetAttribute('primvars:st:indices').Get()) + cellphone_mesh_st_indices)
rot_z90 = Gf.Quatf(Gf.Rotation(Gf.Vec3d(0, 0, 1), 90).GetQuat())
animation = find_first_of_type(human_skeleton.GetPrim(), 'SkelAnimation')
rotations = animation.GetAttribute('rotations')
_ = rotations.Get()
_[index] = rot_z90 * _[index]
rotations.Set(_)
Here is an imperfect solution of attaching a cellphone to a person’s hand. Firstly download the resources we need:
https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Isaac/2022.1/NVIDIA/Assets/AnimGraph/Characters/NVIDIA/biped_demo.usd
https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Rocks/Rock_Small_01.usd
Suppose we want to attach the rock to the person’s right wrist.
- Drag biped_demo.usd into an empty stage.
- Update the rock’s path to your local path.
- Run the code in script editor.
- Try turn the wrist and you can see the rock follows the wrist.
- There are some geometry error here; we’ll further try to figure out the cause; hopefully it gives a pointer to how it can be solved.
1 Like
I will have a look ASAP
@shangrul , thanks for the reply.
- The rock was 100x bigger than the biped so I first had to change the
MPUto0.01before dragging on the biped. Perhaps you’ve been testing in Composer, not Issaac 4.1? - It renders a polygon bar which seems to follow the wrist, but the rotation and movement of the wrist indeed moves the rock!
- Would you be able to help me with a “cleaner” solution? Like a function I could call by providing a path to xform I want to attach and a bone?
- What does this “magic” matrix do?
cellphone_xform = Gf.Matrix4f([[1,0,0,0],[0,1,0,0],[0,0,1,0],[-83,136,0,1]]) - Instead of writing such code maybe it will be easier to revive the
omni.anim.xform_constraintsextension?
This is very close to what I am looking for. Wrapping it up in a function and removing the strange artifact will get me across the finish line.
Ideally attach(path_to_item, path_to_bone). I need it for mobile phones in hands + hats and glasses on heads.
PS. there are a lot of hardcoded magic values that forbid me from implementing this myself, as I simply don’t understand what these magic numbers mean.
The workaround that works is documented here:
I used isaac but will also try in composer - I’ll look into this further; the matrix that you mentioned is used to pose the rock near the hand (skeleton space), you can adjust it to make it fit better too.
1 Like
I believe my workaround from the linked post is a better approach. You guys should restore the constraint plugin. Otherwise everyone will have their own implementation and its just a huge waste time for humanity.