Can not add mode for mesh fitting on custom face mesh

Hello,

I have followed the online tutorials about character transfer and used it to import a MetaHuman Face mesh (as usd file) in audio2face. When I try to assign the points to green openmouth Mark to my MetaHuman mesh, I can select points on Mark but it does not let me assign points to the MetaHuman. The points can therefore not transfer. I do not know why this is not working, or perhaps because I need to import a skin texture (but I am not sure how I can import that from a MetaHuman file). Could you please look into why mesh fitting the points does not work?

Kind regards

Hello @elmusic2! Thanks for reaching out to us. I have contacted the team to come and assist you with your questions.

hi @elmusic2 ,
There’s a chance that the meta human usdskel has skelAnimation in there that changed the position of the the target head.
Can you try select the lod0 mesh and use this menu Toolbox>Built-in>Mesh>Stamp Mesh ?
It will create a stamp of the mesh where the default points would be. You can use this mesh for tagging correspondence instead.
Also, can you expand the LOD0 mesh on Stage view? what else is parented under it?

Hello @esusantolim, thank you very much for your reply. With the suggested Stamp Mesh I could indeed do the Mesh Tagging on the MetaHuman. However, the Mesh Fitting and Post Wrap are still not working. When trying to use the a2f pipeline to get the MetaHuman in sync with Mark it is also not working yet. I got the following error:

2022-05-28 15:59:29 [Warning] [omni.usd] Warning (secondary thread): in _TriangulateFaceVarying at line 308 of E:\w\ca6c508eae419cf8\USD\pxr\imaging\hd\meshUtil.cpp – numVerts and verts are incosistent [/World/Tori_FaceMesh/LOD0_stamp_result]
2022-05-28 16:03:05 [Warning] [omni.anim.wrap.plugin] Wrap deformer evaluation skipped - invalid point count
2022-05-28 16:05:03 [Warning] [omni.usd] Warning (secondary thread): in _TriangulateFaceVarying at line 308 of E:\w\ca6c508eae419cf8\USD\pxr\imaging\hd\meshUtil.cpp – numVerts and verts are incosistent [/World/Tori_FaceMesh/LOD0_stamp_fitted_01]

2022-05-28 16:05:06 [Error] [omni.kit.charTransfer.commands] Traceback (most recent call last):

  • File “c:\users\vislab\appdata\local\ov\pkg\audio2face-2021.3.3\exts\omni.kit.charTransfer\omni\kit\charTransfer\commands.py”, line 104, in _do_fitting*
  • register_laplacian_func = register_laplacian_factor(L, landmarks)*
  • File “c:\users\vislab\appdata\local\ov\pkg\audio2face-2021.3.3\exts\omni.kit.charTransfer\omni\kit\charTransfer\fitting\registration.py”, line 66, in register_laplacian_factor*
  • return partial(_register_laplacian_solve, L, landmarks, landmark0_mask, spla.splu(lhs))*
  • File “c:\users\vislab\appdata\local\ov\pkg\audio2face-2021.3.3\exts\omni.audio2face.player_deps\deps\py\scipy\sparse\linalg\dsolve\linsolve.py”, line 339, in splu*
  • ilu=False, options=_options)*
    RuntimeError: Factor is exactly singular

as shown in the screenshot.

Do you know what could be the cause that the Stamp Mesh can not be fitted? Do you perhaps also know how to import a custom face mesh for the MetaHuman (as the only option is grey or green in a2f).

Kind regards

Also, the expanded LOD0 mesh in Stage view looks like this:
stage_lod

Are you able to share the mesh via DM or email? esusantolim@nvidia.com
GeomSubset is fine, but the mesh also needs to be one continuous mesh.
You can try run the menu (while selecting the mesh) Mesh > Mesh Separate
It would create multiple new meshes if your original mesh is made of multiple polygon pieces.

Hello @esusantolim,

I have shared the mesh via DM about a week ago. Could you please let me know if you found any more information, especially about the custom skin texture material?

Kind regards