Issues in animating a head mesh with audio2face

I’m trying to test audio2face. I’m exporting a simple head mesh from Maya as usd.
When I export it through Maya’s file>export selection>OmniTranslatorExport format and then connect it to a2f pipeline, it gets animated but all the vertices are deformed and pulled down (as seen in the image).
When I try to export through the Omniverse menu>export, with the same settings, the mesh doesn’t get distorted but it doesn’t get animated either.

Any idea why either of these issues are happening? Are there any clear guidelines for importing head meshes?
I didn’t see any such guidelines on the YouTube tutorials.

Thank you.

Hi @romanm, thanks for trying Audio2Face. I see that you attached your custom mesh in the “Audio2Face tab - attached mesh”, and that caused the issue. Audio2Face is trained to animate the pre-defined template face mesh (which is mark), and you cannot attach a custom mesh in the “attached mesh” tab. In order to animate your custom face mesh, please use “character transfer” tab. You can find a series of tutorial on this topic from audio2face video tutorial.

I tested it through character transfer instead.
Thank you.

Hello dear Nvidia Developer Team!

I also have trouble meshing the head in my full body xform model.
In the attachement is the correspondent usd file, screenshot and a video showing the problem.
I also tried following the tutorial on youtube.

2021-05-04 11-10-53.mkv (9.2 MB) Rin (FBX).usd (15.2 MB)

Hi Andrea!
I tried Audio2Face with Anime Characters too! The first few times i had some issues but i was able to fix it with this:

  1. Make sure the head mesh does not have sub-geometries, also known as GeomSubset: You can see it in the Stage panel, and this have an icon of a box with a red color. The subgeometries can be: Eyebrows, Eyelashes, etc. The mesh of the head must be a single one.
    If you have a single head mesh with eyebrows / eyelashes, you can try to remove these and leave only the head mesh.

  2. The head mesh must be on a scale of:
    X: 1.0 Y: 1.0 Z: 1.0 ✔
    Instead of:
    X: 100.0 Y:100.0 Z:100.0 🚫

This can be solved in Blender. If you import your model into Blender, then export it as FBX, and in the Export window you must check the box “Transform> ✔ Apply Transform” (It’s on the right side .)
Then you open Omniverse Create, convert the .FBX model to .USD.
Finally, open Audio2Face and open the .USD file that you converted, select the head, go to Property Panel > Transform > Scale , and make sure the head mesh has the scale of X: 1.0 Y: 1.0 Z: 1.0

Once you have this you can try using Character Transfer, and see if it worked!

1 Like

It is correct currently GeomSubsets are not supported. It will cause error / crashes if you have that in your mesh. This is something we are going to support in future update so GeomSubset will still work or at least not crash.

The other feature we are considering adding is a “Separate” function (similar to what you would have in Maya) so you can separate your mesh that are part of GeomSubset. Then you won’t have the GeomSubset issue.

For now, it is better to not have GeomSubset as suggested for the Target character mesh.

Side note: RE FBX, you can also convert it directly in Audio2Face app. We also included the Asset Importer extension, just load that from the Extension Manager and you get the same FBX conversion functionality like you do in Create. We included it for this purpose.