Audio2Face Not Working - Model Scaling and Rotation issue

AI & Data Science Digital Humans (closed) Audio2Face (closed)
1

Can somebody please help me?

Im experiencing an issue with Audio2Face. After selecting all of the correspondence points on my model and mark_openMouth’s face and select Begin Mesh Fitting, my model’s scale and rotation goes crazy, see attached images.

This becomes “fixed” when I go through Post Wrap, and the Model returns to its original scale and rotation, but the points are now making the base of the model move instead of the face?

More information, this occurs in the console after Begin Mesh Fitting finishes running

2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands] Traceback (most recent call last):
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]   File "c:\users\carls\appdata\local\ov\pkg\audio2face-2022.2.1\exts\omni.kit.charTransfer\omni\kit\charTransfer\scripts\commands.py", line 331, in _do_fitting
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]     register_laplacian_func = register_laplacian_factor(L, landmarks)
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]   File "c:\users\carls\appdata\local\ov\pkg\audio2face-2022.2.1\exts\omni.kit.charTransfer\omni\kit\charTransfer\scripts\fitting\registration.py", line 67, in register_laplacian_factor
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]     return partial(_register_laplacian_solve, L, landmarks, landmark0_mask, spla.splu(lhs))
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]   File "c:\users\carls\appdata\local\ov\pkg\audio2face-2022.2.1\exts\omni.audio2face.player_deps\deps\py\scipy\sparse\linalg\dsolve\linsolve.py", line 339, in splu
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]     ilu=False, options=_options)
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands] RuntimeError: Factor is exactly singular
2023-06-21 13:53:58  [Error] [omni.kit.charTransfer.scripts.commands]
2

Scale after Mesh Fitting
Scale after Mesh Fitting1920×1200 425 KB

This shows the absolute scale of the model when you zoom out after Mesh Fitting. Why?

3

Disappear after Mesh Fitting
Disappear after Mesh Fitting1920×1200 407 KB

This shows the a2f templates either side of where the model head used to be, before pressing Post-wrap

5

Hi @JackISO and welcome to the forums!

Instead of scaling your main mesh, can you try putting it under a new group and scale this new group to match Mark’s size before starting the character transfer process?

6

Hello @Ehsan.HM , I am not scaling the main mesh at all, i am only translating the location of the a2fTemplates, not scaling or rotating anything else.

7

I see… Are you scaling Mark? From the snapshot, it looks like Mark has been scaled. Can you reset their scale to 1.0 and scale your character group instead?

8

Apologies, you are correct, in that screenshot mark has been scaled. However I have already attempted the same process with mark at scale 1.0 and it did not change the outcome.

9

No worries, are you able to share your scene, or the model so I can try it too?

1 Like
11

bust.usda (26.8 MB)

Included is the bust USDA File.

12

Thanks for providing your file. When I added your mesh as Skin in Character Transfer tab, it errored (a red triangle appears), if you hover over that triangle, it shows you the error.

Basically your mesh is made of multiple separate meshes, but Audio2Face needs one mesh with all vertices merged to work properly. You can separate and get rid of extra meshes you can follow the instructions given by the error.

Screenshot_2

1 Like
13

I have attempted this it has fixed the issue, thank you very much.

However, I would like to add that I have attempted the to do this (without singular mesh) on another machine and it has worked fine, with no scaling errors and the mouth works as intended.

But on this machine, it does not work without making it a singular mesh.
Could I provide you with some technical specifications of both machines to look at this in the future to help out?
The drivers used on both machines, the graphics cards? operating systems?

Thanks again @Ehsan.HM

14

Glad it works!
It’s strange if you were able it get Audio2Face to work propertly on a mesh with multiple geometry islands. Sure, would love to see both scene files and the machine specs for sure.

1 Like
15

I will provide these in this thread tomorrow, thanks for the help again 😊

1 Like
16

Machine 1 - Laptop - Razer Blade Pro 17 (2019).
Windows 11 Home (version 10.0.22621 Build 22621)
x64-Based
Intel i7-9750H @ 2.60GHz 6 Core
16GB RAM
NVIDIA GeForce RTX 2060 (laptop)
NVIDIA Studio Driver - Version 535.98

This machine was able to complete the mesh and post-wrap no problems without separating the mesh.

Machine 2 - Desktop PC
Windows 11 Pro (version 10.0.22621 Build 22621)
x64-Based
11th Gen Intel i9-11900F @ 2.50GHz 8 Core
32GB RAM
NVIDIA GeForce RTX 3090
NVIDIA Studio Driver 535.98

This machine was not able to complete the mesh and post wrap without the mesh being separated, and the cause of this post.

I have attached two files in an enclosed .zip. Machine 1 USDA and Machine 2 USDA. Thank you again for your help @Ehsan.HM. If you make any developments I’d be interested to hear why this was caused!

Audio2Face USDA.zip (96.5 MB)

1 Like
17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.