I am having issues creating blendshapes in audio2face. I have set up the correspondence values, mesh fitting, post wrap, jaw motion prep, tongue mesh fitting, and tongue mesh post wrap. None of these generated any errors or issues. I then added the a2f pipeline using Claire. At this point, I generated the 46 blendshapes. Although the 46 faces were created, the blendshapes all looked the same. 46 identical facial expressions. Also, when i play the audio, the driver face, Claire, moves correctly, but the Rain face I imported from blender does not move. The eyes pop in and out, but the face doesnât move. I have tried many different things to fix this, but nothing has worked. i canât get the driver mesh to drive the target mesh properly. I am using audio2face 2023.1.1.
@justin_renfro i am just another OV user here; would it be possible to provide more information with your setup like posting screenshots of your A2F settings? that would give the mods/devs more context as to what you are working with. without seeing anything, i can only make an educated guess that something may have been âoffâ before or during the character transfer step.
until then, i can only suggest visiting and following the two-part tutorials closely by Charles - https://www.youtube.com/watch?v=Etztivmjny4 (if you havenât had the chance already, that is). his tutorials working with Rain were pretty informative.
Thanks for replying. I just provided a screenshot of the basic setup and the identical blendshapes. I watched the videos by Charles, but I could not figure out what I did incorrectly. In the previous videos using the early versions of audio2face, it seemed like the a2f pipeline was automatically created with the Mark head. In the latest version, it isnât automatic. Maybe there is a step that I am missing with creating the pipeline. Also, in his video, he cut the head off of a complete body mesh. I used the Rain head that was already separated. However, Iâm unsure about whether or not that should make a difference. I was thinking that the issue could be axis alignment or scaling.
Hi @justin_renfro Can you try redoing the setup from scratch and this time generate blendShapes before applying the A2F Pipeline? The latest A2F has a bug where blendShapes are wrong if you apply A2F pipeline before generating them.
Also make sure to set the BlendShape solver weights values to 0.01 to compensate for the scale change between Blender and A2F.
For the eyes, make sure to set the eye meshes are selected and Pivot is set to Compute from Selected Mesh.
Hi Ehsan. I tried applying the blendshapes before the pipeline and the other suggested steps. I still could not get anything working correctly. The generated blendshapes were still all identical. I am also confused about the way the blendshapes are generated. Isnât the a2f pipeline needed in order to drive the driver head, which in turn creates the blendshapes on the target mesh? Also, I still cannot get the a2f pipeline driver mesh to drive the target at all. The mesh fitting and post wrap seem to work, but there is still no connection with the a2f driver head. I have another question as well. If I load the file alien_fullface.usd, shouldnât the alien head already be linked to the driver Mark head. When I play the default audio file, the Mark head moves, but the alien head does not at all.
to confirm on my end, âalien_fullface.usdâ does have Markâs and the alienâs head linked.
Hi. Do you have the latest version of audio2face? 2023.1.1? Also, when you open the file, does the alien head speak the default audio immediately, or do you need to do something beforehand?
i am on 2023.1.1 as well and nothing plays when file opens (not until i hit play, that is). i noticed you have cache off, did you have any issue installing the OV Launcher/A2F? just in case, see if you can upload the latest log file in case thereâs anything related to this odd behavior. you can find the log file here:
C:\Users\%username%\.nvidia-omniverse\logs\Kit\Audio2Face\2023.1
Hi, I have cache on now, but I still canât get the alien_fullface to work correctly. Only the driver Mark head moves. There is no connection to the alien. I tried allison_fullface.usd, and it has the same issue. The target mesh doesnât move at all.
sorry, i only pointed out the cache in order to determine whether there were any installation issues. while having cache is strongly recommended, i cannot say with a level of certainty that itâs the root of the problem. if you can provide the log as well, it would provide the mods/devs with more context to what lead to the problem you are encountering (because it might not be related to what you did but related to software/hardware).
@justin_renfro Could you please send your latest Audio2Face log file? It should be located inside C:\Users\<USERNAME>\.nvidia-omniverse\logs\Kit\Audio2Face on Windows and ~/.nvidia-omniverse/logs/Kit/Audio2Face on Linux?
I have included my latest log file.
kit_20230927_132329.log (18.8 MB)
can you try upgrading your graphics driver for your RTX 2060 (currently you are using 512.77)? i would also go into your Windowâs device manager and disable the intel card under âdisplay adaptersâ. see if that makes a difference in A2F after these two steps.
Hi. Thank you so much for your help! Your suggestions worked. The alien_fullface file works correctly now.
fantastic, hopefully the Rain tutorial works for you now đ€
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.