Im new to audio2 face, and I bought my rtx 3070 ti solely to use it to capture facial animation for unreal characters. I managed to create a animation for a character I have in a scene in unreal engine 5, but my problem is that I just cant load it correctly into my character in unreal. I did create a server and tried sending the USD animation file to unreal, but all I get is the face mesh with no animation. I searched for tutorials that explain this procedure, but could not find any on youtube, they all explain the process for metahumans, but not a custom cartoon character. I hope somebody can help me or point me to a tutorial on how to send my animation to unreal and load it into my character there.
Hello @falconking, welcome to the forums! Hopefully I can help you out with your problem. Did you import the custom character you want to transfer animation onto into audio2face? Metahumans work without having to import to audio2face b/c they use blendshapes that match what metahumans have defined (arkit curves). If you are attempting to use a custom character that has blendshapes that are different from those curve names, you will need to import the character into audio to face and do a character transfer. Then you can generate curves for the blendshapes you already have on the character. If there are no blendshapes for the character, you will need to use audio2face to create the arkit curves for your character and then re-import those onto your custom character. This tutorial walks through the steps of doing the character transfer.
BlendShapes Part 4: Custom Character Conversion and Connection in Omniverse Audio2Face - YouTube
Hello. I did the character transfer and successfully animated the character. my problem is when I try to send it to unreal and how to “load” the animation into my character’s face. I haven`t found a tutorial that explains that part.
Did you use your characters existing blendshapes or the Mark character blendshapes?
Also, I cant make the grey character blink, and neither my character. I noticed the default character when I open the program has eyes, and he moves them and blinks, but the template man dont have eyes and doesn`t blink. how can I make my character blink and move her eyes ? is it possible ? I cant find any tutorials about this either.
My character has blendshapes but I used the characters (Mark) following one of the tutorials. I still don`t know how to use my character’s blendsahpes.
I can check on the eye movement and blinks… I believe they are being controlled separately through actinograph, but I’ll have to look into that further. Are you on the latest build of Audio2Face?
yes, the last one, latest launcher, etc.
So if you use Mark’s blendshapes, then they will not map back to your character in Unreal onto your character. At that point you would need to import new blendshapes onto your character that match the ones on Mark. However, if you watch the video clip… around 3 minutes in, they transfer from the custom character that is mapped to Mark onto another version of the custom character. When they do this, it will use the blendshapes of the character used in the blendshape mesh option. That way you should be able to map back to your existing blendshapes.
I tried exporting to blender the USD animation of the character and it loads correctly, I can see the animation playing in the imported face. my problem is when I try to load the animation to unreal. I don`t k now the procedure or or how to apply it to my model there.
I’m going to test importing into Unreal, however, if you are not using the custom character blendshapes it will not re-import onto the character, it will be a separate asset.
Ill try using the characters blendshapes now.
Hello @falconking. I also had issues trying to import the animation using the import facial animation, even with the same blendshape names. I am putting in a ticket to look into this, I will update you with progress. In the meantime, you can use this as a work around. Deactivate the Omniverse USD plugin and make sure the Epic USD Importer is activated. Although it can’t import just the animation clip, it could bring in the data with skeletal mesh. In Audio2Face create a new scene. You can delete the default light. Drag your USD file of your custom character with the blendshapes into the Stage. Then drag the USD animation clip into the Stage as well. Apply the animation onto the character by dragging the animation clip from the Stage onto the character in the viewport.
You should see some change in the character constituting the animation being applied. Save out this new USD file. It should be small since it’s just referencing the other USD files. In Unreal… import that USD file.
Make sure to have Geometry clicked on, because it will not import the animation without it. This should import a skeleton/skeletal mesh and animation clip. I hope this helps until we get things ironed out on our side.
Thanks!
Thanks, I will try this method. Im having another issue now. I noticed my face model comes from blender without the shape keys (blendshapes) I tried exporting as FBX and exporting as a USD from the blender version from onmiverse launcher, but I cant the the blendshapes in audio2face. that is why I could not do the transfer of the blendshapes to the original face. Whats the propper way to export the face from blender with all the keyshapes ? should I use USD or FBx ?
Also, since this method does import the skeleton and creates a new skeleton mesh, you technically don’t need to use the custom character blendshapes anymore.
I tried this, but when I try to load the animation in unreal using the Epic USD plugin, all I get is a material or texture asset, no animation of any kind. I`m using the latest UE version.
I was able to get that setup to work. Would you be able to upload the USD file of the character and the USD animation file created from A2F? Thank you!
sure. can I upload it here ?
Yes, I think zipping up and uploading here would work well. Thanks
I will upload it now