Custom metahuman face animation

AI & Data Science Digital Humans (closed) Audio2Face (closed)
1

I am animating a custom metahuman in UE using a2f, using the standard workflow. Wondering would it lead to any improvements in the quality of animation, if I import my custom metahuman mesh and set up character transfer in a2f and then generate the blesdshape solve ? is that something that can be done to improve the quality of animation ?

2

We’ll test this internally and will let you know.

3

Here’s a test I did.

Top: Unreal Engine Meta-Human driven by Mark in Audio2Face (Very easy and quick)

Bottom: Meta-Human driven by the solved blendShape of the same character with 52 ArKit shapes generated inside Maya (Needs a lot of manual work)

Do you think the bottom was worth it?

4

Thank you that helps. I wanted to know if there is a room for imporvement, if I put in the effort. Looks like there is. Depending on the usecase it might be useful to go down that path. Thanks for calrifying. Is there any tutorial or steps to do this ? I do not use maya. I am thinking of importing the metahum to a2f and generating blendshape in a2f and finetuning it.

5

@Ehsan.HM and others in the group. Is there any video tutorial that can be followed to achieve this ? in some work flows which involve real life characters this could be very useful.

6

Do you mean a tutorial that shows steps to achieve the “MetaHuman ArKit” version in the video above? If so, we don’t have it. But I can prepare one.

7

@Ehsan.HM Indeed , that will be helpful for many, especially where realistic workflows are involved. Thanks for that initiation.

8

Here’s the tutorial. Hope it helps

9

Thanks that helps. Will give it a quick try and let you know.

10

@Ehsan.HM tried testing this workflow for our project. But there is a stopper for us, it seems we can only export the UE4 versions of the Metahumans to Maya. We have a UE5 custom metahuman created using mesh-to-metahuman workflow. Is there any way this asset can be used ? can we work with the base mesh used for creating the metahuman instead ? This was captured using Polycam. Any insights that yourself or the group members could shed on this would be much appreciated.