Originally published at: https://developer.nvidia.com/blog/simplify-and-scale-ai-powered-metahuman-deployment-with-nvidia-ace-and-unreal-engine-5/
At Unreal Fest 2024, NVIDIA released new Unreal Engine 5 on-device plugins for NVIDIA ACE, making it easier to build and deploy AI-powered MetaHuman characters on Windows PCs. ACE is a suite of digital human technologies that provide speech, intelligence, and animation powered by generative AI. Developers can also now access a new Audio2Face-3D plugin…
Really nice. We use it with ue5 and open ai realtime api. so my question is: is it possible to plug in an audio stream to a2f?
another question or more a request was:
would it be possible to provide an event for when “animate character from wav file” or also from wave object was finished?
another topic: i build one for android and after launching the app it immediatly closes and i get this error:
LogProperty: Error: Struct type unknown for property 'StructProperty /Game/MetaHumans/Common/Face/Face_AnimBP.Face_AnimBP_C:AnimGraphNode_ApplyACEAnimation'; perhaps the USTRUCT() was renamed or deleted?
If I look in the anim bp the node is called ApplyACEFaceAnimation. Don’t know why it’s looking for ApplyACEAnimation.
Unreal Engine 5.4.4 and ACE for 5.4
hi, i noticed in the demo (https://youtu.be/KCp70qzmGgY?t=1398) that the eyes are blinking even in idle mode. since i’ve been using the kairos workflow since last year, this has become one of our main concerns.
here’s my recent test, but the blink animation doesn’t seem to be showing: https://youtu.be/eXhQ2IZMbkQ
could you please guide me on how to achieve these types of blinking animations? thank you in advance!
Hi hi! I’m getting the following error in Unreal when playing the demo, no lipsync or audio plays:
[nvaim][error][a2x.cpp:159][audio2xCreateInstance] Error creating GRPC channel: Unable to establish connection to server. Current state: 3
Is there something I’m missing? I guess I’ll need to configure something to activate the audio2face service in the cloud before trying to connect through Unreal Engine. Is there a complete guide how to set up everything step by step and most importantly end to end? to be honest it is really hard to piece all toguether, spooning info from multiple different sources.
PS: It would be wonderfull if the coments on the youtube videos were not dissabled.
Hi, I am developing with NVIDIA ACE, but cannot find anywhere to download the NVIDIA ACE UE5 plugin version 2.4. Will it be released soon?