Audio to face doesnt sync to Unreal Engine through Livelink

AI & Data Science Digital Humans (closed) Audio2Face (closed)
1

First, I cant open any of the sample project, it just stucks at the opening pop up and crashs over and over(I tried to uninstall and reinstall audio2face, didnt help).

However, I managed to setup localhost with Nucleus, found the sample projects there.
Now I can open the Mark_resolved_arkit project, active streamlivelink, and connected it with Unreal engine, as shown below. You see lips moving in A2F but nothing happens in Unreal. It just doesnt sync.
But it works pefectly with I do livelink between Iclone and Unreal.

Please can someone help with this? Thanks so much.

2

@yumshbian have you tried to turn on the Enable Audio Stream option?

there should be several videos on YT that walk you through how to set up livelink between A2F and UE in addition to the official doc.

5

any luck toggling on the Enable Audio Stream option in A2F?

6

I tried switching it on and off, but didnt do anything

7

well, sound did got passed on to UE, but not animation.

10

2024-05-05 04:56:59 [1,525,269ms] [Info] [omni.audio2face.utility.scripts.utils] Skin mesh has been verified: /World/mark_head_model_hi_published/c_headWatertight_hi
2024-05-05 04:56:59 [1,525,270ms] [Info] [omni.audio2face.utility.scripts.utils] Waiting for Audio2Face Instance to be ready: /World/audio2face/CoreFullface – DONE
2024-05-05 04:56:59 [1,525,270ms] [Info] [omni.audio2face.utility.scripts.utils] Setting A2F track [len = 9.377 sec; samplerate = 48000]
2024-05-05 04:56:59 [1,525,272ms] [Info] [omni.audio2face.utility.scripts.utils] Resampling A2F track: 48000 → 16000
2024-05-05 04:56:59 [1,525,282ms] [Info] [omni.audio2face.utility.scripts.utils] Setting A2E track [len = 9.377 sec; samplerate = 48000]
2024-05-05 04:56:59 [1,525,283ms] [Info] [omni.audio2face.utility.scripts.utils] Resampling A2E track: 48000 → 16000
2024-05-05 04:56:59 [1,525,294ms] [Info] [omni.audio2face.utility.scripts.utils] Waiting for Audio2Face Instance to be ready: /World/audio2face/CoreFullface – DONE
2024-05-05 04:56:59 [1,525,294ms] [Info] [omni.audio2face.utility.scripts.utils] Setting A2F track [len = 9.377 sec; samplerate = 48000]
2024-05-05 04:56:59 [1,525,295ms] [Info] [omni.audio2face.utility.scripts.utils] Resampling A2F track: 48000 → 16000
2024-05-05 04:56:59 [1,525,304ms] [Info] [omni.audio2face.utility.scripts.utils] Setting A2E track [len = 9.377 sec; samplerate = 48000]
2024-05-05 04:56:59 [1,525,305ms] [Info] [omni.audio2face.utility.scripts.utils] Resampling A2E track: 48000 → 16000
2024-05-05 04:56:59 [1,525,315ms] [Info] [omni.audio2face.utility.scripts.utils] Async Task Done: <Task finished name=‘Task-4483’ coro=<A2FCoreFullfaceInstance._init_a2e_processor_async() done, defined at e:/audio2voice/audio2face-2023.2.0/exts/omni.audio2face.core/omni/audio2face/core/scripts/instance_fullface.py:592> result=None>
2024-05-05 04:56:59 [1,525,528ms] [Info] [omni.audio2face.utility.scripts.utils] The scale factor is adjusted for the target face, scale factor: 1.0000187274321588
2024-05-05 04:56:59 [1,525,711ms] [Info] [omni.audio2face.utility.scripts.utils] Audio2Face Blendshape data loaded /World/mark_bs_arkit/neutral
2024-05-05 04:56:59 [1,525,732ms] [Info] [omni.audio2face.utility.scripts.utils] Waiting for Audio2Face Instance to be ready: /World/audio2face/CoreFullface – DONE
2024-05-05 04:56:59 [1,525,732ms] [Info] [omni.audio2face.utility.scripts.utils] Setting A2F track [len = 9.377 sec; samplerate = 48000]
2024-05-05 04:56:59 [1,525,734ms] [Info] [omni.audio2face.utility.scripts.utils] Resampling A2F track: 48000 → 16000
2024-05-05 04:56:59 [1,525,745ms] [Info] [omni.audio2face.utility.scripts.utils] Setting A2E track [len = 9.377 sec; samplerate = 48000]
2024-05-05 04:56:59 [1,525,747ms] [Info] [omni.audio2face.utility.scripts.utils] Resampling A2E track: 48000 → 16000
2024-05-05 04:57:07 [1,533,424ms] [Info] [omni.audio2face.exporter.scripts.livelinksender] Socket connected: localhost, 12030
2024-05-05 04:57:21 [1,547,493ms] [Info] [omni.audio2face.exporter.scripts.audiosender] Socket connected: localhost, 12031
2024-05-05 04:57:23 [1,549,519ms] [Info] [omni.audio2face.exporter.scripts.audiosender] Initiating audio: WAVE:48000:1:32:3
2024-05-05 04:57:38 [1,564,489ms] [Info] [omni.audio2face.exporter.scripts.audiosender] Ending audio

11

do you mind uploading the most recent log in its entirety? you can find it in the path here - %userprofile%\.nvidia-omniverse\logs\Kit\Audio2Face

12

Hi Thank you so much. I finally got it work, it actually had nothing to do with A2F, I reimported the metahuman to the Unreal project, it fixed everything.

1 Like
13

just out of curiosity, was there anything done differently when you reimported the metahuman that made it work the second time around?

14

no, thatis everything I did, I guess it had something to do with animation blueprint. Because I was using the old metahuman for Icone livelink, probably I made some settings I cant remember.

15

oh, good to know. thanks for circling back with your solution!

16

No problem, thank you!