Audio2face Streaming Player animation in Machinima to combine Animations

[jebastin.raja]

Hi team,
I am using 2023.1.1 A2F and Machinima 2022.3.0 Applications.
I have used streaming player to created the A2F pipeline . In A2F, facial expressions are working well. Also this Character have the body animations. After I did the connecting points in Machinima all animations are working well.
I want to add multiple animation in the same Character in Machinima ,also I want to change the audio using streaming Player without exporting cache file in A2F. Is there any alternate way to connect A2F and Machinima in lively?
For example I want to animate the Character using audio and custom body animation in real time according to the user input (audio file).
This is possible ?

[Ehsan.HM]

Yes this is possible. As long as the character has the same blendShape targets as the one solved in Audio2Face, you should be able to stream the Audio2Face blendShape animation weights and use them in another app, e.g. Unreal Engine or Avatar Stream App.
Here’s a tutorial showing the process: Avatar Stream Application in Omniverse Audio2Face - YouTube

[jebastin.raja]

Thanks @Ehsan.HM . This will work for livestreaming with custom Avatar facial expression and multiple body animations ? for creating multiple body animation according the audio input Do you have any reference available . If available could you please share some reference ? that will be helpful.
Thanks.

[Ehsan.HM]

The facial expression can be applied to the body with any animation applied to it. As far as I’m aware you can only apply one single animation to a SkelRoot at a time. So you should switch between them or implement a way to mix animation clips.

[jebastin.raja]

Thanks @Ehsan.HM . I have an doubt . If the Character had body animation and the same one I tried with A2F Can I combine both the animation in Avatar stream ? It is possible ?

[esusantolim]

Yes it’s possible.
The menu that setup stream receiver in avatar stream up does indeed create new SkelAnimation.
But if you look at the OmniGraph created, you can direct to write the received float array to drive other SkelAnimation instead as long as that other SkelAnimation (the body animation) also contains the same blendshapes list.