Is it possible to convert streaming audio to facial animations(like into .mp4) and stream it, in real-time throught APIs or any pipelines.
I have to create a virtual AI assistant with video and audio as output, as a web service, any deveopment so far?
Hello @mabdullah3121! Have you checked out the Omniverse Avatar Cloude Engine (ACE)? You can also take a look at the Streaming Audio player extension You can also check out the Audio features in this document: Audio — Omniverse Audio2Face documentation
To date, I have seen no independent, third party demonstrations or tutorials of this functionality.
Please see the Virtual Beings Facebook group for more on AI virtual assistants with video and audio output.