Hi, I would like to use audio2face to animate a chatbot I’m creating. I already have the NLP part of the chatbot, as well as the speech2text and text2speech parts. In fact, to interact with the bot locally, what I do is to launch an Audio2Face window, configure the streaming player and send audios to it via Python. However, I’d like to be able to integrate this into a webpage, rendering the animated face in real-time, as I do in the Audio2Face studio, but via web. I’ve seen that this must be possible due to this demonstration video: Intelligent End-to-End AI Chatbot with Audio-Driven Facial Animation - YouTube However, I haven’t been able to guess how to do something like that in the video… I’ve seen you can export the usd cache from audio2face, but that’s not what I need, as I’d need to animate the usd in real-time; that’s useful only for applications where the audios are fixed, not generated by an AI model…
Hello @usuario3863! Welcome to the Community! Wow super excited to hear what you are working on! I send your questions over to the development team to help get you some answers!
Hi @usuario3863 , thanks for the ping.
You can use one of the streaming extensions Omniverse provide if you like to stream through a webpage. We did recently find some of those extensions are not exposed in A2F and we have a hot fix patch coming that fixes that.
Upon doing that, just put the a2f app to full screen and stream the output to the web, that should be it. In the Misty demo above, that was just Omniverse running full screen viewport, that’s it.
hope this helps
Would it be possible to create this kind of AI bot or like Misty but for an AR application? What’s the performance requirements to run this?
Hey, I am working on the same project. It would be great if we could connect!
I am also working on similar project, is there any update for this? thanks
Few options there, ACE allow you to do cloud based runtime of A2F but is in closed beta right now. Public release will have to wait a bit.
You can also look at the headless, rest API of A2F. You can then stream those results to other engines
For using OV, you can build your own app with just the renderer and host in cloud and stream results out. Those are all part of the latest A2F already released.
Which update exactly do you mean?
Is there any update to this thread ? I would like to know how to get some training on how to integrate python to the Audio2Face or a location where I can get some guidance on how to integrate and expose python for text to speech and speech to text integrations like mentioned above. It does not have to be a web application and could be just A2Face running at time of use. Please let me know how to get some help on this