Python API/SDK to inject custom parameters. in omniverse audio2face

Hello, I am currently working with omniverse audio2face and I was wondering if there’s an external API/SDK (python or node or whatever language) that can be used to input the audio file along with the post processing parameters.

Thank you

Hello @ali.ak007! Welcome to the community!

Audio2Face is built of several components that are meant to be modular depends on the need of each app. We are exploring different scenarios on how to get Audio2Face into different usecases in Omniverse, and our development priority currently is to focus on Audio2Face integration throughout Omniverse apps.

You are able to access many parts of Omniverse and Audio2Face through Python and OV python SDK. If you have particular questions, let us know.

You can use python to access almost all of the UI so loading audio and edit post processing parameters is supported.

Let us know what your use cases are and we can help more

Thanks for your reply,

I want to be able to stream an audio file from a python script to audio2face, not uploading it from audio2face directly.
Let’s say for example, I want to speak or record my voice, and input the resulting audio to audio2face and have the avatar repeat what I said or just say new stuff in real time. (I say hello avatar, and it will reply Hi for example), so more like a back and forth conversation with the avatar.

The final product will be like the MISTY bot, so I just want to know how to stream the processed audio to audio2face using external libraries and plugins.

If you can also elaborate more regarding the python SDK, it would be great.
Thank you!

If you like streaming audio, that is a feature we are adding now, specifically to make TTS streaming input easier.

If you just want to use python to load audio, that is doable. But you will need the UI loaded I believe. A2F has some dependencies right now looking for inputs from the UI at the moment. So sounds like from your use case, you may want to wait for the new streaming feature that is coming.


1 Like