Client for Streaming Audio Player

AI & Data Science Digital Humans (closed) Audio2Face (closed)
1

I would like to stream audio from my microphone to a Streaming AudioPlayer instance in A2F, but test_client.py only shows how to send FILES to A2F, not a live input.

Can anyone please point to a client/script that I can easily adapt for this purpose? Thanks!

1 Like
2

Hi, and thanks for this request - we’re seeing if we can work on something :)

Have you signed up for the Omniverse Insider program? Log in | NVIDIA Developer

Cheers,
Edmar

P.S. Fall GTC 2022 Sessions are now available for free viewing: GTC 2022: #1 AI Conference

P.P.S. Share your work in Omniverse! Submit to the Omniverse Gallery Page! Omniverse Gallery Submissions | NVIDIA

he / him | LinkedIn | Discord ID: Prof E#2041
Community Engagement for Omniverse: " A New Era of Digital Twins & Virtual Worlds with NVIDIA Omniverse" Video
What is NVIDIA Omniverse? Video | Check out the “Meet the Omnivore” blog series here

Omniverse Resources:
Omniverse | Documentation | Tech Requirements | Videos | Twitter | Discord | Twitch

The Omniverse documentation site has more info and links to all tutorial videos (under the “Learning and Feedback” category on left nav bar): Omniverse Platform Overview — Omniverse Developer documentation

PLEASE NOTE: If you are reporting a bug/issue, please provide OS, GPU, GPU Driver, the version of the app, and full log file (if applicable). For crashes, please zip and provide a link to your logs → C:\Users\ [YOUR NAME] \ .nvidia-omniverse\logs

1 Like
3

Appreciate building suspense but I need to make some technology calls to deliver on timelines.

Is live audio streaming something you are planning for a 2022 release or should I start doing micro batching and stitching together results?

4

Hi, have you tested the live mode in the audio player? This will convert your live audio into facial animation.

image

5

Yes it is great.

I this requires a microphone input instead of an audio stream.

My workflow is:

  • Audio stream
  • Into headless Audio2Face (this is what I am hoping to get information on)
  • Rendering of Face in headless mode and video capturing it
  • Streaming out resulting video to an endpoint (I will be using ffmpeg)
6

@adammpolak Have you tried the soundcard python package? It reads from microphone input and provides the data in the same format as PySoundfile which is used in the audio streaming example.

7

Will give this a shot, thank you!

@mati-nvidia how can I encode the resulting animation in real-time as a video stream?

I cannot find this in the documentation

8

@adammpolak Thanks for starting new topics for your follow-up questions. I’ve followed up there.

1 Like