CXR 3.0: Bidirectional Audio - how to receive sound in app?

Hi everyone

according to the latest release notes (CloudXR 3), it is possible to have bidirectional audio as in audio from the HMD to the server. Sending audio from the server to the HMD works flawlessly, the other way around not so much.

It is unclear to us, how to access the audio input from the HMD on our app running on the server; we were hoping CloudXR would expose a microphone to the system we could then grab as an input, but this is not the case neither on the AWS AMI provided by Nvidia nor our local servers. We’ve added the -ra/-sa flags, but so far have no clue how to process this data serverside (or where).

Is there anything we are missing?

Greetings

Dominique

Hi all -

We will look into this.

Thanks!

Veronica

Hi Dominique,

Have you looked at the Windows and Android Sample clients that show how this can be achieved using the CloudXR cxrSendAudio() API and the cxrClientCallbacks.RenderAudio() callback?

Hi Dominique,
Thank you for trying out the new back channel audio feature! The CloudXR server installer should be installing a virtual audio device driver on the server machine (see NVIDIA CloudXR Server — NVIDIA CloudXR SDK documentation) These devices can be read to and written from using the Windows Audio SDK of your choice. The examples Greg mentioned above are in our sample client (they use SDL) and can be used as a reference when implementing server side read/write audio as well.

A good way to test if things are working is to open up a teleconferencing app like Slack etc from within SteamVR’s desktop view and try to initiate a call. Apps that can read from a physical microphone on Windows should not need any changes to be able to read from our virtual microphone device. One thing to note is that the audio devices are only created when a client device connects (see NVIDIA CloudXR Server — NVIDIA CloudXR SDK documentation).

Please let us know if you have any further issues or feedback when trying this out.

Thanks!
-Arjun