CloudXR 3.0 Sound Issues With Oculus 2

Gaming and Visualization Technologies XR – VR/AR/MR CloudXR (VR and AR Streaming)
1

Hi Guys,

We have a VM setup with windows 10 Professional, CXR 3.0 and Steam VR 1.20.4.

After restart of the VM, the system volume goes back to 0, here something even stranger happens:

  1. While the volume is still at 0, we can still hear system sounds and steamVR app sounds
  2. We can’t hear sound from an unreal engine build we are trying to run
  3. Closing the unreal engine game, and reopening it, fixes the sound - until the next reboot!
  • In the SteamVR setting menu, under audio, I can see “Nvidia HDMI Out (Virtual Audio Driver WDM)” as an option and I can select it manually, but also, this is the default driver selected by windows sound device settings.

Posting the full log here, but wanted to post this line: “Send audio: Enabled, Receive audio: Disabled, Foveation: Two pass” not sure if this means that’s anything is wrong with audio, but it “sounds” like it (pun intended).

NVIDIA CloudXR v3.0 (CL# 30188084), built on Jul 14 2021 21:08:56.

[05:27:14.164] Cleaning up older files.
[05:27:14.165] VisionServerDriver::Init()
[05:27:14.165] CloudXR single instance mutex created, first one.
[05:27:17.332] VisionServerDriver::RunFrame() – creating VisionHmdDriver
[05:27:17.332] VisionHmdDriver()
[05:27:17.332] VSync to photons: 0.0200.
[05:27:17.334] Adapter 0: Description=‘NVIDIA Quadro RTX 4000’
[05:27:17.479] Direct3D setup done.
[05:27:17.479] Send audio: Enabled, Receive audio: Disabled, Foveation: Two pass
[05:27:17.481] Starting FrameServerDX11.
[05:27:18.387] Retry loading stream server dll from cxr directory
[05:27:18.670] Stream Server: Net Stream Server Instance Created
[05:27:18.670] Using an app provided render device for video processing.
[05:27:18.670] Stream Server: CloudXR video streams created
[05:27:18.670] Stream Server: Input stream created
[05:27:18.670] Stream Server: Audio send stream created
[05:27:18.683] Stream Server: streaming instance started - waiting for a client…
[05:27:18.683] VisionHmdDriver::VisionHmdDriver succeeded.
[05:27:31.314] Stream Server: configuration update packet from client streamConn= 0000024869215040
[05:27:31.630] FrameServer::OnNewConnection()
[05:27:31.630] Client connected.
[05:27:31.630] Stream type 4 connected.
[05:27:31.630] Input Stream connected.
[05:27:31.630] Stream type 2 connected.
[05:27:31.630] Audio Send Stream connected.
[05:27:31.632] Stream type 1 connected.
[05:27:31.632] netStreamServer::connectVideoStream streamSource=0x6e98e760, m_streams[VIDEO_STREAM0]=0x6e98e760 m_connectedVideoStreams=1
[05:27:31.632] Stream type 1 connected.
[05:27:31.632] netStreamServer::connectVideoStream streamSource=0x6e98f3e0, m_streams[VIDEO_STREAM1]=0x6e98f3e0 m_connectedVideoStreams=2
[05:27:31.632] All Streams connected.
[05:27:31.643] Creating audio device
[05:27:31.650] NvAudCapAudioSource: Registering virtual endpoint/s
[05:27:34.117] VisionServerDriver::Start()
[05:27:34.127] Render parameters: 1832 x 1920 @ 72.00 (max 2198 x 2304)
[05:27:34.127] Target queue time: 0.012000
[05:27:34.127] Display type: 2, IPD: 0.064000
[05:27:34.127] Projections: (-1.150368 1.150368 -1.191754 1.191754) x (-1.150368 1.150368 -1.191754 1.191754)
[05:27:34.127] Controller type: 2
[05:27:34.127] VisionServerDriver::AddedHMD
[05:27:34.127] VisionServerDriver::Added all controllers
[05:27:34.138] VisionHmdDriver::Activate(0)
[05:27:34.138] FrameServer::Start
[05:27:34.138] VisionHmdDriver::GetComponent(IVRDisplayComponent_002)
[05:27:34.138] VisionHmdDriver::GetComponent(IVRDriverDirectModeComponent_005)
[05:27:34.138] returning IVRDriverDirectModeComponent interface
[05:27:34.138] VisionHmdDriver::GetComponent(IVRCameraComponent_003)
[05:27:34.138] VisionHmdDriver::GetComponent(IVRVirtualDisplay_002)
[05:27:34.138] VisionHmdDriver::GetProjectionRaw
[05:27:34.138] VisionHmdDriver::GetProjectionRaw
[05:27:34.142] VisionHmdDriver::GetComponent(IVRControllerComponent_001)
[05:27:34.142] VisionControllerDriver::Activate(1)
[05:27:34.143] VisionControllerDriver::GetComponent(IVRDisplayComponent_002)
[05:27:34.143] VisionControllerDriver::GetComponent(IVRDriverDirectModeComponent_005)
[05:27:34.143] VisionControllerDriver::GetComponent(IVRCameraComponent_003)
[05:27:34.143] VisionControllerDriver::GetComponent(IVRVirtualDisplay_002)
[05:27:34.160] VisionControllerDriver::GetComponent(IVRControllerComponent_001)
[05:27:34.160] VisionControllerDriver::Activate(2)
[05:27:34.160] VisionControllerDriver::GetComponent(IVRDisplayComponent_002)
[05:27:34.160] VisionControllerDriver::GetComponent(IVRDriverDirectModeComponent_005)
[05:27:34.160] VisionControllerDriver::GetComponent(IVRCameraComponent_003)
[05:27:34.160] VisionControllerDriver::GetComponent(IVRVirtualDisplay_002)
[05:27:34.169] VisionControllerDriver::GetComponent(IVRControllerComponent_001)
[05:27:34.169] Handling defered client chaperone setup from RunFrame.
[05:27:34.169] Programming client Chaperone
[05:27:34.226] DestroyAllSwapTextureSets(pid=8572)
[05:27:36.169] CreateSwapTextureSet(pid=8756, res=2196 x 2304)
[05:27:36.170] CreateSwapTextureSet(pid=8756, res=2196 x 2304)
[05:27:36.216] Exchanging the sync texture 0000000000000000 with 0000000040000982.
[05:27:36.300] DestroyAllSwapTextureSets(pid=7396)
[05:27:36.355] DestroyAllSwapTextureSets(pid=9056)
[05:27:36.356] Set a chaperone with origin type 1 and area 1.500 x 1.500 m.
[05:27:36.404] DestroyAllSwapTextureSets(pid=6060)
[05:27:37.203] DestroyAllSwapTextureSets(pid=8768)
[05:27:44.043] CreateSwapTextureSet(pid=8708, res=1832 x 1920)
[05:27:44.049] CreateSwapTextureSet(pid=8708, res=1832 x 1920)
[05:27:44.204] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:27:44.216] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:27:44.697] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:27:44.749] Recreating the composition texture for eye 0. From 0 x 0 to 2196 x 2304. Source res: 2196 x 2304.
[05:27:44.761] Recreating the composition texture for eye 1. From 0 x 0 to 2196 x 2304. Source res: 2196 x 2304.
[05:27:55.500] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:27:55.578] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:27:58.294] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:27:58.619] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:27:59.834] Failed to acquire the surface mutex, 102
[05:27:59.834] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:28:10.227] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:29:17.176] Failed to acquire the surface mutex, 102
[05:29:17.176] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:29:17.322] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:29:22.356] Failed to acquire the surface mutex, 102
[05:29:22.356] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:29:22.506] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:29:26.531] Failed to acquire the surface mutex, 102
[05:29:26.531] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:29:26.667] Exchanging the sync texture 0000000040000982 with 0000000040000FC2.
[05:29:40.219] Failed to acquire the surface mutex, 102
[05:29:40.219] Exchanging the sync texture 0000000040000FC2 with 0000000040000982.
[05:29:43.218] DestroyAllSwapTextureSets(pid=8708)
[05:29:59.707] Stream type 1 stopped.
[05:29:59.707] Stream type 1 stopped.
[05:29:59.707] Stream type 4 stopped.
[05:29:59.707] All streams stopped.
[05:29:59.707] Stream type 2 stopped.
[05:29:59.707] Client disconnected.
[05:29:59.709] Skipping nvstPushStreamData for eye 0 because the connection has been torn down.
[05:30:02.681] DestroyAllSwapTextureSets(pid=8712)
[05:31:28.865] VisionServerDriver::EnterStandby()
[05:31:28.974] VisionServerDriver::LeaveStandby()
[05:31:28.974] DestroyAllSwapTextureSets(pid=7204)
[05:31:29.052] DestroyAllSwapTextureSets(pid=3728)
[05:31:29.145] DestroyAllSwapTextureSets(pid=3524)
[05:31:29.146] DestroyAllSwapTextureSets(pid=3796)
[05:31:29.150] DestroyAllSwapTextureSets(pid=9552)
[05:31:29.152] DestroyAllSwapTextureSets(pid=3756)
[05:31:29.152] DestroyAllSwapTextureSets(pid=3584)
[05:31:29.186] DestroyAllSwapTextureSets(pid=8988)
[05:31:29.248] DestroyAllSwapTextureSets(pid=8756)
[05:31:29.338] VisionControllerDriver::Deactivate
[05:31:29.338] VisionControllerDriver::Deactivate
[05:31:29.338] VisionHmdDriver::Deactivate
[05:31:29.338] deactivated
[05:31:29.506] VisionServerDriver::Cleanup()
[05:31:29.506] FrameServer::End
[05:31:29.506] NvAudCapAudioSource: Could not get audio frame (error 10)
[05:31:29.659] Stream Server: stopped the server instance
[05:31:29.659] Stream Server: stopped the server instance
[05:31:29.659] CloudXR single instance mutex released and closed.
[05:31:29.659] DLL_PROCESS_DETACH

3

Hi there!
Thank you for trying out the CloudXR 3.0 SDK. The posted log snippet does not seem to show any issues with the CloudXR server, “Send audio: Enabled, Receive audio: Disabled, Foveation: Two pass” means that audio from server to client is enabled and back channel audio from client to server is disabled which is the default behavior. On a server with no attached audio output the CloudXR server attempts to create a virtual audio device, “Nvidia HDMI Out (Virtual Audio Driver WDM)” The lifecycle of this device is that it gets created when an HMD first connects and we set it to the default audio output for the system. The volume being set to 0 is a side effect of how the device works, since it is virtual its volume cannot be changed. Could you give me a few more details on the sequence of steps you perform to start a CloudXR session to this server?

Thanks!
-Arjun