Hi all, I’m a graduate school computer science researcher currently working on XR security (XR device and XR apps). In order to automatically test software without depending on a real VR device, I’m trying to simulate the inputs from the XR device and directly feed them into the application. I have been searching for solutions (e.g., XR device simulation, driver simulation, etc.) but seems there is no full XR device simulation without an actual headset. Recently I came across CloudXR and notice that in the stack of CloudXR there is a virtual HMD driver, which to my understanding simulates a real HMD and makes SteamVR think that an actual HMD is connected. The interactions (e.g., controls, etc.) are fed from the client side via network packets, although the CloudXR client app still requires an actual HMD to run. So my idea is to simualte the network communication and interact with the CloudXR server without an actual HMD. In other words, I want to automate the interaction for software testing purpose by directly feeding the constructed network packets to the server. I understand that CloudXR is close sourced, but is it possible to get some understanding of the traffic between server and client (e.g., packet headers, encryptions, packet formats, etc.)? I saw from the log that there is an individual Nvidia GameStream log, so I assume it uses some alternative of GameStream protocol (which is also close sourced, but there is a GeForce Now version client side open sourced implementation called moonlight) for streaming contents, but that’s all what I got for now. Any help or suggestion is appreciated!
no unfortunately the streaming is all closed source and proprietary.
You >could< from a Windows client do all the fake simulation of an HMD that you want. The server doesn’t know any better. If you use windows client with -w, that puts it in windowed mode, where the mouse moves the view a bit (that’s the only input hooked up, windowed mode is really a debug mechanism). So you can fake whatever headset you want, fake the controllers, etc., so long as you respond as expected.
Thank you for your reply! So as far as I understand from your reply, the windowed Windows client can fully interact with the streamed game without a physical HMD connected? Or do you mean that I need to simulate the device response within the client to make the server think a physical device is connected? Thanks!
yes, the windowed client fakes an hmd and controllers. If you want to do automated testing, you just need to feed the right data into the client, and obviously modify the client to convert that data into faux ‘state’ of hmd and/or controllers.