We’ve begun integrating the CXR 4.0 Unity plugin (version 0.1.0) into a new Unity project, aiming to transition our native Android CloudXR client to Unity. While we’re excited about the potential benefits of a Unity client, we’re currently facing significant issues, particularly with the plugin’s basic example.
Our setup follows the official getting started guide, but we’re encountering inconsistent frame rendering on various Quest headsets (Quest 2, 3, and Pro). We’re using Unity 2022.2.12f1, noting a possible typo in the guide (recommending 2022.2.12fa). Also, the guide references the CXR 0.0.0 version, not 0.1.0, which might hint at compatibility issues or additional configuration needs.
You can review our implementation in our demo repo here. We’ve tested the CXR server both locally and remotely, encountering similar problems:
The app occasionally crashes on stream start, which we’ve temporarily bypassed with a delayed CXR connection start.
We’re also experiencing frame rendering issues when we do get the steam to work:
Frames are often misaligned, primarily showing black space.
Frames appear reversed, as if in a mirror. [See a video of this here]
We are really looking forward to leveraging Unity’s features for our CloudXR client. However, these issues are currently blocking us from going down this path. We would be extremely grateful for any help or insight on these issues.
@dominique.sandoz I have not found a solution to this or heard anything from Nvidia on if they are looking into this. It’s reassuring to hear that another person has run into this issue, though I don’t see how this plugin is working for anyone. I would love to see an example of this working for someone!
I had some luck by disabling Reprojection in the Cxru Reprojection Shader (_ReprojectionEnabled false). It gives other artifacts but at least the full field of view is covered, which I think does not matter because it is TERRIBLY SLOW, the latency between movement and update of the frame is unbelievable.
I was never more disappointed with something coming from Nvidia than with this.
Unfortunately, that is the behavior I am seeing as well when it does connect with the correct rendering. Unless an Nvidia dev can point out something I’m not doing correctly in my Unity setup, this plugin is not ready for primetime yet (as of 0.1.1). I’m really looking forward to being able to migrate my app to Unity, so hopefully the plugin will be updated soon to get the same level of performance that I get from the native implementation.
Please try to reset the camera rotation for XR Rig. If you start a new scene or anytime you change XR Rig make sure to reset the camera viewport to its intended forward. That seemed to fix the issue as long as no recentering is done using Meta button recentering.