CloudXR 4.0 Unity Plugin fails on the view offset

Gaming and Visualization Technologies XR – VR/AR/MR CloudXR (VR and AR Streaming)
1

Hi!

We’ve tested both the Unity-plugin app and the Android-build app of CloudXR 4.0 on the same Quest3.
We use SteamVR on the AWS EC2 server.

The former has an offset issue (image below) in its display, while the latter is rendered correctly every time.

Could you advise on adjusting settings within the Unity-plugin app to correct the viewing issue?

unity-plugin-NG-left-eye
unity-plugin-NG-left-eye2160×2160 101 KB

These two cases are different between resolution, IPD and Projections matrix on the attached server logs even using the same Quest3 and server (CloudXR Server - SteamVR Log).

Attached are logs of the server/client and screenshot images for investigation.
(XrPlatformFingerprint.txt / CloudXR Server - SteamVR Log / Streamer Client Log / Streamer Server Log / etc).

Please let me know if any other logs are needed.

  • OK-android-build.zip (290.4 KB)
    11:19:48.038 I (CxrDeviceDriver) Render parameters: 2064 x 2208 @ 72.00 (max 2476 x 2648)
    11:19:48.038 I (CxrDeviceDriver) Target queue time: 0.012000
    11:19:48.038 I (CxrDeviceDriver) Display type: Stereo, IPD: 0.063317
    11:19:48.038 I (CxrDeviceDriver) Projections: (-1.376382 1.376382 -1.428148 1.428148) x (-1.376382 1.376382 -1.428148 1.428148)

  • NG-unity-plugin.zip (217.8 KB)
    11:28:40.697 I (CxrDeviceDriver) Render parameters: 1680 x 1760 @ 72.00 (max 3024 x 3168)
    11:28:40.697 I (CxrDeviceDriver) Target queue time: 0.012000
    11:28:40.697 I (CxrDeviceDriver) Display type: Stereo, IPD: 0.063215
    11:28:40.697 I (CxrDeviceDriver) Projections: (-1.376382 0.839100 -1.428148 0.965689) x (-0.839100 1.376382 -1.428148 0.965689)

Apps were built exactly in accordance with the manual.
https://docs.nvidia.com/cloudxr-sdk/unity_guide/guide.html
https://docs.nvidia.com/cloudxr-sdk/usr_guide/cxr_ovr_client.html

2

@AndreusNvidius can you jump in with any suggestions?

3

ping as it is really a show stopper for roll out

4

@ogasawara.taku, @dominique.sandoz :

Your main issue is a bug that I’m currently trying to track down.

The secondary issue, that the projection matrices and resolutions do not line up, is a known issue. The short version is that the Unity plugin uses Unity’s OpenXR-based plugins to talk to the headset, while the native client uses the deprecated Oculus OVR API. The Oculus API provides a way to get physical resolution, while OpenXR only provides “recommended” and “max” resolutions. The former is intended to guide on-device 3D games, the latter is set to the texture size limit (8192x8192 or something).

I will need to dig in and see whether the resolution is exposed directly in the configuration.

5

Can you please try the steps described here and report back?

1 Like
6

Thank you so much for providing the workaround for the Tracking Mode “floor”!
It modified correctly the misaligned issue in our environment (Quest3 - AWS EC2), as shown in the attachment.

unity-plugin-tracking-floor
unity-plugin-tracking-floor1440×1440 86.1 KB

But, in the case of the Tracking Mode “floor,”
the view in the Quest slightly shifts back in the opposite direction after finishing a head turn, as if the inertia works.

Could you please let me know if you have any way to deal with this?
We tried changing some settings on Unity, but it had no effect.

Note that we had previously selected the Tracking Mode as “device” following the below:
https://docs.nvidia.com/cloudxr-sdk/unity_guide/guide.html?_gl=1*1muibfw*_gcl_au*NzI3NTU1MjEzLjE3MTU3NjAwNDk.#configure-xr-cameras-controllers-and-attach-the-cloudxr-plugin

On the Tracking Mode “device” the view did not move as if the inertia works.

7

Thanks so much for the response.

That documentation was based on a very early workflow, possibly predating a moment when I shifted to using OpenXR’s “stage” as the foundation of our streaming.

This “inertia effect” may be a new thing; have you fully re-enabled the reprojection shader / reprojection in the shader?