Hi,
we are following the outlined steps for setting up a CloudXR client with the Unity Plugin and it seems to stream correctly from the server in general but as seen in the attached screenshots the client renders it strangely on 2 floating planes to our left and right, any idea what can be wrong?
@AndreusNvidius can you offer any thoughts?
This is baffling. I think someone else may be seeing this behavior as well.
When you have these “floating planes”, is the streaming frozen / the content of the plane itself is static, or does it continue to update the texture rendered on the plane as streamed from the server?
It keeps updating and it seems to be a problem with the reprojection. When I disable it in the CxurReprojection shader, I don’t have a smooth image, but the panels are gone and I have a “normal” VR image.
Oh, I think maybe I know what’s happening in the case you documented above.
Generally speaking, “this bug” is that there is a mis-alignment between coordinate systems. It tends to show up as some fraction of the visible display being black, but the visible region stays in the same place in screen space, and what is seen continues to update with ongoing head motion (i.e., streaming is happening). The coordinate systems are rotated relative to each other, and usually horizontally, which is probably the most common way people reset their coordinate system with e.g. the Quest menu button. Exactly which coordinate system has gotten out-of-whack was a complete mystery to me, your disable-the-shader hint gives me a direction to pursue. Thanks!
What I think might be happening in the screenshots you show above is one or both of two things.
Alright, I may have a lead on this issue.
@dominic.ermark , @xaver.freiherr-loeffelhol : Can you please check what the Tracking Origin Mode is set to?
If it is NOT set to “Floor”, could you try this with the reprojection shader enabled, and see if the issue recurs?
Hi,
so we changed that but still have some problems. In general when I connect it looks fine and the Steam VR home menu is normal. But once the app starts now it is just moved to the top so the bottom half of my screen is black. But still when I look around it seems to move correctly. So some progress I would say :)
Does this happen with any app you launch from SteamVR, or just the app you’re targeting (which, of course, is the most important)?
I also tested it with the CloudXRServerSample.exe, when I use the device as the origin I get the floating planes per eye, when I use the floor the images seem to be correct, but the rotation offset feels a bit odd.
And as Xaver said above, for unreal it is even broken with the floor as the origin.
Okay, this is starting to narrow things down, appreciate all your help with this.
Unreal has some serious issues that I need to nail to the wall. One person has sent in a complete repro (project files and everything), so I will be investigating this as I have time.
In the meantime, could you test with a non-Unreal app, like The Lab or Google Earth?
Hi,
actually those apps I do not have access to. But also interestingly any Unity App I want to test with Cloud XR 4.0 directly crashes on startup if we do not connect with CloudXR before starting the App. I am also experimenting currently as suggested to first connect with the Windows Client to mock the behavior of the deprecated Client Descriptors, but the stream I get then afterwards is also pretty strange (but have to test a bit more to describe what happens there).
I believe that starting an app before an HMD is connected/identified is not a well defined / well supported path in SteamVR, or really even OpenXR. Unless the app handles the condition that OpenVR / OpenXR is not ready to be fully initialized, and gracefully sits looping waiting for that to happen, there’s not a lot that can be done.
The removed client descriptor was the workaround, and the workaround for dealing with its removal might be tricky (“pre-connecting” with a fake client that pretends to be the target HMD).
