So, I tested this with CloudXR 3.0 and 3.2, just now, on both Quest 1 and Quest 2, and it’s a 100% repro for me. All you have to do is try to hold the two controllers’ rings together and they have a gap between them.
The same exact controller pose when you pause CloudXR client, even temporarily, instantly makes the controller poses correct.
Is this a known bug, and if so, is there a workaround?
Let me know, thanks.
Here’s a quick video showing the discrepancy between the controller poses between the local VRAPI-reported pose and that which SteamVR sees through CloudXR libraries:
You can clearly see here, as I toggle back and forth, that the poses don’t match.
This is the same behaviour in 3.0 as 3.2 and several of our customers reported having to use SteamVR Advanced Settings to adjust the controller poses, in order to make the controllers line up properly. This shouldn’t be necessary, right? Anyway it looks like there’s something wrong to me so I’d like a confirmation. I tested it on several machines and on Quest 1 and 2 and several versions of CloudXR and SteamVR. All the same issue.
@GJones-NVIDIA-XR-Team @Veronica_NVIDIA any idea here? This is really a problem for us.
@tegradave Hi Dave, I was just wondering if this is a known issue. The repro steps are very straightforward. Just hold the controllers together ring-to-ring and see if there’s a gap or not.
I’ve been experiencing the same issue. Have you managed to fix it?
Not yet, we put it lower priority for now but have more serious (actually show-stopper) problems with the Quest client connecting / disconnecting. There’s a bug in the connection handling code somewhere we think.
I would love to hear from Nvidia about this, if it’s a known issue on their end and if there is a fix incoming at some point.
I did some comparisons between AirLink and CloudXR 3.2 in SkyrimVR, and the controller poses are different there too, in the amount of forward rotation that’s applied.
In AirLink, if I hold a sword or a torch or a mace, it sticks upwards as if I were holding the grip using the touch controller’s grip axis, in the exact same direction. In CloudXR, it points forward by maybe 30 degrees.
you can check client code(at least on oculus as I saw), they have manual offset where you can edit by yourself.
yeah, I’m aware, I just don’t understand why the default value was chosen to be that, as it doesn’t match what SteamVR games expect. So I consider that a bug in the sample code. What worries me more is that the value is actually correct for controllers in SteamVR home (at least as far as rotation goes), but incorrect for in-game. Which would be a far more serious, platform-level bug with CloudXR.
I applied some rotation and position offsets to get it much closer.
For example, the default rotation offset in the sample is set to 0.4 radians, I find 0.8 is much closer to the expected value so that the controllers are pointing the same direction. There is also a position offset needed, in worldspace rotated by the original controller rotation quaternion (prior to the rotation offset). I’ll share the values here to save anyone else the headache. A bit more attention to detail is warranted here I think, the rotation is way off in the sample and there is no translation, which is totally necessary as well.
How exactly do you offset the position? I can’t seem to find the area that lets me do that, and the pose is way off.
The way I did it was by rendering controllers locally and remotely at the same time and lining them up visually, using a tool I created to adjust and deduce approximate translations and rotations to the controller poses as they are sent to CloudXR client library:
Controller translation offsets (in GL right handed Y+ is up coordinates): (-0.007, 0.042, -0.014)
Controller rotation offsets (in degrees, euler angles): (-6.402, 0, 0)
Honestly it’s kind of sad that after all this time this wasn’t fixed. It doesn’t take a detective to notice the controller poses are way off by default, so you have to manually fix them. I hope CloudXR 4.0 solves this but if it doesn’t, I will try to get this bug escalated because it’s unacceptable for controller poses to be wrong.
The CXR library should act as a transparent transport layer from local to remote poses, as if you had plugged in the headset to your PC and were using it directly, nothing more, nothing less.
Can I borrow that code lol
Try just adding -0.6402 to X (or Z) when your code when setting the cloudxr controller poses.
since we’re sending raw pose data, it is possible that steam is making assumptions it shouldn’t and is further shifting them. Yes, we definitely can see the ‘controllers don’t touch in steam’ issue, we’ve just had much bigger issues/features as our focus, so this has been on the backburner but not investigated as yet. I’ve raised up the visibility, unclear if we’ll have cycles in 4.0 to debug and address (again, unclear why they aren’t in the right places, might be steam interop, might be we’re getting poses relative vs absolute and not translating between the two wherever it’s needed…0.
There’s a question of whether the SteamVR poses more closely relate to Aim vs Grip poses in OpenXR. We are using Aim but IMO for the controller itself it makes more sense to use Grip, semantically at least.
However, in OpenXR, we really want the two, Aim vs Grip poses, to be fully independent as they are returned by the local runtime. I hope CloudXR 4.0 more closely mimicks OpenXR’s way of thinking of these things.
Will CloudXR 4.0’s client code samples still use the now-ancient OVR api? Or will the client apps be migrated to OpenXR too (not just on the server side). We already have an OpenXR client that uses CloudXR but there are numerous issues / unknowns specifically vis-a-vis not only controller poses, but also IPD / FOV values not updating properly per frame (using the appropriate cloudxr flags seems to be a no op), and various resolution vs refresh rate wavey lines issues (reported in other threads). But the Aim poses are off by like 30 degrees. Try opening SkyrimVR with AirLink and then try it with CloudXR, you’ll see, holding swords are in a wildly different direction.
It’s been a real nightmare navigating what resolutions actually minimize the horizontal or vertical wavey lines problems. It’s super strange also that disabling foveation entirely and using my local Wifi connection results in inferior image quality to using foveation. Foveation should simply save bandwidth, or, at the same bandwidth, permit higher quality in the foveal region. But since Quest 2 / XR2 has a 150 mbps HEVC decoding bitrate limit in hardware, that’s easily saturated by a good local router so it should be possible to disable foveation on CloudXR and get a similar high quality edge-to-edge result as Virtual Desktop or Air Link does. But it doesn’t. It looks significantly worse with foveation disabled at all resolutions, regardless of whether the bitrate is throttling performance / image quality potential.
