Hi,
We are trying to connect CloudXR to Unity and there is not a lot of documentation around it. Since there is no SDK for Unity to use CloudXR could you point toward an appropriate documentation ? Also, if there is another pipeline or a way to use Cloud XR with Unity what platform it will support is it Android and iOS? How does the cloud rendering works with Unity, should we connect it through the Unity XR plug-in framework ?
Please advise.
Regards,
Belal Badr
We have created apps that run on Steam, but we have not been able to use Unity with RTX raytracing, there has been some sort of DirectX12 bug
I’m also curious if there’s any documentation for this.
It´s in Unreal, but may be similar in Unity, I connected it using SteamVR in Unreal, with or without Raytracing. My only problem in any case is that all the opaque objects are semi transparent… But Raytracing works
Are you trying to use Unity to create a CloudXR Client in Unity or create a AR/VR server application that streams using CloudXR? If you’re looking at streaming a server application, note that CloudXR leverages OpenVR via SteamVR for streaming. Therefore, if you build a Unity project and use the SteamVR plugin for Unity (and the associated camera), the server will use CloudXR to stream the application to a CloudXR Client.
Hello,
In my case, I’m trying to stream CloudXR to a Unity client. We have a project where we’d like to switch from Quest native to CloudXR at runtime.
I was able to implement to whole pipeline in Unity (by studing the Quest native sample). Basically I blit the buffers directly to Unity left/right cameras. The buffers seem to be fine (I compared them with the native sample application in RenderDoc) but for some reason the stereo is broken when testing in the headset.
I was wondering if there is some matrix that needs to be applied at the very end or if anything has to be done in the context of Unity to get it work properly. I read in this forum that some people were able to get it work so any clue would be greatly appreciated. Thanks!
Following up, what is the officially supported way via Nvidia to support VRUI with CloudXR on a native, HMD-side client, such as Unity?
I’ve tried starting Unity Android activity that would launch the CloudXR activity in the same app. However, that doesn’t work as the VRAPI doesn’t like (crashes) to get started by Unity and then by CloudXR in the same process even though Unity seens to go through the onPause lifecycle.
Tried putting CloudXR activity in a different process, but Activity doesn’t display on-screen when started – though it seems to run. What works is Unity app starting CloudXR app, but that’s not Store friendly.
Manually just feeding the CloudXR frames to Unity seems complicated overkill, but it’s a solution – thanks.
Bumping that thread as after a couple of weeks of research and trial and error, we’ve reached the same point as you had. Stuck with VRAPI crashes,only thing we got to work is a duo-apk setup which is not what we are looking for, and we don’t have the resources or knowledge necessary to research and implement a custom solution. Sucks but hopefully NVidia will come up with their own Unity integration sooner or later.
There seems to be many independent efforts to port CloudXR to Unity – reading through the posts in this forum. I’ve started my own port because there’s no other way to get one.
Meanwhile, it doesn’t seem like a priority for Nvidia.
I belive it is rather clear that belalbadreid wanted to create a client side Unity app. If so, how can that be done? Would NVIDIA consider providing a Unity client app demo? I belive it would greatly help with adoption.
Actually an open protocol for Edgecloud XR streaming would be the best. I suggest that as as something the Metaverse Standards Forum could work on, and where NVIDIA could make a huge contribution.
With regards to server side apps: maybe NVIDIA can consider working with Unity, and perhaps Steam on resolving that DirectX 12 issue with OpenVR that hinders RTX ON mode for CloudXR server apps via SteamVR?