Apple CloudXR 3.0 camera view has 2 fps

Gaming and Visualization Technologies XR – VR/AR/MR CloudXR (VR and AR Streaming)
1

Hey

I’ve tried today XR 3.0 lib. Nice update to the app !
When I try to run it tho, I got about… 2-4 fps in camera performance. The XR 3d render/stream runs smoothly. But the camera feed is a slide shot. I tried both debug and release mode.

The camera is smooth before I specify point on the grid. After that slide show.

I also have a sea of >
2021-07-23 07:50:23.786290+0100 CloudXRClientObjC[1669:1082367] Metal GPU Frame Capture Enabled

2021-07-23 07:50:23.787716+0100 CloudXRClientObjC[1669:1082367] Metal API Validation Enabled

2021-07-23 07:50:31.974660+0100 CloudXRClientObjC[1669:1082367] Camera permission request grant status: 1

2021-07-23 07:50:42.008672+0100 CloudXRClientObjC[1669:1082115] fopen failed for data file: errno = 2 (No such file or directory)

2021-07-23 07:50:42.008859+0100 CloudXRClientObjC[1669:1082115] Errors found! Invalidating cache…

…

2021-07-23 07:47:15.971254+0100 CloudXRClientObjC[1664:1080212] FATAL ERROR: could not find array index of cameraPoseMatrix

2021-07-23 07:47:15.971430+0100 CloudXRClientObjC[1664:1080212] Mismatch: looking for 0.568533 0.904770 1.226193

2021-07-23 07:47:15.971506+0100 CloudXRClientObjC[1664:1080212] Mismatch: using 0.568445 0.904795 1.225976

2

Hi,
What AR application are you using on the server?
-John

3

No app. Just steamVR default room that shows up when VR device connects. All I see are the circular lines surrounding me.

The lines render fine at 30/60fps. The camera feed behind at 2fps.

But Vred Pro 2021/2022/etc is also lagging.

5

So @CXRDev , what should I change in the settings to have a non-stuttering image from the real camera? I have AR application(written in unity) and IOS client.

6

Slawek, in the file CXRViewController.m, replace lines 693 and 694 with this code block:

        int writeIdx = cameraImageCacheWriteIndex-1;
        if (writeIdx < 0)
            writeIdx = cameraImageCacheSize-1;
        [encoder setFragmentTexture:cameraTextureCacheY[writeIdx] atIndex:0];
        [encoder setFragmentTexture:cameraTextureCacheCbCr[writeIdx] atIndex:1];

Thanks,
John

1 Like
7

Hey

I’m very happy with the image I see behind my XR view. I want to see my camera feed behind as I’m making AR app using a workstation Vred Pro engine. The NvidiaXR 1 & 2 worked fine but had issues with tracking on apple. Now the tracking appears to be “working” (hard to test with shuttering camera view) but then the camera view is broken :D

I have to port the xr3 ver to my app and test it there.

I just wonder why the example is broken.

8

Hi, can you try the change I recommended above?

Thanks,
John

9

Will do will do! Thank you for the recommendation :- )