Issues with Object Placement in AR/VR Projects using CloudXR and Unreal Engine on iOS

Gaming and Visualization Technologies XR – VR/AR/MR CloudXR (VR and AR Streaming)
1

Hello, CloudXR Community,

I’m reaching out for advice regarding an issue I’m facing with augmented reality (AR) and virtual reality (VR) projects on iOS using CloudXR. After setting up the CloudXR server runtime along with SteamVR and running the AR sample project provided by CloudXR, I’ve managed to stream the scene to my iPhone, experiencing the content in augmented reality. However, I’m encountering a significant challenge: as I move the phone camera around, the virtual objects do not maintain their correct placement within the real world.

To further investigate, I created a simple VR project in Unreal Engine using the OpenXR plugin. This project contains a simple cube positioned at the origin (0,0,0). Running this VR project, I noticed the same issue—the cube is visible in the iPhone scene but its position does not accurately sync with the real-world environment as I move the phone.

I have confirmed that all software and plugins are up to date, and I followed the setup instructions for both CloudXR and the Unreal Engine project meticulously. Here are the details of my setup:

  • CloudXR Server Runtime version: 4.0.0
  • SteamVR version: 2.4.4
  • Unreal Engine version: 5.3.2
  • iOS version on iPhone: 17.3.1
  • iPhone model: iPhone 13 pro max
  • Windows server on AWS
  • Using DCV Viewr to connect remotely to the server (When I connect the cloudXR app to the server the resolution of the DCV changes automatically! I guess it is because it adds a new virtual monitor to the windows)
  • Wifi connection for iPhone (latency 6-34ms, jitter 1ms)
  • The virtual grid that the cloudXR application shows on the floor looks accurate, so I know it is not an issue from ARKit itself.

I am looking for any advice on how to solve this issue. Specifically, I’m wondering:

  1. Are there any known issues with object placement in AR/VR projects using CloudXR with Unreal Engine on iOS?
  2. Could this issue be related to calibration or configuration settings within CloudXR, Unreal Engine, or on the iOS device itself?
  3. Are there specific settings or adjustments recommended for ensuring accurate object placement in AR/VR scenes streamed to iOS devices?

Any insights, suggestions, or guidance from the community would be greatly appreciated. I am eager to resolve this issue and continue developing AR/VR experiences using CloudXR.

Thank you in advance for your help!

1 Like