VR Rendering Issue with USDZ-based Gaussian Splatting in Isaac Sim 5.0

Gaming and Visualization Technologies XR – VR/AR/MR XR – VR/AR/MR
1

Hi,
I’m currently working with Isaac Sim version 5.0 on Ubuntu 22.04, using an Oculus Quest 2 connected via WiVRn (Monado). My system is running NVIDIA Driver 580.95.05.

Following the instructions described in the GitHub - nv-tlabs/3dgrut: Ray tracing and hybrid rasterization of Gaussian particles , I successfully converted a .ply file containing Gaussian Splatting data into a USDZ format using the ply_to_usd.py script. The converted scene loads and displays correctly in Isaac Sim in desktop mode.

However, when I connect to the same scene using VR (via OpenXR and WiVRn), the Gaussian Splatting content does not render properly. The rest of the scene appears fine, but the 3D Gaussian points either disappear, flicker, or fail to render at all in the VR viewport.

Environment:

  • Isaac Sim: 5.0

  • VR Headset: Oculus Quest 2

  • Connection: WiVRn (OpenXR via Monado)

  • OS: Ubuntu 22.04

  • NVIDIA Driver: 580.95.05

Steps Taken:

  • Converted .ply to USDZ using ply_to_usd.py

  • Verified correct rendering in desktop mode

  • Enabled all relevant VR extensions (omni.kit.xr.system.openxr, omni.kit.xr.profile.vr, etc.)

  • Connected via WiVRn and confirmed controller input and scene navigation work

  • Gaussian Splatting content fails to render in VR mode

Question:
Is there a known limitation or workaround for rendering USDZ-based Gaussian Splatting in VR mode within Isaac Sim 5.0?
Are there specific settings, extensions, or rendering flags required to make this work?
Any guidance or suggestions would be greatly appreciated.

Thanks!

2

Hi!

I’ve been trying to display an USDZ-based Gaussian splatting just like you in a VR Headset and I got the same problem of failing to render at all in VR Viewport.
Have you found some stuff, fix or answer to this problem.
I tried playing with RTX, rendering quality, foveation settings but nothing works.

Thanks for any help!

3

Hi, have you solved this problem?

4

Hi @KentoS , were you able to see the gaussian in AR mode?