VR Rendering Issue with USDZ-based Gaussian Splatting in Isaac Sim 5.0

Gaming and Visualization Technologies XR – VR/AR/MR XR – VR/AR/MR
1

Hi,
I’m currently working with Isaac Sim version 5.0 on Ubuntu 22.04, using an Oculus Quest 2 connected via WiVRn (Monado). My system is running NVIDIA Driver 580.95.05.

Following the instructions described in the GitHub - nv-tlabs/3dgrut: Ray tracing and hybrid rasterization of Gaussian particles , I successfully converted a .ply file containing Gaussian Splatting data into a USDZ format using the ply_to_usd.py script. The converted scene loads and displays correctly in Isaac Sim in desktop mode.

However, when I connect to the same scene using VR (via OpenXR and WiVRn), the Gaussian Splatting content does not render properly. The rest of the scene appears fine, but the 3D Gaussian points either disappear, flicker, or fail to render at all in the VR viewport.

Environment:

  • Isaac Sim: 5.0

  • VR Headset: Oculus Quest 2

  • Connection: WiVRn (OpenXR via Monado)

  • OS: Ubuntu 22.04

  • NVIDIA Driver: 580.95.05

Steps Taken:

  • Converted .ply to USDZ using ply_to_usd.py

  • Verified correct rendering in desktop mode

  • Enabled all relevant VR extensions (omni.kit.xr.system.openxr, omni.kit.xr.profile.vr, etc.)

  • Connected via WiVRn and confirmed controller input and scene navigation work

  • Gaussian Splatting content fails to render in VR mode

Question:
Is there a known limitation or workaround for rendering USDZ-based Gaussian Splatting in VR mode within Isaac Sim 5.0?
Are there specific settings, extensions, or rendering flags required to make this work?
Any guidance or suggestions would be greatly appreciated.

Thanks!

2

Hi!

I’ve been trying to display an USDZ-based Gaussian splatting just like you in a VR Headset and I got the same problem of failing to render at all in VR Viewport.
Have you found some stuff, fix or answer to this problem.
I tried playing with RTX, rendering quality, foveation settings but nothing works.

Thanks for any help!

3

Hi, have you solved this problem?

4

Hi @KentoS , were you able to see the gaussian in AR mode?

5

Hi!

I’m not able to the gaussian in VR.

Isaac Sim version is 5.0.

I expect 6.0 can work.

6

Yes, this is a known limitation in NVIDIA Isaac Sim 5.0. Gaussian splatting (e.g., via 3dgrut) depends on custom shaders and ray-tracing paths that don’t fully translate to VR through OpenXR, especially with Monado.

Workarounds include disabling ray tracing, forcing rasterized rendering, lowering LOD, or testing with NVIDIA’s native OpenXR runtime. Full VR support for Gaussian splats remains experimental and may require custom shader adaptation.