Best practices for always-on-top Measure Tool visualization with CloudXR and Apple Vision Pro

[Mojatto1997]

Note: I’m Japanese and not very fluent in English.
The wording might sound unnatural or inappropriate.
Sorry about that in advance!

Hi,

I am working on an Omniverse application (based on the Isaac Sim template) that streams the viewport to Apple Vision Pro via CloudXR.
My goal is to visualize point‑to‑point measurements created by omni.kit.tool.measure directly in the streamed 3D space on Apple Vision Pro.

What I want to achieve

• Use the Point‑to‑Point tool of omni.kit.tool.measure and show, on Apple Vision Pro:
  • the start and end points of the measurement
  • the line segment representing the measured distance
  • the numeric measurement label

Current approach and problem

• I understand that viewport overlays (for example omni.ui.scene.Line drawn in a SceneView) are rendered only in the desktop viewport UI and are not included in the image that is streamed through CloudXR.
  Because of this, I cannot simply reuse the existing overlay implementation of omni.kit.tool.measure on Apple Vision Pro.
• As a workaround I currently create simple 3D USD objects (lines, points, labels) in the scene to mimic the measurement visualization.
  This works in principle, but those objects are occluded by other geometry.
  When the user moves the viewpoint behind other objects, the measurement line and points are no longer visible.
• Ideally I would like an “always on top” rendering of these measurement objects.
  A similar effect already exists in Omniverse for:

  • selection outlines and gizmos that remain visible even when the object is partially or fully behind other geometry.
    

    

  • some debug / clash‑detection viewports where objects are emphasized independently of normal scene occlusion.​
    However, these effects seem to be implemented in the core rendering pipeline (Carbonite/Kit) and I could not find a public API or example to reuse them from a custom extension.

Questions

1. Is there any recommended way to render measurement‑like visuals (lines, points, labels) in 3D space so that they behave like viewport overlays but are visible in the CloudXR stream (i.e. from the XR camera’s point of view), instead of only in the desktop viewport UI?
   • For example, can omni.ui.scene primitives such as SceneView / Line be driven by a 3D camera (CloudXR / OpenXR camera) and rendered into the same render product that CloudXR streams?
2. For USD scene objects, is there a documented or recommended best practice to render them “always on top” (ignoring depth test or using a special render layer) in the Omniverse RTX renderer?
   • I am aware that I can approximate this with emissive / unlit materials and by offsetting geometry toward the camera, but I would like to know if there is an official way similar to the internal selection‑outline / gizmo passes.

Additional notes

• I only need to support the Point‑to‑Point sub‑tool of omni.kit.tool.measure.
• The important requirement is that the measurement visualization remains visible from any viewpoint on Apple Vision Pro, even when other objects are in front of the measured points.

Environment

• Application: Isaac Sim template–based custom app
• Isaac Sim: 5.1.0
• Omniverse Kit SDK: 107.3.3
• CloudXR Kit: 5.0.1
• Client Library Version: 2.67.0
• Measure Tool extension: omni.kit.tool.measure 200.0.2 ​

Any guidance, examples, or pointers (even low‑level Carbonite/Kit SDK APIs) on how to achieve this “always‑on‑top measurement visualization” in a CloudXR + Apple Vision Pro setup would be greatly appreciated.
Thank you in advance for your help!

[Richard3D]

I am sorry for the delay in answering this question. I apologize. This is quite a complex question, and it will take some time to get answers from the XR engineers. Let me research this and get back to you. Thanks again.

[Richard3D]

Question 1: Rendering measurement visuals (lines, points, labels) visible in the CloudXR stream

Short answer:omni.ui.scene / SceneView overlays are desktop-only UI widgets — they render on top of the viewport compositing pipeline and are NOT part of the render product that CloudXR streams. You can’t directly feed omni.ui.scene primitives into the XR camera’s render product.

XRUsdLayer managed objects (Kit 109+ / CloudXR 6) — RECOMMENDED

The Omniverse Spatial extensions provide XRUsdLayer — a specialized system for managing XR UI elements in the USD stage with minimal latency using USD-RT (bypasses standard USD composition). Since these are actual USD prims, they’re rendered by RTX and included in the CloudXR stream.

For measurement lines:

1. Create an XRUsdLayer for your measurement tool
2. Add USD geometry (BasisCurves for lines, small Spheres for endpoints) as managed assets
3. Use add_reorient() with device_up_right alignment for billboarding labels
4. Call set_transform() / set_position() with use_usd=False for real-time updates

Python
import omni.kit.xr.core
from pxr import Gf

xr_core = omni.kit.xr.core.XRCore.get_singleton()

measure_layer = xr_core.create_xr_usd_layer(
usd_path=“/_xr/measurements”,
meters_per_unit=0.01,
up_axis=‘y’
)

line_path = measure_layer.add_asset(
path=“/_xr/measurements/line_01”,
group=“measurements”,
file_path=“/path/to/measurement_line.usd”,
visible=True
)

measure_layer.set_transform(
path=“/_xr/measurements/line_01”,
transform=your_transform_matrix,
transform_type=omni.kit.xr.core.XRTransformType.stage,
use_usd=False # Fast USD-RT path
)
Docs:Scene Integration — Omniverse Spatial

---

───

Question 2: “Always on top” rendering for USD scene objects in the RTX renderer

Short answer: There is NO public API equivalent to the internal selection-outline/gizmo passes. Those use renderer-level hooks not exposed to extension developers.