Unable to start AR in isaac lab by using handtracking device(oculus quest pro)

Omniverse Isaac Sim
,
1

Note: For any Isaac Lab topics, please submit your topic to its GitHub repo (GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim) following the instructions provided on Isaac Lab’s Contributing Guidelines (Contribution Guidelines — Isaac Lab Documentation).

Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.

Isaac Sim Version

[√] 4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
[√] Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: 3070Ti
  • Driver Version: 572.83

Topic Description

Detailed Description

Hi:
When executing the command:
./isaaclab.bat -p scripts/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --teleop_device handtracking

to control the Franka arm via Oculus Quest Pro in AR mode (the START VR mode works normally), clicking the ‌START AR‌ button in the Isaac Sim window triggers a console error. The error log suggests compatibility or configuration issues with the AR module, and the log is attached below.

Steps to Reproduce

  1. run ./isaaclab.bat -p scripts/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --teleop_device handtracking
  2. click START AR in isaacsim window
  3. throw an error
    (Add more steps as needed)

Error Messages

2025-04-15 02:56:24 [23,972ms] [Error] [omni.kit.xr.system.openxr.plugin] Failed to find system with HMD form factor/ View configuration
2025-04-15 02:56:24 [24,098ms] [Error] [omni.kit.xr.ui.window.viewport.viewport_layer.viewport_layer] Failed to activate OpenXR: Failed to find system with HMD form factor/ View configuration

Screenshots or Videos

(If applicable, add screenshots or links to videos that demonstrate the issue)

Additional Information

What I’ve Tried

(Describe any troubleshooting steps you’ve already taken)

Related Issues

(If you’re aware of any related issues or forum posts, please link them here)

Additional Context

kit_20250414_000001.log (1.1 MB)

2

I observed that the VR initialization successfully loaded the ‘carb.graphics-direct3d.plugin’ (as shown in the image below), whereas the AR option failed to load this component.

image
image1300×930 141 KB

4

Thank you for your interest in Isaac Lab. To ensure efficient support and collaboration, please submit your topic to its GitHub repo following the instructions provided on Isaac Lab’s Contributing Guidelines regarding discussions, submitting issues, feature requests, and contributing to the project.

We appreciate your understanding and look forward to assisting you.

5

Thank you for the reminder. It has been uploaded to the issue part.

1 Like