Deploying RL Stair-Climbing Policy from Isaac Lab to Isaac Sim Without Raycast Support

[xorhks2566]

Note: For any Isaac Lab topics, please submit your topic to its GitHub repo (GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim) following the instructions provided on Isaac Lab’s Contributing Guidelines (Contribution Guidelines — Isaac Lab Documentation).

Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.

Isaac Sim Version

4.5.0

Operating System

Ubuntu 22.04

GPU Information

• Model: nvidia 4070Ti
• Driver Version: cuda 12

Topic Description

Detailed Description

I trained a quadruped robot for stair-climbing locomotion using Isaac Lab with reinforcement learning. The policy utilizes raycast-based terrain scanning for perception, and it works well within Isaac Lab.

However, when I try to deploy the trained policy inside Isaac Sim 4.5.0, it fails to climb stairs. After checking, it seems Isaac Sim does not support GPU-based raycast sensors (used in Isaac Lab) out of the box. As a result, the policy is unable to perceive the terrain elevation properly, and stair climbing fails.

I would like to ask the community:
Has anyone successfully deployed a reinforcement learning policy for stair climbing in Isaac Sim?
If so, how did you replicate or approximate the raycast terrain perception used in Isaac Lab?

Any guidance or best practices would be greatly appreciated. Thank you!

Steps to Reproduce

1. Train a quadruped policy in Isaac Lab using raycast-based height sensing.
2. Export the trained policy.
3. Run it in Isaac Sim without raycast sensor support.
4. Observe the robot failing to climb stairs.

Additional Information

What I’ve Tried

• Verified that the policy works in Isaac Lab with raycast enabled.
• Attempted deployment in Isaac Sim using the same trained policy, without raycast, resulting in failure.

Additional Context

I’m exploring possible workarounds such as synthetic heightmaps, point clouds, or preprocessed terrain data, but ideally would like to match the training setup closely. Any help from someone who tackled this would be very valuable.

[phennings]

Thank you for your interest in Isaac Lab. To ensure efficient support and collaboration, please submit your topic to its GitHub repo following the instructions provided on Isaac Lab’s Contributing Guidelines regarding discussions, submitting issues, feature requests, and contributing to the project.

We appreciate your understanding and look forward to assisting you.