Retrieve lidar data failed using the code rep.orchestrator.step

Isaac Sim Version

5.0.0

Operating System

Ubuntu 22.04

GPU Information

  • Model: 5090
  • Driver Version: 575

Topic Description

Detailed Description

I can’t retrieve lidar data with rep.orchestrator.step but simulation_app.update. I just edit some code at standalone_examples/api/isaacsim.util.debug_draw/rtx_lidar.py.

# SPDX-FileCopyrightText: Copyright (c) 2022-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import argparse
import sys

parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config", type=str, default="Example_Rotary", help="Name of lidar config.")
args, _ = parser.parse_known_args()

from isaacsim import SimulationApp

# Example for creating a RTX lidar sensor and publishing PCL data
simulation_app = SimulationApp({"headless": False})
import carb
import omni
import omni.kit.viewport.utility
import omni.replicator.core as rep
from isaacsim.core.api import SimulationContext
from isaacsim.core.utils import stage
from isaacsim.core.utils.extensions import enable_extension
from isaacsim.storage.native import get_assets_root_path
from pxr import Gf

# enable ROS bridge extension
enable_extension("isaacsim.util.debug_draw")

simulation_app.update()

# Locate Isaac Sim assets folder to load environment and robot stages
assets_root_path = get_assets_root_path()
if assets_root_path is None:
    carb.log_error("Could not find Isaac Sim assets folder")
    simulation_app.close()
    sys.exit()

simulation_app.update()
# Loading the simple_room environment
stage.add_reference_to_stage(
    assets_root_path + "/Isaac/Environments/Simple_Warehouse/full_warehouse.usd", "/background"
)
simulation_app.update()

lidar_config = args.config

# Create the lidar sensor that generates data into "RtxSensorCpu"
# Possible config options are Example_Rotary and Example_Solid_State
_, sensor = omni.kit.commands.execute(
    "IsaacSensorCreateRtxLidar",
    path="/sensor",
    parent=None,
    config=lidar_config,
    translation=(0, 0, 1.0),
    orientation=Gf.Quatd(1.0, 0.0, 0.0, 0.0),  # Gf.Quatd is w,i,j,k
)
hydra_texture = rep.create.render_product(sensor.GetPath(), [1, 1], name="Isaac")

simulation_context = SimulationContext(physics_dt=1.0 / 60.0, rendering_dt=1.0 / 60.0, stage_units_in_meters=1.0)
simulation_app.update()

# Create the debug draw pipeline in the post process graph
writer = rep.writers.get("RtxLidar" + "DebugDrawPointCloud" + "Buffer")
writer.attach([hydra_texture])

annotator = rep.AnnotatorRegistry.get_annotator("IsaacExtractRTXSensorPointCloudNoAccumulator")
annotator.attach(hydra_texture)

simulation_app.update()

simulation_context.play()

while simulation_app.is_running():
    simulation_app.update()
    # rep.orchestrator.step(rt_subframes=1, delta_time=1.0 / 60.0, pause_timeline=False)
    data = annotator.get_data()
    print(data)

# cleanup and shutdown
simulation_context.stop()
simulation_app.close()

Steps to Reproduce

  1. run the code above

  2. in the while loop. try simulation_app.update and rep.orchestrator.step

  3. simulation_app.update can produce point cloud data but rep.orchestrator.step can not

Closing this as a known issue that has been resolved in Isaac Sim 6.0.0.

The RTX Lidar sensor pipeline did not properly synchronize with rep.orchestrator.step() in versions 4.5.0 through 5.1.0, causing empty point cloud data. This was confirmed as a bug (GitHub #110) and affects multiple users.

If you’re still on a pre-6.0 version, workarounds include using simulation_app.update() directly, or setting omni:sensor:Core:skipDroppingInvalidPoints=True on the lidar prim.

Related threads: