RTX Lidar principle

Omniverse Isaac Sim
1

Does anyone know the detailed working principle of RTX Lidar for generating point cloud?

2

Hello,

RTX lidar simulates the light rays of a lidar sensor, then can return them as a point cloud either to visualize or be recorded for synthetic data.

Lets look at the code first.

import omni.kit.commands
from pxr import Gf
import omni.replicator.core as rep
lidar_config = "Example_Rotary"

# 1. Create The Camera
_, sensor = omni.kit.commands.execute(
    "IsaacSensorCreateRtxLidar",
    path="/sensor",
    parent=None,
    config=lidar_config,
    translation=(0, 0, 1.0),
    orientation=Gf.Quatd(1,0,0,0),
)
# 2. Create and Attach a render product to the camera
render_product = rep.create.render_product(sensor.GetPath(), [1, 1])

# 3. Create Annotator to read the data from with annotator.get_data()
annotator = rep.AnnotatorRegistry.get_annotator("RtxSensorCpuIsaacCreateRTXLidarScanBuffer")
annotator.attach(render_product)

# 4. Create a Replicator Writer that "writes" points into the scene for debug viewing
writer = rep.writers.get("RtxLidarDebugDrawPointCloudBuffer")
writer.attach(render_product)

You can run this in script editor and it will create a running lidar when you hit play.

This will also make a action graph holding the setup of the pipeline

to open the graph
Window > Visual Scripting > Action graph

image

Select - Edit Action Graph

Select - Render/postprocess/SDGPipeline
image

SDG Omnigraph

image
image2122×732 113 KB

If you want to change the color and size of the displayed points you can adjust that in the last node in the attributes panel

image
image1565×617 60.9 KB

Back to the code

The line

lidar_config = “Example_Rotary”

can be changed to the sensor you want from the list below, which are the parameters of real world sensors.
These can be found here /AppData/local/ov/pkg/isaac-sim-4.0.0/lidar_configs
DONT MODIFY THESE FILES

image
image1128×1011 90.7 KB

To get the pointcloud data You’ll need to add somewhere in your script, create new tab in script editor like before run the code, you’ll see a print out in the console of an array

data = annotator.get_data()
points = data["data"]
print (points)

image
image795×308 23.5 KB

thats your actual pointcloud array

For an in-depth look at all the functions and nodes check out

https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/isaac_sim_sensors_rtx_based_lidar.html

hope this helps

3

Thanks for your patient answers. I have successfully implemented how to generate point clouds using RTX Lidar in the Isaac Sim. What I may want to know is whether RTX Lidar generates point clouds through real-time ray tracing technology, and what is its principle?

4

Yes it’s based on ray tracing, so it’s simulating light rays the same way that light is produced for RTX rendering in modern games