Isaac Sim RTX Lidar, no objectId

[ucabjej]

Hello,

I have come across an issue when trying to get objectIds from the RTX Lidar.

When using IsaacSim 2023.1.1 the code runs, but does not run on 2022.2.1 (another issue entirely).

This code is all run from the script editor and there is an object (cone) placed in the scene (has debug lidar points on it). The spawn code is run before play. The data retrieval is run during simulation playing.

When running this code in the script editor, I see the debug Lidar points fine, however, when I go to try and get the objectIds of all hit objects the array is empty. This code comes from the doc page RTX_LIDAR.

import omni.kit.commands
from pxr import Gf
import omni.replicator.core as rep
lidar_config = “Example_Rotary”

1. Create The Camera

_, sensor = omni.kit.commands.execute(
“IsaacSensorCreateRtxLidar”,
path=“/sensor”,
parent=None,
config=lidar_config,
translation=(0, 0, 1.0),
orientation=Gf.Quatd(1,0,0,0),
)

2. Create and Attach a render product to the camera

render_product = rep.create.render_product(sensor.GetPath(), [1, 1])

3. Create a Replicator Writer that “writes” points into the scene for debug viewing

writer = rep.writers.get(“RtxLidarDebugDrawPointCloudBuffer”)
writer.attach(render_product)

4. Create Annotator to read the data from with annotator.get_data()

annotator = rep.AnnotatorRegistry.get_annotator(“RtxSensorCpuIsaacCreateRTXLidarScanBuffer”)
annotator.initialize(outputObjectId=True)
annotator.attach(render_product)

The code that is used to try and get the objectIds is below:

output_lidar_data = annotator.get_data()
print(output_lidar_data)

Below is the output of this code:
{‘azimuth’: array(, dtype=float64), ‘beamId’: array(, dtype=float64), ‘data’: array([[ 9.708219 , -0.5087862 , -1.7421757 ],
[ 9.70936 , -0.50884604, -1.6008642 ],
[ 9.701449 , -0.5084315 , -1.4605494 ],
…,
[ 9.703027 , -0.49263918, -1.0434356 ],
[ 9.710629 , -0.49302518, -0.9054116 ],
[ 9.708224 , -0.49290308, -0.7684523 ]], dtype=float32), ‘distance’: array([9.876413, 9.853597, 9.823942, …, 9.771396, 9.765202, 9.751056],
dtype=float32), ‘elevation’: array(, dtype=float64), ‘emitterId’: array(, dtype=float64), ‘index’: array([ 204, 206, 208, …, 921558, 921560, 921562], dtype=uint32), ‘intensity’: array([0.03194074, 0.03438428, 0.03642835, …, 0.03662872, 0.03836866,
0.03758959], dtype=float32), ‘materialId’: array(, dtype=float64), ‘normal’: array(, dtype=float64), ‘objectId’: array(, dtype=float64), ‘timestamp’: array(, dtype=float64), ‘velocity’: array(, dtype=float64), ‘info’: {‘numChannels’: 128, ‘numEchos’: 2, ‘numReturnsPerScan’: 921600, ‘renderProductPath’: ‘/Render/RenderProduct_Replicator’, ‘ticksPerScan’: 3600, ‘transform’: array([ 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., -0.,
-0., 1., 1.])}}

Does anyone know why this is not working?

Best,
Jon

[mcarlson1]

ObjectID is not output by default. Set the outputObjectId to true and it will fill.

annotator.initialize(outputObjectId=True)

https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/isaac_sim_sensors_rtx_based_lidar/node_descriptions.html#create-rtx-lidar-scan-buffer

[mcarlson1]

OK, I see what is happening here… sorry to answer your question by stating what you already know :)

It looks like, because the annotator and the writer use the same Node (IsaacCreateRTXLidarScanBuffer) for output, that the annotator initialization is not able to set the flag when it should.

SOLUTION:
Declare the annotator before the writer and things should work out as you expect… or set the outputObjectId directly on the node in the action graph.

[ucabjej]

Thanks for getting back to me.

I have tried your solution and it works, thank you for this.

I also have another question related to the RTX Lidar data.

The sensors can be loaded from config files e.g. “OS0_128ch20hz1024res”.

However, these sensors have pointfields in their pointcloud2 messages such as t, and ring.

Aside from the timestamp (which does not seem to exactly match t, as well as intensity which is not in the same range), I can not seem to find the information on the ring that each point belongs to. I assume this would have been calculated in the node anyway but wanted to ask if there is a way to get this.

[mcarlson1]

In IsaacSim We don’t encode the output intensities to match lidar output, you would have to take the 0-1 values and scale it yourself.