Isaac sim 6.0 What are the functions of the RTX Sensor?(Non-Visual Material)

Omniverse Isaac Sim
, ,
1

Hello everyone. I’ve long had questions regarding Non-Visual Materials. I’ve conducted numerous tests ever since older software versions and hope to get an accurate answer.

I ran tests on Non-Visual Materials using the RTX Sensor, yet the results differ from what I expected.

I assumed that Non-Visual Materials would introduce noise to the point cloud generated by the RTX Sensor. For instance, the point cloud would behave differently when the sensor hits highly reflective surfaces, mirrors or glass — glass, in particular, would allow the sensor signal to pass through. I clearly recall that this feature worked in version 4.5. However, after testing version 6.0, it seems this functionality no longer exists. Regardless of which Non-Visual Material I use, the point cloud always appears on the surface of objects.

I view the point cloud in RViz. Currently, no matter how I adjust the Non-Visual Materials, points are always generated on the object surfaces. When it comes to point cloud generation, the RTX Sensor seems to offer no advantages over the PhysX LiDAR.

10
101186×713 79.3 KB

11
11746×319 25.7 KB

12
12742×509 60.7 KB

9
9505×488 58.3 KB

3

Hi @Julie_A From the screenshot, I do not see an obviously issues related to non-visual material. One thing to confirm is that those material prims are actually bound to the sphere/panel mesh prims, since the screenshot only shows the material prim attributes.

There are two things for Isaac Sim 6.0:

  1. After changing non-visual material attributes, please save and reload the stage before checking the result. These material attributes may not be reflected live in the RTX sensor material database.
  2. RViz XYZ point positions alone are not a reliable way to tell whether non-visual materials are being used. For opaque static geometry, the RTX LiDAR can still return points on the object surface. The material information is exposed through RTX sensor auxiliary data such as material ID and intensity.

Can you try the following checks?
In the Isaac Sim viewport, switch to RTX - Real-Time 2.0 > Debug View > Non-Visual Material ID. The sphere and panel should show different material-ID colors if the material binding and encoding are working. Please refer to the document RTX Sensor Non-Visual Materials — Isaac Sim Documentation

The current Isaac Sim 6.0 example to compare against is:
source/standalone_examples/api/isaacsim.sensors.experimental.rtx/apply_nonvisual_materials.py

That example applies NonVisualMaterial to objects, creates an RTX LiDAR with full auxiliary output, and inspects the returned intensity data.

If your setup still shows no difference in matId or intensity after save/reload, please share a minimal USD or Python script so we can check whether this is a material binding issue or an RTX sensor regression.

4
  1. I have confirmed that I am using the non-visual material.
  2. I checked the Debug View, and everything works fine. You can refer to my screenshots.
  3. I can also obtain normal values for both matId and intensity.

I exported and inspected the point cloud of a single frame, and the results are consistent with what is displayed in RViz. The XYZ coordinates of all point cloud points lie on the surface of the objects, showing no difference from data captured by conventional sensors.

What I want to know is: can the LiDAR generate point cloud noise based on different materials?

For example, producing noise caused by laser penetration through glass or reflections off mirror surfaces. Currently the point cloud data is overly clean and lacks such noise. Besides, the sensor I used is configured with the Example_Rotary template.

screenshot-20260630-112853
screenshot-20260630-1128531117×899 57.5 KB

5

@zhengwang It seems that it still isn’t generating any point cloud noise.

8

This seems to be a bug. I have created internal ticket for the team to investigate. Thanks for bringing it up!

Internal ticket: 6426827

9

Hi @Julie_A ! The internal team is requesting more clarifications on this issue:

  1. Are you using the same scene consistently between Isaac Sim 4.5/5.0/5.1/6.0?
  2. It seems like materials are applied correctly based on the debug view. Please try debug view only colors prims based on the base material (not the coating or attributes) - this is a documentation gap we’re addressing.
  3. The Example_Rotary config has omni:sensor:Core:maxReturns set to 2. Could you please try increasing return count to 3, and modifying one of the materials to be (eg) glass, water, or something else visually transparent?