Issue with Isaac 5.10 "Non-Visual Materials" Feature

Isaac Sim Version

5.1.0

Operating System

Ubuntu 22.04

I’m encountering some issues while using the “RTX Sensor.” I would like to obtain some point cloud noise data.
First, I created an “RTX sensor.”

sensor = LidarRtx(
    prim_path="/World/lidar",
    translation=np.array([0.0, 0.0, 0.0]),
    orientation=np.array([1.0, 0.0, 0.0, 0.0]),
    config_file_name="Example_Rotary",
)

I configured my materials according to the documentation and added properties. The final result is shown in the screenshot below. Please let me know if there are any errors.And I applied the material to the cube.

Then I captured its point cloud data, but it doesn’t seem to be working as expected. No noise is being generated. I’m wondering if this is due to incomplete functionality in Isaac 5.10, or if there’s an error in my operation?


Yeah I have the same question. I am trying to somehow get the intensity values from Isaac onto ROS2 to see if that helps.

@adiljus2
I managed to create some noise, although it doesn’t seem to be very good.
Here are some modifications in my code that you can try.

from isaacsim import SimulationApp
import numpy as np
import os
from datetime import datetime

config = {
    "headless": False,
    "app_setting": {
        "/app/sensors/nv/lidar/outputBufferOnGPU": True,
        "/app/sensors/nv/radar/outputBufferOnGPU": True,
        "/app/sensors/nv/lidar/auxOutputType": "FULL",
        "/app/sensors/nv/lidar/skipDroppingInvalidPoints": True,
        "/app/sensors/nv/lidar/outputFrameOfReference": "GLOBAL",
        "/app/sensors/nv/lidar/elementsCoordsType": "CARTESIAN"
    }
}
kit = SimulationApp(config)
sensor = LidarRtx(
    prim_path="/World/lidar",
    translation=np.array([0.0, 0.0, 0.0]),
    orientation=np.array([1.0, 0.0, 0.0, 0.0]),
    config_file_name="Example_Solid_State",
)


However, I’ve encountered a new issue regarding creating an “RTX Sensor”. I haven’t been able to create a custom “RTX Sensor”, and can only create the “Sensor” from the example. I’ve faced many problems during the customization process.

I save the point cloud every 5 frames, but when I check it, there are some missing parts in the red-line section. I’m not sure why this is happening.

import omni
from pxr import Gf

# Specify attributes to apply to the ``OmniLidar`` prim.
sensor_attributes = {'omni:sensor:Core:scanRateBaseHz': 20}

_, sensor = omni.kit.commands.execute(
    "IsaacSensorCreateRtxLidar",
    translation=Gf.Vec3d(0, 0, 0),
    orientation=Gf.Quatd(1, 0, 0, 0,),
    path="/lidar",
    parent=None,
    config="None",
    visiblity=False,
    variant=None,
    force_camera_prim=False,
    **sensor_attributes,
)

When I create a custom sensor, it appears to be unusable. I’m not sure which parameters in “sensor_attributes” are mandatory.

Hi, I am trying to do something similar and would like to configure materials that work with the RTX Lidar sensor. Could you guide me on how you configured the material seen in the image containing properties panel of the material? I tried doing something similar but I dont get the Extra Properties tab, instead i tried configuring the non visual materials as documented here. However, that doesn’t seem to work.

@poorvi
Regarding the “Extra Properties tab,” simply entering “omini:simready:nonvisual:attributes” can still create properties, but this usage is considered an older method and has now been updated to the approach described in the documentation. Both methods remain functional.

When I use Isaac 5.0, it occasionally works, but most of the time it does not perform reliably.
When I use Isaac 5.10, it has never succeeded even once.

Hello @Julie_A

I did what you suggested and now I am able to see the material as non visual when I view in the debug mode. So, its confirmed that it is categorizing material as non visual. However, even if I choose aluminium as the material with retroreflective attribute, I am not able to see any noise as what is seen in your image. I am also analysing the intensity values and realized that no matter what the non visual material is, everytime the intensity distribution in the area is almost similar and around 1. something value, which doesn’t sound realistic. I considered contrasting materials like aluminium, wood, etc. but the values still remain the same.

My major point of interest would be intensity and noise in the virtually generated pointcloud. Any help related to this would be helpful.

If you don’t mind, could you share the script/usd file along with the lidar config file so that I can debug it on my end?

@poorvi

Sorry for replying a bit late. This happened because when I connected to ROS in Isaac 5.10 version, I encountered the same situation as you, where point cloud noise was not being generated. My screenshot is from Isaac Sim 5.0 running in Standalone mode. Could you please try both versions 5.0 and 5.10 separately? You will also need to set some configurations. Although some config options are deprecated, I’m unsure if the lack of noise generation might be due to these config settings. I think it’s worth trying.

from isaacsim import SimulationApp

config = {
    "headless": False,
    "app_setting": {
        "/app/sensors/nv/lidar/outputBufferOnGPU": True,
        "/app/sensors/nv/radar/outputBufferOnGPU": True,
        "/app/sensors/nv/lidar/auxOutputType": "FULL",
        "/app/sensors/nv/lidar/skipDroppingInvalidPoints": True,
        "/app/sensors/nv/lidar/outputFrameOfReference": "GLOBAL",
        "/app/sensors/nv/lidar/elementsCoordsType": "CARTESIAN"
    }
}
kit = SimulationApp(config)

An internal ticket to add a minimal USD example for RTX LiDAR with non-visual materials has been created, which should serve as a helpful reference once available.

Following up — a complete end-to-end example for RTX LiDAR with non-visual materials (applying materials, creating a sensor, and reading intensity via the GenericModelOutput annotator) is available in Isaac Sim 6.0:

standalone_examples/api/isaacsim.sensors.rtx/apply_nonvisual_materials.py

Please try with the latest 6.0 release. If you still encounter issues, open a GitHub issue at Issues · isaac-sim/IsaacSim · GitHub with a minimal repro script.