Solid State RTX Lidar Config

I have been trying to model a solid state lidar using the RTX Lidar of Isaac-Sim. I read the documentation here.

What I am trying to do:

  • Trying to model a solid state lidar which
    • scans row-by-row starting from below
    • has no time-offset between channels of the same row
    • Let’s say the pointcloud dimensions are (columns,rows)=(100,50) and I scan 100 points at once (bottom row) and these emitters move up a row and scan the next row and this repeats 50 times for one complete scan.
    • Lidar frequency is 10 Hz meaning I should receive 10 complete scans per second.

Anyways, I have a few doubts about the documentation.

  • For my use case, which is recommended?
    • One emitter state with 5000 points in which fireTimeNs list changes its value at every 100th element
    • 50 emitter states with each emitter state representinǵ a row
  • What is the difference between reportRateBaseHz and scanRateBaseHz which would lead me to the next questions
    • What is the difference between firing pattern and full firing cycle in the documentation? (in my case, which is one row and which is one complete scan)
    • I assume scanRatesHz is a list of configurable scanning frequencies. How does it help in the simulation if we have to change the actual frequency values ourselves?
    • What does ‘tick’ in relation to scans mean; is it per row or per complete scan?
    • I understand that the RTX lidar renders frames with a texture which is then post-processed to generate the pointcloud - which of the above frequencies correspond to this frequency of rendering frames?
  • What is the purpose of waveLengthNm, minReflectance, minReflectanceRange because I wrote a simple MDL file with spectral properties (defining reflectance values for a set of wavelengths) and I see no effect. For eg., I set the waveLengthNm in the config file to 885 nm and created a material which reflects only light of wavelength=520nm and another material which reflects only light of wavelength=885nm and I saw no difference in the pointcloud (I expected that the rays won’t reflect and hence no points of intersection for the first material)
  • What is intensityMappingType which I don’t see in the documentation but the console says that it is missing.
  • I also read from one of the forum posts that there are solid-state specific parameters like numLines and numRaysPerLine which are not yet included in the documentation; these parameters are more intuitive but how do we use this to write emitterStates

I believe the Example_Solid_State.json file is similar to what you need, but it’s rows have 1000 columns instead of 100.

  • For my use case, which is recommended?

    “numLines”: 50,
    “numRaysPerLine”: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100,
    100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100,
    100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100,
    100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100,
    100, 100, 100, 100, 100, 100],

And unless the parameters of the emitters changes, you only need one state.

  • What is the difference between reportRateBaseHz and scanRateBaseHz which would lead me to the next questions

I’m not sure there is one for the solid state lidar. I’ll pass that question up the chain, but I have only seen these be equal in solid state lidar.

  • What is the difference between firing pattern and full firing cycle in the documentation? (in my case, which is one row and which is one complete scan).

Nothing, your firing pattern has 5000 firings for a full scan. It fires at 10Hz, so you fire 10 patterns a second.

  • I assume scanRatesHz is a list of configurable scanning frequencies. How does it help in the simulation if we have to change the actual frequency values ourselves?

scanRatesHz is not used.

  • What does ‘tick’ in relation to scans mean; is it per row or per complete scan?

per complete scan. Your solid state lidar has 5000 emitters. tic is a bad word, should probably be Hz

  • I understand that the RTX lidar renders frames with a texture which is then post-processed to generate the pointcloud - which of the above frequencies correspond to this frequency of rendering frames?

None. The rendering is done at the FPS of the application. So if your running at 30fps, then you will get about 1/3 of your emitters firing per frame.

  • What is the purpose of waveLengthNm, minReflectance, minReflectanceRange because I wrote a simple MDL file with spectral properties (defining reflectance values for a set of wavelengths) and I see no effect. For eg., I set the waveLengthNm in the config file to 885 nm and created a material which reflects only light of wavelength=520nm and another material which reflects only light of wavelength=885nm and I saw no difference in the pointcloud (I expected that the rays won’t reflect and hence no points of intersection for the first material)

I’m not sure waveLengthNm is used outside atmospheric interference, which is not in IsaacSim yet.
minReflectance is the min return power at a range that is considered valid. below that, the return may be invalid. The minReflectanceRange is the range to consider that min value at.

  • What is intensityMappingType which I don’t see in the documentation but the console says that it is missing.

You should not be getting an error, your intensityMappingType should default to “LINEAR”, like in the example json files.

  • I also read from one of the forum posts that there are solid-state specific parameters like numLines and numRaysPerLine which are not yet included in the documentation; these parameters are more intuitive but how do we use this to write emitterStates

Huh, those got left out by mistake, I’ll update the docs on my end. They will read:

numLines: Number of rows per full scan.

numRaysPerLine: Number of emitters in a row.

They are not in emitter states… but the number or array entries in the emitter state parameters is numLines*numRaysPerLine

Solid state config files tend to be large compared to rotary config files because of this fact.

1 Like

Regarding the scan rates for solid state lidar, a more knowledgeable person than I responded:

“The reportRateBaseHz and scanRateBaseHz are the same for solid state configurations. It makes a difference when you have a different lidar system (like a rotary lidar) where you have a tick that encompasses a sub scan of the full scan (Think of it as a sub field of view that will encompass the full field of regard). The scanRate is the rate at which a full PC is completed (which is the same for solid state or rotary). The reported rate is the rate per tick. Since solid states are basically one tick, the reportBaseRate and the ScanRate are the same.
Have you taken a look at the Example_Solid_State.json configuration? Based upon what the nature of the question at hand, the segmentation in angle and time can be fully defined. It would require some modifications to the file to support the specific scan timing. This is a good place to start to grasp how a simple solid state lidar can be configured. It should provide the full context on a lot of the following questions about how the solid state lidar is configured
Note the wavelength, minReflectance, and minReflectanceRange are metrics that help define the capability of the lidar in regards to intensity thresholds.
The intensity mapping is yet another way to scale the output. A real sensor can scale and quantize intensities and this is a feature where one can use different mapping types to achieve a different scaled output. A linear mapping type is probably what is desired here”

1 Like

Hey @mcarlson1!
First of all, many thanks to your reply! It has been really resourceful. Also, sorry for my late response, I was experimenting with the RTX Lidar so that I can get back to you with more understanding.

I might have a few more doubts.

  • scanRatesHz: It appears that I cannot skip the scanRatesHz because it raises an error and the app crashes.

    [Warning] [lidar.lidarprofilereader] Member scanRatesHz not found or not an array!
    [Error] [lidar.lidarprofilereader] Not a valid primary profile
    [Warning] [omni.drivesim.sensors.nv.lidar.lidar_core.plugin] No valid ray type in the lidar profile.
    

    Anyways, I created a list with the frequency value in it.

  • Extension Scripting: I tried to use the script in the documentation in one of my extensions but the render product was not being attached. I created a seperate post here. I tentatively solved it, but would be helpful if you could explain why it wouldn’t work the way I tried.

  • Pointcloud Generation Rate: I now attach the ROS1PublishPointCloud OGN to the post-process SDG Pipeline. I was just curious to know how I can modify anything such that the pointcloud will be published only after a full scan (in my case 5000 points in a ROS msg published at 10Hz)

  • AOVs and MDL: What should I add in the SDG Pipeline to access the AOVs like the surface normal and the MDL info at the point of intersection apart from the x,y,z co-ordinates

  • Intensity Mapping Type: What are the possibilities other than “LINEAR”?

PS: I am trying to model a LiDAR with high fidelity and I am exploring how I can leverage the power of RTX LiDAR. This is why I want to dig deep into what all the config parameters mean and how we can use them.

Thanks in advance for your reply!

Regarding the wavelength, minReflectance, and minReflectanceRange, I understand that the properties are rather used for calculation and not direct modelling in the physics scene. I have a question there. Reflectance is basically a property of materials. Here, how do we modify the reflectance of a material in the scene. Like, how does the Lidar check the minReflectance in the scene?

These may help.
https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/isaac_sim_sensors_rtx_based_lidar/solid_state_config.html

RE: scanRateHz
scanRateHz is not used. It’s a relic but since the loading code for the config has not changed you still need it to exist. As long as it is formatted correctly it will be fine and ignored. Try

"scanRatesHz": [5.0, 10.0, 15.0, 20.0],

RE: extension scripting…
looking at the link next.

RE: point cloud generation rate
It is a much requested feature that we return only a full 360 scan (or full pattern firing from solid state). This is not a feature yet. But we are aware of the need.
I believe the correct way for you to do it now would be to make your own plugin with python nodes in it that took the output of the compute rtx lidar point cloud node and/or the read rtx lidar data node and computes a full scan based on some time or angle parameters from them.
There may be a way to set up a network in omnigraph without writing your own node, but I don’t know enough about the node magic to know for sure.

RE: AOVs and MDL
The material ID that is output by the read rtx lidar data node is the sensor material, and until we document how to add those to a scene, they will all probably be 31… so no use…
The hitPointNormal will be available in the next patch for isaac sim.

RE: Intensity Mapping Type:
the intensity mapping it a post render scalling of the intensity value from 0-1 to some other value.
the types that work are LINEAR and NONLINEAR_BIG_STEP.
all intensity mapping types use the “intensityScalePercent” value, which defaults to 255%.
If you set
“intensityScalePercent”: 100,
in the config, you will get a 1-1 map of the 0-1 intensity values in the ReadRTXLidarData node and the mapped intensity from the ComputeRTXPointCloud node. If you leave it to the the default 255%, then the mapped intensity will be in the range 0-2.55.
With LINEAR intensityMappingType these are a straight line.
The other mapping that works is nonlinear_big_step… so lets say you set

“intensityScalePercent”: 250,
“intensityMappingType”: “NONLINEAR_BIG_STEP”,
“intensityMapping”: [0, 0, 0.2, 0.6, 0.6, 0.9, 1.0, 1.0]

Then you would get a mapping like this… This will hopefully be ducumented in our next release… but it’s a post process, so it does not change how the rtsensor render works… just maps the intensity value after, like transforming the points to world space in the Compute Point Cloud node does.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.