RTX LiDAR Helper publishes incomplete point clouds — missing points in certain angular regions

[Kryptos_sim]

Isaac Sim Version

5.1.0
5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 24.04
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

• Model: RTX 5090
• Driver Version: 580.95.05

Topic Description

Detailed Description

When using the RTX LiDAR Helper in Isaac Sim 5.1.0, the published point cloud occasionally lacks data points in specific angular sectors.

Steps to Reproduce

1. Make a ground plane

2. Make a RTX LiDAR (I tested Ouster OS1, VLS128)

3. Make Simple Action Graph (play tick → render product → RTX LiDAR Helper)

4. check “publish one scan”

5. Play

(Add more steps as needed)

Error Messages

(If applicable, copy and paste any error messages you received)

Screenshots or Videos

(If applicable, add screenshots or links to videos that demonstrate the issue)

Additional Information

What I’ve Tried

(Describe any troubleshooting steps you’ve already taken)

Related Issues

(If you’re aware of any related issues or forum posts, please link them here)

Additional Context

(Add any other context about the problem here)

I haven’t tested it thoroughly, but even when ‘publish one scan’ is unchecked, it seems that some points in the pointcloud are missing at certain angles when viewed in Debug View.

[zhengwang]

Hi @Kryptos_sim I am not able to reproduce this issue with Isaac Sim 5.1.0.

What is “publish one scan”? Do you mean “publish full scan” in the ROS2 RTX Lidar Helper node?

Have you tried to create the action graph with tools → Robotics → ROS2 OmniGraphs → RTX Lidar?

[Frigi]

Since I’m facing the same issue of incomplete scans, I attach my situation to this thread:

Isaac Sim Version

5.1.0
5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 24.04
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

• Model: RTX 2000 Ada

• Driver Version: 580.95.05

Topic Description

Detailed Description

When using the RTX LiDAR Helper in Isaac Sim 5.1.0, the published point cloud occasionally lacks points in specific angular sectors.

The sectors, usually span a 6th of the full rotation (visual inspection).

I set up with the reproduction steps below.

I additionally set up rviz to show the published scans and made an observation. When zooming the viewport to the ground plane, the scans have much fewer failures, meaning the occurancy of missing sections goes almost to 0. On the other hand, when zooming far out, or moving the viewport through the ground the scans become faulty.

In another project, I’m running the RTX Lidar together with a particle simulation, which slows down the simulation to around 4 Hz, but consistently produced complete scans (I recorded the data as a ros2 bag and inspected the replay of the bag)

Steps to Reproduce

1. Make a ground plane

2. Make a RTX LiDAR (I tested Ouster OS0 (all available Variants))

3. Create the action graph with Tools → Robotics → ROS2 OmniGraphs → RTX Lidar (check the “Point Cloud” option)

4. check “Publish Full Scan” in the PointCloudPublish node

5. Play

Error Messages

(If applicable, copy and paste any error messages you received)

Screenshots or Videos

Additional Information

What I’ve Tried

(Describe any troubleshooting steps you’ve already taken)

Related Issues

(If you’re aware of any related issues or forum posts, please link them here)

Additional Context

I agree with the first description, that it seems that some scans are missing when unchecking the “Publish Full Scan” option and viewing it in Debug View.

[zhengwang]

@Frigi Did you try the Isaac Sim Compatibility checker to see if your GPU satisfy the minimum requirement?

I have RTX 5000 Ada

[Kryptos_sim]

My reply was delayed because I’ve been quite busy lately.

Like @Frigi, I followed @zhengwang’s instructions and set things up using tools → Robotics → ROS2 OmniGraphs → RTX Lidar, but it still returns incorrect point clouds. (The issue appears not only in the ROS2 topic but also in the Isaac Sim debug view.)

For reference, @zhengwang is correct — it’s “publish full scan”

Just in case, I ran the Isaac Sim Compatibility Checker, but everything showed up green.

Here are my system specs:

• (265K + RTX 5090 + driver 580 + 128GB RAM)

• (14700K + RTX 4070 + driver 570 + 64GB RAM)
  (Although the compatibility checker shows orange, it’s only due to GPU RAM, so I don’t think it’s a major issue.)

Is there anything that might be different between your setup and mine?
Or any other things I should check?

By the way, I’ll be working on an assignment using Isaac Sim again soon, so I’ll have more time to look into this in detail.

[Kryptos_sim]

Additionally, I’d appreciate it if you could share any updates on the “The Isaac Compute Odometry Node causes position drift (especially, large z drift)” I asked about previously.

[Frigi]

@zhengwang The compatibility checker shows red in VRAM, since my GPU has only 8 GB of VRAM. The rest is green. I thought the undersized VRAM shouldn’t be an issue since there is still more than half the memory free while the simulation is running.

[marmik18]

I am facing a similar issue and I think resources are not the problem since everything was working fine for me until v4.5.0

[BaSiSa]

I’m experiencing a very similar issue using:

• Isaac Sim 5.1.0
• Ubuntu 22.04
• GPU: RTX 4090, Driver 550, nvcc version 11.8

I was updating the lidar config that I was using in Isaac Sim 4.2.0, converting it to USD so it would work properly in 5.1.0. I used convert_lidar_json_to_usda.py to generate new USD file from my modified JSON.

To test it, I placed a few cubes in the scene and verified that the LiDAR output looked correct and the publish rate was steady at 10Hz.

The problem started after I:

1. Saved the newly generated LiDAR USD file
2. Opened the USD that contains my vehicle (created with the PhysX Vehicle Wizard)
3. Replaced the old LiDAR with the new one and adjusted the settings

After doing this, when I ran the simulation the LiDAR output became corrupted – just like in the video and screenshots shared earlier in this thread.

Eventually I discovered the cause:
Whenever the FPS drops, the LiDAR output becomes corrupted. When FPS stays stable at 60, the LiDAR data looks normal. (at least in my setup)

** When I limit the fps to 30, LiDAR publish rate decreases to 5hz but the data looks normal.

[zhengwang]

We have already reached to the internal team and have an internal ticket to keep track of it