Unable to publish organized point cloud (is_dense: false, skipDroppingInvalidPoints: true, keepOnlyPositiveDistance: false) from RTX Lidar to ROS

[ilya.yatsyshyn]

Isaac Sim Version

5.0.0

Operating System

Ubuntu 22.04

GPU Information

• Model: NVIDIA GeForce RTX 5080
• Driver Version: 575.57.08

Topic Description

Unable to publish organized point cloud (is_dense: false) from RTX Lidar to ROS.

Detailed Description

I rely heavily on the organized point cloud:

1. number of points is constant, points indexes don’t change
2. some points have NaN/0,0,0 values if they were not returned
3. sensor_msgs.msg._point_cloud2 is_dense field is set to false.

Screenshots or Videos

Additional Information

What I’ve Tried

1. setting skipDroppingInvalidPoints to true on the lidar prim
2. searching for keepOnlyPositiveDistance param of the ComputeRTXLidarPointCloud, but none of these seem to exist in 5.0.0. I would fall back to 4.5.0, but there is a much more critical issue with dragging points I experienced myself ( RTX Lidar drag points issue in debug view and full scan point cloud publishing )
3. searching for workarounds, such as sending emitterId and row alongside XYZ to further populate empty points - no luck

Related Issues

1. Unable to get Organized Point Cloud , RGB Point Cloud from Isaac Sim via Omni.Ros2.Bridge in ROS2
2. Pointcloud configuration options (intensity, noise, pointcloud structure) for Isaac Sim and ROS Interface

[zhengwang]

We recently released 5.1.0 OSS on github. Could you please try it and see if this issue still persists? Thanks!

[ilya.yatsyshyn]

Hello,

Sure I can try, but I need to know what flag or config to look for. I’ve checked 5.1 docs before publishing this ticket and found nothing new regarding this topic.

Can you point me in some specific direction?

[zhengwang]

Hi @ilya.yatsyshyn! I reached out to the internal team and confirmed that this is not available for Isaac Sim 5.0.0 or 5.1.0. But I created an internal feature request and they can target to release this feature in Isaac Sim 6.0.0.

For now, you can update the Isaac Sim open source codebase to achieve your goal:

The places to modify are:

• In code file:

  • below existing line 431 add:

  fillIndices(indicesValidBuffer.data(), m_maxPoints, m_maxThreadsPerBlock, cudaDeviceIndex);
  

  • Comment out existing lines 1065-1073
  • Comment out existing line 1075
  • Above existing line 1074, add:

  state.numValidPointsHost =  db.inputs.enablePerFrameOutput() ? numElementsToCopyToCurrentBuffer : state.m_maxPoints;
            CUDA_CHECK(cudaMemcpyAsync(state.numValidPointsDevice, state.numValidPointsHost, sizeof(int),
                                         cudaMemcpyHostToDevice, pointCloudStream));
  

• For this code, set is_dense to False here.

The above changes should do:

• bypass the point validity check in IsaacCreateRTXLidarScanBuffer so all points are considered “valid”
• Set is_dense in the ROS2 PointCloud2 message to false always

Note that this may dramatically increase PointCloud2 size, increasing VRAM consumption and lowering frame rate.