Point Cloud Lag in Odom Frame with Simulated RTX LiDAR in NVIDIA Isaac Sim

Omniverse Isaac Sim
1

Isaac Sim Version

4.2.0
4.1.0
4.0.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: RTX 4080 Super
  • Driver Version: 535.183.01

Topic Description

Hello everyone,

I’ve encountered a strange issue with a simulated RTX LiDAR sensor attached to a mobile manipulator in NVIDIA Isaac Sim. When I visualize the point cloud topic in rviz, displaying it in the odom frame, there seems to be a slight lag in the point cloud. It becomes apparent because the points that should hit the robot’s base appear with a noticeable offset. The cloud is “lagging” behind and “jumping” to catch up with the robots rotation.

Interestingly, this behavior disappears when I visualize the point cloud in one of the robot’s local frames (e.g., base_link or the sensor’s own frame).

For reference, I’m publishing the odometry information as suggested in NVIDIA’s documentation for Isaac Sim, available here.

Has anyone experienced similar issues or have insights into why this might be happening? Could it be due to a delay in frame transformations, or something else in the setup?

Thanks in advance for your help!

image
image460×484 13 KB

image
image903×473 51 KB

image
image911×719 79.8 KB

2

A few things to think about here.

I think what you are seeing is the update rate of the lidar vs the frame rate of the viewport. Say your lidar is 10Hz and you are running at 60Hz it will look off if you pause it. I’ve had similar experiences, it starts getting very technical real fast.

If you set a camera with a frame rate of 10 fps or make the scene run at 10Hz it should resolve the issue you are seeing.

Also there is an “issue” with the way lidar works when you rotate it. say the scan takes 0.1 of a second to do a full scan. Each point has a point in time that it returns if you rotate the lidar, it dosent know its turned. So if turned a lidar at 10 rpm you would be seeing very strange results, if you are following me here. Its basically rolling shutter but for lidar.

Its not a bug its just the way rotating lidars work, solid state lidars are better (as far as I’m aware) but also have a scan time so dont eliminate it.