Incorrect depth image generated by camera in IsaacSim


In my project, I was trying to send camera data (rgb and depth) into ROS, and then convert it to point cloud via this node
While using it with gazebo, it works out perfectly:
Original pointcloud, published by simulated camera:

Estimated pointcloud from depth image:

Both pointclouds in the same time:

As can be seen it’s pretty the same, what means, that this node works, as expected. But when I’m trying to use same node with data from Isaac Sim, point cloud look distorted and shifted:

BTW, here is depth image from GazeboSim:

And from Isaac Sim:

Looks like CamerInfo, or Depth Image itself is wrong in Isaac Sim. Is it planned by the Isaac Sim team, to implement point cloud publishing from a simulated camera? Or maybe you have a solution, how to fix this problem?

Looks like a 90 degree rotation clockwise is missing for the bottle environment. Can you see if that helps? Are you using uint16 depth or float in Gazebo/input into depth_image_proc? I’ll try to repro. Thanks.

The distortion was a bug in sim where the vertical aperture used in the camera intrinsics was not computed correctly. Its fixed for the next release.

To work around this temporarily you can compute the correct vertical aperture value and enter it in the USD Camera Prim:

vertical_aperture = image_height / image_width * horizontal_aperture

Where the horizontal_aperture value is what is currently set in the camera prim in the stage. If no camera prim exists, create a new one and make it active in the viewport window.

Which release would this be, I am using 2021.1.1 and the issue is still apparent.

The distortion issue due to the vertical_aperture was fixed for 2021.1.1.
Is this the issue you are referring to?

Yes, I am also having some issues with incorrect pointclouds built off the depthmap generated by isaac sim. Will make a separate post to further explain the issue.

Update: That post is now up Depth Map Issues

1 Like