Zed camera with depth and point cloud

Hello there,

I want to simulate a zed camera with depth and also point cloud data in Isaac SIM . However it seems that isaac sdk only provide left and right camera image in isaac.ZedCamera component and the simulation method for stereo camera varies a lot. Ex. in sdk, carter generate depth with TimeSynchronizer from color image in UE4 and sdk does not provide a sample code for generating point cloud data.

I am wondering what will be the best way to simulate the zed camera with isaac sdk? Anyone could provide some suggestions?


Best Regards,

Hi Rosy,
You can use Coarse-to-Fine Stereo Depth (sample in //apps/samples/stereo_matching_depth/stereo_matching_depth.app.json) or Stereo Depth DNN (sample in //apps/samples/stereo_depth_dnn.app.json)
Both receive a pair of stereo images and output a depth image.

To transform a Depth Image into a Point Cloud take a look at isaac::rgd_processing::DepthImageToPointCloud . Examples on this component being integrated in an app: //apps/samples/hgmm_matching/hgmm_matching.app.json and //apps/samples/realsense_camera/realsense_camera.app.json