Adding RGB-D sensor to robotic end-effector

[kang.yunfeng]

Hey there,

I use Isaac Sim 4.1.0 and OmniGibson, and I already add kuka iwwa7 arm and robotiq 2f-85 gripper to Isaac Sim. Then I tried to add Realsense D435i sensor to the wrist. However, there are only 455 sensor intergrated in Isaac Sim (but with the error of missing reference found, I also cannot add it successfully).

Could you tell me how to add Realsense D435i sensor (or maybe other RGB-D sensor) to the robot in Isaac Sim? Maybe directly in the software or through the urdf file?

Thanks a lot in advance!

Best regards,
Yunfeng

[VickNV]

@kang.yunfeng

However, there are only 455 sensor intergrated in Isaac Sim (but with the error of missing reference found, I also cannot add it successfully).

Could you share the details of this issue? Have you experienced the same problem in Isaac Sim 4.2?

@kang.yunfeng

Could you tell me how to add Realsense D435i sensor (or maybe other RGB-D sensor) to the robot in Isaac Sim? Maybe directly in the software or through the urdf file?

You can use the URDF importer to bring in a USD asset for your desired camera to obtain the mesh. Alternatively, you could create a USD with camera prims positioned appropriately. Then, from a script or extension, add the necessary annotators (such as distance to image plane, RGB, etc.) to the render products associated with those camera prims.