Dear Nvidia Isaac Sim Team,
Hello. First of all, thank you so much for developing this awesome simulation environment! I am developing the simulation environment based on the following settings: Nvidia Isaac sim version (4.0.0), Ubuntu 20.04, and ROS1 Noetic.
We would like to use an RTX lidar (e.g., Ouster OS 1) to collect ground-truth semantic point clouds. I was able to follow the RTX Lidar sensor tutorial and used the ROS1 RTX Lidar Helper Omnigraph node to successfully collect raw XYZ point clouds.
Because these point clouds do not contain semantic labels, I searched the forum and found this reference that used object ID → get prim path of the object → use prim path to get semantic data of the object to obtain semantic point clouds. While this approach works, the problem is that this approach requires looping every hit object of Lidar, which slows down the process a lot (e.g., a point cloud of 1 Hz; please refer to below code).
def _get_semantic_pc(self):
# get active render_product_path
node = og.Controller.node("/lidarGraph/createRenderProduct")
attribute = node.get_attribute("outputs:renderProductPath")
render_product_path = og.Controller.get(attribute)
# get pc with object ids using annotator
annotator = rep.AnnotatorRegistry.get_annotator(
"RtxSensorCpuIsaacCreateRTXLidarScanBuffer",
init_params={
"outputObjectId": True,
}
)
annotator.attach([render_product_path])
point_cloud = annotator.get_data()
# get each object id's semantic id in point cloud
# NOTE looping through each point in point cloud is very slow
semantic_ids = []
for object_id in point_cloud["objectId"]:
prim_path = syntheticdata_interface.get_uri_from_instance_segmentation_id(object_id)
semantic_data = prims_utils.get_prim_attribute_value(
prim_path,
attribute_name="semantic:Semantics:params:semanticData"
)
# semantic_pc_id_map maps each semantic class (str) to ID (int)
# e.g., self.semantic_pc_id_map = {"floor": 0}
semantic_ids.append(self.semantic_pc_id_map[semantic_data])
I also tried to use the Read RTX Lidar Data Node to grab the semantic IDs in the point clouds, but I noticed that the node no longer has the Semantic ID output in Isaac Sim 4.0.0.
I hope to ask whether there is a much faster way of acquiring the semantic point cloud than my approach above. Also, Similar to how the ROS Camera Helper and PhysX Lidar have enabling semantic options that can be used to publish semantic point clouds, I hope to ask whether RTX Lidar Helper node will have the enabling semantic option in future releases.
I appreciate your help and look forward to hearing back from you.