Hello,
I’d like to create a camera to get the point clouds and segmentation masks using the replicators. I found the doc about Point Cloud, however, when I launch the following code:
from omni.isaac.kit import SimulationApp
headless = False
simulation_app = SimulationApp({"headless": headless}) # we can also run as headless.
import omni.kit
from pxr import UsdGeom
import omni.replicator.core as rep
stage = omni.usd.get_context().get_stage()
# Create cube
cube_prim = stage.DefinePrim("/World/Cube", "Cube")
UsdGeom.Xformable(cube_prim).AddTranslateOp().Set((0., 5., 1.))
# Create render products
rp = rep.create.render_product("/OmniverseKit_Persp", (1024, 1024))
# Get the annotators for the data
semantic_segmentation_annotator = rep.AnnotatorRegistry.get_annotator("semantic_segmentation")
pointcloud_annotator = rep.AnnotatorRegistry.get_annotator("pointcloud")
distance_to_camera_annotator = rep.AnnotatorRegistry.get_annotator("distance_to_camera")
# Attach annotators
semantic_segmentation_annotator.attach([rp])
pointcloud_annotator.attach([rp])
distance_to_camera_annotator.attach([rp])
rep.orchestrator.step()
# Access annotator data after each replicator process step
print("Instance segmentation: ", semantic_segmentation_annotator.get_data())
print("Point cloud: ", pointcloud_annotator.get_data())
print("Depth: ", distance_to_camera_annotator.get_data())
It generates an error:
[Error] [omni.graph.core.plugin] Assertion raised in compute - cannot reshape array of size 0 into shape (0,newaxis)
File "/home/ragi/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/ogn/python/_impl/nodes/OgnPointCloudGenerator.py", line 19, in compute
rgb = db.node.get_attribute("inputs:rgb").get_array(False, False, 0).reshape(width*height, -1)
This error does not appear if I remove the semantic_segmentation annotator but I need this annotator to be able to access to the field IdToLabels.
Is there a way to use one camera to get both the segmentation, idToLabel info and the point clouds without error ?