Is there any mapping connecting the instanceId and the path of a prim?

[ilija.kocic]

The title pretty much sums it up. The docs say that “The instance id is assigned in a way that each of the leaf prim in the scene will be assigned to an instance id, no matter if it has semantic labels or not.”
I think I’m missing something very obvious here, but how do you know which instanceId belongs to which prim?

For context, I’m using cam prims and diverse annotators and get_current_frame() to get occlusions, but I have a hard time connecting which occlusion ratio belongs to which prim.
I came across idsToLabels in info part of the instance_id_segmentation annotator output , but get dubious mapping where indices are matched to “INVALID” strings: {'6': 'INVALID', '7': 'INVALID', '20': 'INVALID', '14': 'INVALID'}

From inspecting the values, it seems it is the exact order in which the prims were instantiated.
However, I would like to get a clear connection between path and id.
The reason being that I need to sort through outputted occlusion ratios in the frame and match them to their corresponding prim.

Any help is greatly appreciated!

[ilija.kocic]

Update to this post: I can definitively confirm that the instanceId is not necessarily encoding the order of instantiation, as I’ve preserved that order elsewhere and yet for a larger number (say >5) of instances this seems to change and the order is lost. And neither is the unsorted list of instances given by get_current_frame() as is. Tested both possibilities in code as a workaround, alas neither worked.

[ahaidu]

Hi there,

it seems labeling the assets will provide you with the prim paths, see this script editor example:

import asyncio
import omni.usd
from omni.isaac.core.utils.semantics import add_update_semantics
import omni.replicator.core as rep


omni.usd.get_context().new_stage()
rep.create.light()
cube = rep.create.cube(position=(0.5, 0.5, 0))
sphere = rep.create.sphere(position=(-0.5, -0.5, 0))

cube_prim = cube.get_output_prims()["prims"][0]
sphere_prim = sphere.get_output_prims()["prims"][0]

rp = rep.create.render_product("/OmniverseKit_Persp", (512, 512))

instance_id_segm_annot = rep.AnnotatorRegistry.get_annotator("instance_id_segmentation")
instance_id_segm_annot.attach(rp)

instance_segm_annot = rep.AnnotatorRegistry.get_annotator("instance_segmentation")
instance_segm_annot.attach(rp)

semantic_segm_annot = rep.AnnotatorRegistry.get_annotator("semantic_segmentation")
semantic_segm_annot.attach(rp)

def print_annot_data():
    print(f"instance id data info: {instance_id_segm_annot.get_data()['info']}")
    print(f"instance segmentation data info: {instance_segm_annot.get_data()['info']}")
    print(f"semantic segmentation data info: {semantic_segm_annot.get_data()['info']}")
    print(80*"-")

async def run_example_async():
    # Get annotator data
    await rep.orchestrator.step_async()
    print(f"Unlabeled prims data:")
    print_annot_data()

    # Label prims
    add_update_semantics(cube_prim, "my_cube_label")
    add_update_semantics(sphere_prim, "my_sphere_label")

    # Get annotator data
    await rep.orchestrator.step_async()
    print(f"Labeled prims data:")
    print_annot_data()

asyncio.ensure_future(run_example_async())

Resulting in:

Unlabeled prims data:
instance id data info: {'idToLabels': {'4': 'INVALID', '3': 'INVALID'}}
instance segmentation data info: {'idToLabels': {}, 'idToSemantics': {}}
semantic segmentation data info: {'_uniqueInstanceIDs': array([1, 1], dtype=uint8), 'idToLabels': {'0': {'class': 'BACKGROUND'}, '1': {'class': 'UNLABELLED'}}}
--------------------------------------------------------------------------------
Labeled prims data:
instance id data info: {'idToLabels': {'4': '/Replicator/Sphere_Xform/Sphere', '3': '/Replicator/Cube_Xform/Cube'}}
instance segmentation data info: {'idToLabels': {'0': 'BACKGROUND', '1': 'UNLABELLED', '3': '/Replicator/Sphere_Xform', '2': '/Replicator/Cube_Xform'}, 'idToSemantics': {'0': {'class': 'BACKGROUND'}, '1': {'class': 'UNLABELLED'}, '3': {'class': 'my_sphere_label'}, '2': {'class': 'my_cube_label'}}}
semantic segmentation data info: {'_uniqueInstanceIDs': array([1, 1], dtype=uint8), 'idToLabels': {'0': {'class': 'BACKGROUND'}, '2': {'class': 'my_cube_label'}, '3': {'class': 'my_sphere_label'}}}
--------------------------------------------------------------------------------

If you are interested in occlusion data you can also use the occlusionRatio from the 3d bbox annotator.

[ilija.kocic]

Hello Mr. Haidu,
thank you for taking the time to answer and craft a working demonstrator just for this post! Studying the examples given for the annotators I started suspecting the lack of labeling was the cause.
However the documentation’s example using the replicator require me to redesign my code quite a bit, since I’ve built a so called Standalone App, using the templates provided by Nvidia Isaac.
I realize now (if I understood you correctly) that both yours and the doc examples were intended to be used with Omniverse Kit’s Script Editor. As someone developing the standalone app, this was entirely unclear to me. Your casual mentioning what the example was for made it click.

In the meanwhile I have been busy looking for a workaround realizing that it is sufficient to match occlusion ratio with the pose of the object, which is exactly what the 2d/3d bbox annotator delivers (as per your suggestion)
However, adding this Annotator to the frame of the Camera prim also leaves me with an entirely empty list.

As with most problems in this line of work, someone had it first: last Friday I’ve found this question: Add_pointcloud_to_frame() API not working for omni.issac.sensor.camera (Isaac-sim 2022.2.0)

It is exactly the same question I’m posing now, but with the 3d-bbox-annotator (not the 3d pointcloud annotator as with the post), my code looks quite similar as I am also making a standalone app, structure-wise they could be considered identical.

The simple solution in order to get some output from the bbox annotator once all is said and done, is to use the replicator and imbue each prim (we are interested in) with a semantic label (copied from the above post’s winning answer):

import omni.replicator.core as rep
semantic_type = "class"
semantic_label = "cone" # The label you would like to assign to your object
prim_path = "/World/Cone" # the path to your prim object
rep.modify.semantics([(semantic_type, semantic_label)], prim_path)

I will attempt to do this and report the results.

[ilija.kocic]

Almost forgot to check back in; As suspected Using the replicator didn’t work for me, as it would have required to rewrite too much, but adding a single line of code to my script where I add prims did the trick:

                add_reference_to_stage(usd_path=projectile_url, prim_path=prim_path)
                projectile_viewers[clone].append(world.scene.add(
                        RigidPrim(prim_path=prim_path,
                                  name='projectile_clone' + str(clone) + '_wave' + str(wave),
                                  # Randomizing change position of each clone
                                  position=clone_pos[clone] + rand_positions[clone][wave],
                                  orientation=clone_orient[clone] + rand_orientations[clone][wave]
                                 )
                    )               
                )
                projectile_prim = stage.GetPrimAtPath(prim_path)
                utils.setCollider(projectile_prim, approximationShape=collider_type)
                mass_api = UsdPhysics.MassAPI.Apply(projectile_prim)
                mass_api.CreateDensityAttr(projectile_conf['density']['value'])
                add_update_semantics(projectile_prim, 'projectile')

What did the trick is the last line (and import of course):

from omni.isaac.core.utils.semantics import add_update_semantics
add_update_semantics(projectile_prim, 'projectile')

All of a sudden bbox annotator outputted what I needed it to.

Thanks for assisting!