What is the standard way to initialize/cleanup an annotator and get its data in a custom OmniGraph node?

[xcali]

The documentation for Omniverse Replicator has examples for getting annotator data when running headless in a Python script.

It is reproduced below for reference:

import omni.replicator.core as rep

async def test_ldr():
    # Add Default Light
    distance_light = rep.create.light(rotation=(315,0,0), intensity=3000, light_type="distant")

    cone = rep.create.cone()

    cam = rep.create.camera(position=(500,500,500), look_at=cone)
    rp = rep.create.render_product(cam, (1024, 512))

    ldr = rep.AnnotatorRegistry.get_annotator("LdrColor")
    ldr.attach(rp)

    await rep.orchestrator.step_async()
    data = ldr.get_data()
    print(data.shape, data.dtype)   # ((512, 1024, 4), uint8)

import asyncio
asyncio.ensure_future(test_ldr())

When running Isaac Sim with a GUI and using custom OmniGraph nodes, however, how should annotators be initialized/released and then used to get data directly from within the node considering you cannot use the orchestrator functions in this context? For example, it is not valid to call rep.orchestrator.step_async() from within the node so how can one make sure that the data is available for access when you need it?

By simply calling get_data() in the compute function of the node I’ve run into various issues such as:

1. The data is None indicating it is not yet available (it may become available at subsequent invocations but that data is likely from the previous timestep).
2. The data is invalid or repeating upon subsequent plays.

I thought there could be some effect changing the pipeline stage, however, I think I should be using the simulation stage for my use case. It seems like the post-render pipeline stage is intended for scheduling after an offline rendering job since the graph does not run at all when I use it. The pre-render pipeline stage instead always runs, even if the simulation is stopped. The on-demand pipeline stage is not applicable for this which leaves just the simulation pipeline stage as the most appropriate.

Clarification on the correct initialization and cleanup procedure of annotators in custom OmniGraph nodes within the context of the asynchronous simulation architecture would be very helpful.

[jlafleche]

Is your custom node added to the annotation graph? If so, you’d want to connect to the annotator’s output pins and pass the data that way. If your node is in a different graph, then it’s a bit less straightforward, but not that different from a script - you would have to create your own way to pass the annotator object to your custom node and then call get_data() from within your node. You still need to call step_async() or run() to have the annotator generate data from an external script however. And you need to be careful that your node is not grabbing stale data - and this seems to be what you encountered. You can do this by having your node wait until the reference time of the annotator graph matches the expeted reference time for the current frame. See the omni.replicator.core.ReadFabricTime for getting the current reference time.