Trigger Randomization

Omniverse Synthetic Data Generation (SDG)
1

Hello all!

I am still busy with generating synthetic data (lidar sensor) and now I am facing some issues.

  • My plan is to trigger the randomization using a custom event. I just saw that there is the opportunity to register triggers using the replicator api:
omni.replicator.core.trigger.register(fn, override=True)

Is it possible to do so? Are there any examples of the registration of trigger events?

  • The next point is to trigger all writers using a custom event node. Is it possible to call a simple function that is acting as a trigger for them? However, I would use the _attach_trigger function for the writers to connect both of them

My overall plan for the data generation is the following:

triggerRandomization()         # trigger randomization
world.step()                   # do physics simulation
triggerWriters()               #  trigger the writers to retrieve the latest data

Are there any suggestions from you? I did not find any documentation to go ahead with this.
Does my plan make sense?

Best regard,
Christof

2

Just for my understanding:

What is the order of execution in action graph? Is it that the randomization is first executed and when it is finished that the sensor measurement is scheduled?

3

Hi @christof.schuetzenhoefer! You’re correct, randomization is executed, then measurement (ie. annotation) is scheduled, but may not be complete by the time the scene is modified for the next frame. Custom triggers for randomization is possible but not yet easily done. Triggers for writers is a feature we’re actively developing and which is not currently possible. Could you give more details on your use case? We may be able to provide you with a workaround until these features are available, and I’d love to incorporate your feedback into the development and ensure we’re offering the best experience.

4

Thanks for the response @jlafleche !

Workflow should look like this (preferable that the functions blocks the execution until the desired processing is done):

triggerRandomization()         # trigger randomization
world.step()                   # do physics simulation
triggerWriters()               # trigger the writers to retrieve the latest data

For my use case, I want to use the replicator for the randomization of a scene. When this is done, then the measurement of the sensors can be initiated (or a way that I can get the correct results for the finished scene). Finally, the writer can be directly triggered from a function (or after one correct measurement?).

In my opinion, it should be guaranteed that the measurement is captured when the randomization is done (this is essentially for synthetic data). Or another idea is to get the first measurement of the randomized scene after the randomization step (additionally the writers are also triggered).

I hope it is clear for my use case and I am looking forward to figuring out a workaround until this becomes a feature!
: -)

Best Regards,
Christof

5

Hi @christof.schuetzenhoefer ! If I understood your need correctly, you effectively want to be able to run the simulation for X frames before capturing. We are working on a better solution for this scenario, but for now, please try the following workaround, and do let us know if you have any questions!

import omni.replicator.core as rep
import omni.timeline

class DummyWriter(rep.Writer):
    def __init__(self):
        self.annotators = ["rgb"]
    def write(self, data):
        print("WRITING")

sphere = rep.create.sphere()
with sphere:
    rep.physics.rigid_body()
    
rp = rep.create.render_product("/OmniverseKit_Persp", (1024, 1024))
writer = DummyWriter()
writer.attach(rp)


async def main():
    # Play timeline
    omni.timeline.get_timeline_interface().play()

    # Render some frames (not captured)
    for _ in range(24):
        await omni.kit.app.get_app().next_update_async()
    
    # Now render+write one frame
    await rep.orchestrator.step_async()
    
    # Override timeline control setting - only need to do this after the first step
    omni.timeline.get_timeline_interface().set_auto_update(True)
    print("STEPPED")
    
    # The step call will pause simulation, so call play again to render more frames
    omni.timeline.get_timeline_interface().play()
    for _ in range(24):
        await omni.kit.app.get_app().next_update_async()
    
    # Now render+write one frame again
    await rep.orchestrator.step_async()
    print("STEPPED")

import asyncio; asyncio.ensure_future(main())

You should see in the console:

WRITING
STEPPED
WRITING
STEPPED
6

Thank you for the workaround!

I have a question:

  • If I want to use some randomization, how can I ensure that the randomization is finished before capturing images/pointclouds?

Otherwise this workaround looks very good to me! :-)

7

Hi @christof.schuetzenhoefer,

Apologies for the delay. If you’re referring to per-frame randomizations, then they will always be performed before writing. (ie. step_async() will trigger randomizations, then schedule the frame for capture)