Replicator on_frame function not working properly

[federico.domeniconi]

The function omni.replicator.core.trigger.on_frame is not working properly.

Calling the function omni.replicator.core.trigger.on_frame(num_frames=100, interval=10) or omni.replicator.core.trigger.on_frame(num_frames=1000, interval=1) leads to the same result but this is not expected. In fact I always end up with 1000 rendered samples and these samples rendered with the two methods are identical.

The documentation says:

• interval – The generation frame interval to execute on.
• num_frames – The total number of frames to execute on.

Therefore if I call the function with the parameters num_frames=100 and interval=10 I expect to render 100 samples, one each 10 time interval, and if I call the function with the parameters num_frames=1000 and interval=1 I expect to render 1000 samples, one each 1 time interval.

[mati-nvidia]

Hi @federico.domeniconi. I’ve moved this over to the SDG forum where they’ll be able to better help you out with this.

[federico.domeniconi]

What is the SDG forum?

[mati-nvidia]

Synthetic Data Generation (SDG)

[valentinhendrik]

Synthetic Data Generation aka Replicator Forum

[dennis.lynch]

Hello,

Sorry the documentation for trigger.on_frame is not clear.

Without a custom Writer, Replicator will always write an image for every frame. What trigger.on_frame does is control when randomizations happen.

For example:

spheres = rep.create.sphere(semantics=[("class", "sphere")], position=(0, 0, 100), count=6)

with rep.trigger.on_frame(num_frames=10,interval=5):
    with spheres:
        rep.modify.pose(
                position=rep.distribution.uniform((-300, 0, -300), (300, 0, 300)),
                scale=rep.distribution.uniform(0.1, 2),
            )

Will generate 50 frames total (10*5) and randomize the position and scale of the spheres every 5th frame.

I can combine this with a separate trigger that performs another randomization

    with rep.trigger.on_frame(num_frames=50):
        with spheres:
            rep.randomizer.color(colors=rep.distribution.normal((0.1, 0.1, 0.1), (1.0, 1.0, 1.0)))

Which will randomize the color of my spheres every frame.

With the two above, I will produce 50 frames, every frame the spheres will change colors, and every 5th frame they will change position and scale.

import omni.replicator.core as rep

with rep.new_layer():

    camera = rep.create.camera(position=(0, 500, 1000), look_at=(0, 0, 0))

    # Create simple shapes to manipulate
    plane = rep.create.plane(semantics=[("class", "plane")], position=(0, -100, 0), scale=(100, 1, 100))
    spheres = rep.create.sphere(semantics=[("class", "sphere")], position=(0, 0, 100), count=6)

    with rep.trigger.on_frame(num_frames=10,interval=5):
        with spheres:
            rep.modify.pose(
                position=rep.distribution.uniform((-300, 0, -300), (300, 0, 300)),
                scale=rep.distribution.uniform(0.1, 2),
            )
    with rep.trigger.on_frame(num_frames=50):
        with spheres:
            rep.randomizer.color(colors=rep.distribution.normal((0.1, 0.1, 0.1), (1.0, 1.0, 1.0)))

render_product = rep.create.render_product(camera, (512, 512))
writer = rep.WriterRegistry.get("BasicWriter")
writer.initialize(
    output_dir="trigger_intervals",
    rgb=True,
)
writer.attach([render_product])

We will try to update the documentation ASAP as well as providing examples, and will try to make the API parameters more clear in a future update.

You might also want to look at trigger.on_time to see if it works for your use-case.

[federico.domeniconi]

Thank you for the explanation.
At this point I have another question: is there a way to render a sample every N frames? I don’t need to randomize anything since I’m running a physics simulation while rendering.

[dennis.lynch]

The “easiest” way to do something like this right now would be to write a custom writer ( you can sub-class BasicWriter or another writer) and have it skip writing certain frames to disk.

They’ll currently still be rendered by the renderer, but won’t be written to disk.

Longer-term, this is a feature that has been requested from multiple sources and we are looking into ways of working with the physics system to introduce callbacks or other methods of conditionally triggering renders.

[lily.chen]

I have no idea how to write a custom writer for setting write one frame every 10 seconds.
Can you provide the Sample code for me? Thank you!

[julian.grimm]

I think you can do this by using rep.trigger.on_time() instead of rep.trigger.on_frame().

https://docs.omniverse.nvidia.com/py/replicator/1.9.8/source/extensions/omni.replicator.core/docs/API.html#omni.replicator.core.trigger.on_time

[danielle.sisserman]

@dennis.lynch I tried your example (changed the numbers a bit):

import omni.replicator.core as rep

with rep.new_layer():

    camera = rep.create.camera(position=(0, 0, 10), look_at=(0, 0, 0))

    spheres = rep.create.sphere(semantics=[("class", "sphere")], position=(0, 0, 0), count=1)

    with rep.trigger.on_frame(num_frames=10,interval=5):
        with spheres:
            rep.modify.pose(
                position=rep.distribution.uniform((-2, 0, -3), (2, 0, 3))
            )
    with rep.trigger.on_frame(num_frames=50):
        with spheres:
            rep.randomizer.color(colors=rep.distribution.normal((0.1, 0.1, 0.1), (1.0, 1.0, 1.0)))

render_product = rep.create.render_product(camera, (512, 512))
writer = rep.WriterRegistry.get("BasicWriter")
writer.initialize(
    output_dir="C:\\Users\\dsiss\\Downloads\\output",
    rgb=True,
)
writer.attach([render_product])

and as a result, the sphere’s position and color are remdomized every five frames (as if both randomizations have the same trigger).

you can see the output here: output - Google Drive

Thank you

[dennis.lynch]

I have a new example too that works with physics. In this example I use an on_time trigger for physics and randomization, and an on_frame trigger to render an image every X-number of frames.

Snippet:

...
    with rep.trigger.on_time(max_execs = 5, interval=2):
        rep.randomizer.create_props()
    manual_trigger = rep.trigger.on_frame(max_execs=5, interval=60) 

writer.attach(render_products=[render_product], trigger=manual_trigger)

The simulation by default plays at 30fps, so with an interval of 60 on the on_frame triggers at the end of the 2 second simulation

My simulation drops fruit from above the box, but only captures an image at the end.