Serious issues with ghosting

Omniverse Synthetic Data Generation (SDG)
1

Hi all, I am having serious issues with ghosting, the only way I am able to fix this is to raise the items far off the table but I need the items on the table. Can anyone suggest the solution for this, project deadline is tomorrow and I have yet to be able to complete the dataset. There should only be 3 apples in each image. Out of 50 images only 5 do not have this. I have tried disabling Sampled Light etc. I am running on an A40 and also experience constant crashing and need to start new stage every time I want to rerun the code.

rgb_0025
rgb_00251024×1024 125 KB

rgb_0020
rgb_00201024×1024 118 KB

rgb_0022
rgb_00221024×1024 119 KB

rgb_0023
rgb_00231024×1024 119 KB

rgb_0024
rgb_00241024×1024 124 KB

2

Code:

import datetime
import omni.replicator.core as rep

basepath = "c:\\Users\\adam\\Desktop\\Omniverse\\Src"
dataset = "Apple"
output_dir = basepath+'\\data\\rendered\\'+datetime.datetime.now().strftime("%Y-%m-%d-%H-%M-%S")+'\\'+dataset

with rep.new_layer():

    TABLE_ASSET = "http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Residential/Furniture/DiningSets/EastRural/EastRural_Table.usd"
    FRUIT = {
        "Apple":  "http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Residential/Food/Fruit/Apple.usd",
        "Orange":"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Residential/Decor/Tchotchkes/Orange_01.usd",
        "Lime": "http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Residential/Food/Fruit/Lime01.usd"
    }
    
    def table():
        table = rep.create.from_usd(
            TABLE_ASSET, semantics=[('class', 'table')])

        with table:
            rep.physics.collider()
            rep.physics.mass(mass=100)
            rep.modify.pose(
                position=(0, 0, 0),
                rotation=(0, -90, -90),
            )
        return table
        
    rep.randomizer.register(table)

    def rect_lights(num=1):
        lights = rep.create.light(
            light_type="rect",
            temperature=rep.distribution.normal(6500, 500),
            intensity=rep.distribution.normal(0, 50),
            position=(0, 110, 0),
            rotation=(-90, 0, 0),
            count=num
        )
        return lights.node

    rep.randomizer.register(rect_lights)

    def dome_lights(num=3):
        lights = rep.create.light(
            light_type="dome",
            temperature=rep.distribution.normal(6500, 500),
            intensity=rep.distribution.normal(0, 500),
            position=(0, 120, 18),
            rotation=(225, 0, 0),
            count=num
        )
        return lights.node

    rep.randomizer.register(dome_lights)
    
    def randomize_asset(fpath, fclass, maxnum = 1):
        instances = rep.randomizer.instantiate(
            fpath, size=maxnum, mode='scene_instance')
        with instances:
            rep.physics.collider()
            rep.physics.mass(mass=100)
            rep.modify.semantics([('class', fclass)])
            rep.modify.pose(
                position=rep.distribution.uniform(
                    (-60, 80, -30), (60, 80, 30)),
                rotation=rep.distribution.uniform(
                    (-90, -180, -90), (90, 180, 90))
            )
        return instances.node
       
    rep.randomizer.register(randomize_asset)
    
    camera = rep.create.camera(
        position=(0, 200, 0), rotation=(-90, 0, 0))
    render_product = rep.create.render_product(camera, (1024, 1024))

    camera2 = rep.create.camera(
        position=(0, 125, 0), rotation=(-90, 0, 0))
    render_product2 = rep.create.render_product(camera2, (512, 512))
                
    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize(
        output_dir = output_dir, 
        rgb=True, bounding_box_2d_tight=True)
    writer.attach([render_product, render_product2])

    with rep.trigger.on_frame(num_frames=50):
        # Table
        rep.randomizer.table()
        # Lights
        rep.randomizer.rect_lights(1)
        rep.randomizer.dome_lights(1)
        # Fruit
        if dataset == "None":
            pass
        elif dataset == "All":
            for fclass, fpath in FRUIT.items():
                rep.randomizer.randomize_asset(fpath, fclass, 3)
        else:
            rep.randomizer.randomize_asset(FRUIT[dataset], dataset, 3)
    
    rep.orchestrator.run()
3

It seems to be a combination of issues that make this happen. One is definitely is you try to get a close-up of an object, in this project I needed close-ups, and every attempt failed, as soon as you get closer to the object whether using camera placement or focal length this ghosting happens.

4

OK so this issue only appears to happen when using RTX Real-Time, so far, changing rendering mode to RTX Interactive (Path Tracing) Seems to allow me to get much closer to subject and no more ghosting, so far. Hope it helps someone.

5

Hi @AdamMiltonBarkerOfficial I def recommend trying subframes to combat ghosting. Check out the pages here:

Rendering with Subframes Examples — Omniverse Extensions documentation (nvidia.com)