It turns into a white asset without being colored in the assets

Omniverse Synthetic Data Generation (SDG)
1

Hi, experts.

When executing the following code, the asset does not get colored and turns white instead. The same issue occurs when using other USD paths.

One possible explanation could be that the rendering is not keeping up with the processing in the writer.
Do you have any advice or suggestions?

import omni.replicator.core as rep

with rep.new_layer():

    # Add Default Light
    distance_light = rep.create.light(rotation=(315,0,0), intensity=3000, light_type="distant")

    camera = rep.create.camera(position=(1000,1000, 1000),rotation=(-35,45,0))
    render_product = rep.create.render_product(camera, (1024, 1024))

    PROPS = 'omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/'
    # PROPS = "omniverse://localhost/NVIDIA/Assets/ArchVis/Commercial/Seating/"
    # PROPS = "omniverse://localhost/NVIDIA/Assets/ArchVis/Commercial/Seating/Caprice/"
    # PROPS = "omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/"

    plane = rep.create.plane(scale=10, visible=True)

    def get_props(size):
        instances = rep.randomizer.instantiate(rep.utils.get_usd_files(PROPS, recursive=True), size=size, mode='point_instance')
        with instances:
            rep.modify.pose(
                position=rep.distribution.uniform((-500, 0, -500), (500, 0, 500)),
                rotation=rep.distribution.uniform((-90, -180, 0), (-90, 180, 0)),
            )
        return instances.node

    rep.randomizer.register(get_props)

    # Setup randomization
    with rep.trigger.on_frame(num_frames=30):
        rep.randomizer.get_props(3)

    # Initialize and attach writer
    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize( output_dir="random_assets", rgb=True)
    writer.attach([render_product])

    rep.orchestrator.preview()

image
image870×911 54 KB

2

@k_m i can’t seem to repro the issue on my end using Isaac Sim 2023.1.0 hotfix.1. which version are you using?

can you double check to see if you can open up a few of the source USD’s to make sure there are materials on the geom prim? and is it possible that you may have toggled on White Mode by accident? or is it only white in render product but is fine in viewport?

image

3

Thank you for checking.
I apologize for any confusion, but I am using Omniverse Code 2022.3.3, not Isaac Sim.

4

i don’t know if the version would really matter to check the original assets and see if you had white mode toggled on. i might install specific version of Code that matches yours to confirm later.

5

Hi @k_m Can you post your log here? I’m curious if the asset can’t find the textures, or a missing material.

6

I have sent the log, image files, and respective codes.
random_assets_code.zip (4.0 KB)
random_assets.zip (6.0 MB)

In the case of “plant_tropical” and “Fruit,” the assets are turning white. There seems to be an error in “plant_tropical,” but there are no specific errors in the case of “Fruit.”

image
image1356×955 131 KB

For “caprice” and “Seating,” the assets do not turn white. However, a color change from the original assets is causing colorful assets to be generated.

image
image1252×956 89.2 KB

image
image1933×1580 273 KB

7

i think there is a lead from the log, but would like to hear @pcallender 's opinion - seems like there is an issue on the shader/material level. again, i am barely a novice in SDG, so i cannot say for sure, but there are repeats of the similar line in the log, such as Destroying MdlShadeNode for prim: ... or Deleted MdlEntity for prim: ...

so when i compared what the NVIDIA assets have versus the screenshot you showed, i was able to correlate the random color and that MDLs were either being removed or unbounded from the mesh during renderoutput that results in the primvar color. below are clips showing the underlying color when shader and materials are removed from furniture assets you tested:

CapriceA_MDL_Issue

JobbaSoffa_MDL_Issue

that said, i cannot say for sure sure whether this behavior is app specific or user setup related.

8

Thank you for checking!

As you noted, there is a possibility that the removal of the shader, as you confirmed, might be the cause.

Additionally, I made a slight change to the code. Instead of using rep.randomizer.instantiate, I used rep.create.from_usd, and I was able to output the image without changing the color of the asset. It’s possible that the processing within rep.randomizer.instantiate might be the cause of the issue.

import omni.replicator.core as rep
import re

with rep.new_layer():

    # Add Default Light
    distance_light = rep.create.light(rotation=(315,0,0), intensity=3000, light_type="distant")

    camera = rep.create.camera(position=(1000,1000, 1000),rotation=(-35,45,0))
    render_product = rep.create.render_product(camera, (1024, 1024))

    PROPS = ['omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Cuban_Royal_Palm.usd',
             "omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Jungle_Flame.usd",
             "omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Honey_Myrtle.usd",
             "omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Japanese_Flame.usd",
             "omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Windmill_Palm.usd",
             "omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Agave.usd",
             "omniverse://localhost/NVIDIA/Assets/Vegetation/Plant_Tropical/Honey_Myrtle.usd"]

    plane = rep.create.plane(scale=10, visible=True)

    def get_props():
        # instances = rep.randomizer.instantiate(rep.utils.get_usd_files(PROPS, recursive=True), size=size, mode='point_instance')
        asset_list = []
        for PROP in PROPS:
            asset_list.append(rep.create.from_usd(PROP))
        instances=rep.create.group(asset_list)
        with instances:
            rep.modify.pose(
                position=rep.distribution.uniform((-500, 0, -500), (500, 0, 500)),
                rotation=rep.distribution.uniform((-90, -180, 0), (-90, 180, 0)),
            )
        return instances.node

    rep.randomizer.register(get_props)

    # Setup randomization
    with rep.trigger.on_frame(num_frames=10):
        rep.randomizer.get_props()

    # Initialize and attach writer
    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize( output_dir=("random_assets_from_usd"), rgb=True)
    writer.attach([render_product])

    rep.orchestrator.preview()

random_assets_from_usd.zip (981 Bytes)

image
image1920×692 103 KB