Texture Occasionally Not Applied When Using Pathtracing Renderer After Disabling Then Enabling Hydratexture

[mark.xue]

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

• Model: A6000
• Driver Version: 535.247.01

Topic Description

Detailed Description

If I use path tracing renderer and follow the optimization tips to disable render products’ hydra texture during simulation, the textures may not be correctly applied.

Steps to Reproduce

1. Set the renderer to path tracing
2. Disable hydratextures of render products
3. Set texture (moving this step up or down does not resolve the problem)
4. Enable hydratextures
5. Run rep.orchestrator.step()

Script to Reproduce

import numpy as np

from isaacsim import SimulationApp

simulation_app = SimulationApp({"headless": True})
import omni.replicator.core as rep
from isaacsim.core.api import World
from omni.kit.viewport.utility import get_active_viewport
from pxr import UsdGeom
import omni.usd

get_active_viewport().updates_enabled = False
rep.orchestrator.set_capture_on_play(False)

rep.settings.set_render_pathtraced(8)
# rep.settings.set_render_rtx_realtime("Off")


world = World(stage_units_in_meters=0.001, physics_prim_path="/PhysicsScene")
world.get_physics_context().enable_ccd(True)

writer = rep.writers.get("BasicWriter")
out_dir = "test_output"
writer.initialize(output_dir=out_dir, rgb=True)

camera = rep.create.stereo_camera(stereo_baseline=50, position=(0, 0, 800), look_at=(0, 0, 0), focal_length=60)
rps = rep.create.render_product(camera, (2880, 1860))


dome_light = rep.create.light(
    light_type="dome",
    intensity=1,
    exposure=10,
)

cube = rep.create.cube(scale=0.1)
material = rep.create.material_omnipbr()
with cube:
    rep.randomizer.materials(material)

shader_prim = material.get_output_prims()['prims'][0].GetChildren()[0]
diffuse_textures = [...]    # list of path to images
world.reset()
for rp in rps:
    rp.hydra_texture.set_updates_enabled(False)
writer.attach(rps)

num_data = 10
for i in range(num_data):
    print(f"==========Generating Data {i+1}/{num_data}============")

    for rp in rps:
        rp.hydra_texture.set_updates_enabled(False)

    shader_prim.GetAttribute('inputs:diffuse_texture').Set(np.random.choice(diffuse_textures))

    for rp in rps:
        rp.hydra_texture.set_updates_enabled(True)

    rep.orchestrator.step(rt_subframes=20)

rep.orchestrator.wait_until_complete()
writer.detach()
for rp in rps:
    rp.destroy()

simulation_app.close()

Error Messages

None

Screenshots or Videos

Expected output, can be produced either by using rtx realtime renderer or by not disabling and then enabling hydratextures. Notice that every image have textures applied.

Actual output, produced when using path tracing renderer and disabling and enabling hydratextures. The black cubes are the ones without proper application of texture. This happens randomly for either camera.

Additional Information

What I’ve Tried

I’ve tried

1. Disabling multi-GPU rendering
2. Moving texture call to be after enabling hydratextures
3. Changing the rt_subframes number to 3, 10, & 20
4. Add 5 world.step() calls or 5 simulation_app.update() calls right before rep.orchestrator.step()