Is it possible to randomize the time of day and other parameters of a dynamic sky using replicator?
Hi @mhz Can you provide a few more details on what you’re looking for? Are you referring to manipulating a direct light in a scene, or something more complex like an extension that simulates sun elevations, colors etc?
Take a look at this extension:
Sun Study — Omniverse Extensions latest documentation (nvidia.com)
I don’t think we have direct replicator integration with this extension (I’m not too familiar with it), but it might be a place to start.
I am looking for a way to modify the time of day of a dynamic sky using replicator. This is to get more variation of lighting conditions in the dataset. I guess that it is not possible to do that right now using replicator, but it would be a useful feature to have. The only way I have seen otherwise is to randomize light using HDR for the environment.
HI mhz.
There is no readily available randomizer function available in replicator for that. And If you only change the time of the day attribute, you won’t actually change the sky appearance
The way around that I used is the following (disclamer: it is a bit tedious).
- Install the sun study extension. This will allow you to modify manually the time of the day while keeping it consistent with the other sky parameters (like the azimut, sun color, etc)
- Open the sun study extension from the menu, and manually change the time of the day using the sun study extension bar. Record each time the value of the following attributes:–>TimeOfDay, SHA, Azimuth, Elevation, SunColor.
- randomize the attributes of the sky’s shader (using rep.modify.attribute) consistently : at each frame, the value of each attribute must correspond to the value recorded for the same time of the day in point 2.
I am not sure if what I have written above makes any sense as I am writing on the fly, but when I get a chance, I will provide some screenshots
2 Likes
Hi @mhz,
@romichel2003 has described exactly what I’d do in your place. Modifying the sun study attributes using rep. I’ve added this to the list of tutorials to cover in the future (at some point). Its a common use case and would be handy to have to point people to.
Thanks for bringing this up!
2 Likes
Sure hope it will be possible to manipulate time of day, weather and such through replicator soon.
Hi folks,
Our team also wants to use replicator to change the time of day for a dynamic sky. I have followed the advice in this thread to use rep.modify.attribute and this works for all parameters except for SunColor. As far as I can tell this is a bug but I cannot find a workaround it. Here is a minimum reproducible example:
import omni.replicator.core as rep
with rep.new_layer():
env = rep.create.from_usd("https://omniverse-content-production.s3.us-west-2.amazonaws.com/Environments/2023_1/DomeLights/Dynamic/CumulusLight.usd")
with rep.get.prims("/Replicator/Ref_Xform/Ref/Looks/SkyMaterial/Shader"):
rep.modify.attribute("inputs:TimeOfDay", 1.23)
rep.modify.attribute("inputs:SHA", 1.23)
rep.modify.attribute("inputs:Azimuth", 1.23)
rep.modify.attribute("inputs:Elevation", 1.23)
rep.modify.attribute("inputs:SunColor", (0.5, 0.5, 0.5))
After running this in omniverse code, I see that the first four parameters were updated correctly, but I also see an error in the console:
2024-01-29 18:32:12 [Error] [omni.graph.core.plugin] /Replicator/SDGPipeline/OgnWritePrimAttribute_04: OmniGraph Error: WritePrimAttribute Error: Attribute ‘inputs:SunColor’ could not be found for prim at /Replicator/Ref_Xform/Ref/Looks/SkyMaterial/Shader
2024-01-29 18:32:12 [Error] [omni.graph.core.plugin] (from compute() at line 241 in /home/nikita/.local/share/ov/pkg/deps/28c70d9b2f0e7eb6e8d60682fea71057/extscache/omni.replicator.core-1.7.7+104.2.lx64.r.cp37/omni/replicator/core/ogn/python/_impl/nodes/OgnWritePrimAttribute.py)
Can you please suggest a workaround or let me know if there is an issue with my code?
Edit with some more info:
After running my above code in Omniverse Code, I am able to run:
with rep.get.prims("/Replicator/Ref_Xform/Ref/Looks/SkyMaterial/Shader"):
rep.modify.attribute("inputs:SunColor", (0.5, 0.5, 0.5))
which changes the SunColor correctly. This workaround doesn’t work for us as we were hoping to run our data generation script headlessly.
@nshymberg After checking this a few times I think I have an idea of what’s happening here.
There’s an optimization we’ve encountered where attributes aren’t accessible until they’re ‘touched’ by the GUI. In the case of headless, this isn’t possible. So here’s the workaround I use …
You can write some python that checks the validity of the attributes on the script run (which only happens once at the beginning). If the attribute isn’t valid, you create that attribute. This won’t override it if it actually exists, but will make it accessible in the script now.
Below is my script example for doing this with textures, but the principle should be the same for you.
Pay specific attention to:
def make_asset_attr_accessible(prim, attribute_name, attr_value):
if not prim.GetAttribute(attribute_name).IsValid():
prim.CreateAttribute(attribute_name, Sdf.ValueTypeNames.Asset, custom=True).Set(attr_value)
print(f'Set prim for {attribute_name}')
and
# Create the primvars on the material - this is to work around an optimization where attributes aren't settable via script until they're viewed in GUI. The workaround is to create them on the prim, then they can be set.
make_asset_attr_accessible(pbr_shader_prim, "inputs:normalmap_texture", "")
make_asset_attr_accessible(pbr_shader_prim, "inputs:diffuse_texture", "")
make_asset_attr_accessible(pbr_shader_prim, "inputs:reflectionroughness_texture", "")
Full script here. Hopefully this resolves your issue!
import glob
import re
import os
from pathlib import Path
from pxr import Usd, UsdGeom, Gf, Sdf, Vt
import omni.usd
import omni.replicator.core as rep
def make_asset_attr_accessible(prim, attribute_name, attr_value):
if not prim.GetAttribute(attribute_name).IsValid():
prim.CreateAttribute(attribute_name, Sdf.ValueTypeNames.Asset, custom=True).Set(attr_value)
print(f'Set prim for {attribute_name}')
def hide_default_light():
# Hide the default light
default_light_prim = stage.GetPrimAtPath("/Environment/defaultLight")
if default_light_prim.IsValid():
xform = UsdGeom.Xformable(default_light_prim)
xform.MakeInvisible()
def get_matching_files(directory, regex_pattern, file_extension):
matching_files = []
# Compile the regex pattern for efficient repeated use
pattern = re.compile(regex_pattern)
# List all files in the given directory
for filename in os.listdir(directory):
# Check if the file has the required extension
if filename.endswith(file_extension):
# Check if the file name matches the regex pattern
if pattern.search(filename):
# Add the matching file name to the list
matching_files.append(os.path.join(directory, filename))
return matching_files
stage = omni.usd.get_context().get_stage()
directory = Path("C:/YOURDIR/texture_lib_small/").as_posix()
file_extension = ".jpg" # Example file extension
patterns = ['_Color', '_Displacement', '_NormalGL', '_Roughness']
# Make lists of all texture channels
diffuse_files = get_matching_files(directory, patterns[0], file_extension)
normal_files = get_matching_files(directory, patterns[2], file_extension)
roughness_files = get_matching_files(directory, patterns[3], file_extension)
# Scene settings
rep.settings.set_stage_up_axis("Y")
rep.settings.set_stage_meters_per_unit(0.01)
hide_default_light()
# Creating a new layer in the stage ensures all changes done can be isolated, and we do not accidentally change our original scene.
with rep.new_layer():
# Create a material
pbr_mat = rep.create.material_omnipbr(count=1)
pbr_primpath = '/Replicator/Looks/OmniPBR/Shader'
pbr_shader_prim = stage.GetPrimAtPath(pbr_primpath)
pbr_shader = rep.get.prims(path_pattern=pbr_primpath)
# Create the primvars on the material - this is to work around an optimization where attributes aren't settable via script until they're viewed in GUI. The workaround is to create them on the prim, then they can be set.
make_asset_attr_accessible(pbr_shader_prim, "inputs:normalmap_texture", "")
make_asset_attr_accessible(pbr_shader_prim, "inputs:diffuse_texture", "")
make_asset_attr_accessible(pbr_shader_prim, "inputs:reflectionroughness_texture", "")
# Create basic replicator objects in the stage - These are created under "Replicator/"
floor = rep.create.cube(position=(0,0,0), scale=(10,0.1,10))
cone = rep.create.cone(position=(0,100,0), scale=2)
camera = rep.create.camera(position=(0,600,1500), rotation=(-22,0,0), focus_distance=200,f_stop=8)
distance_light = rep.create.light(rotation=(-45,0,0), intensity=1500, temperature=6500, light_type="distant")
# Apply the materials to the floor
with floor:
rep.modify.material(pbr_mat)
# This is a randomizer, a special replicator way to organizer the graph logic (not a python function!)
def randomize_texture(shader):
with shader:
rep.modify.attribute("inputs:diffuse_texture", rep.distribution.sequence(diffuse_files))
rep.modify.attribute("inputs:reflectionroughness_texture", rep.distribution.sequence(roughness_files))
rep.modify.attribute("inputs:normalmap_texture", rep.distribution.sequence(normal_files))
return shader.node
rep.randomizer.register(randomize_texture)
# Create the render product and Writer
render_product = rep.create.render_product(camera, (1920, 1080))
writer = rep.WriterRegistry.get("BasicWriter")
writer.initialize(output_dir="_random_textures", rgb=True)
writer.attach([render_product])
# In the created graph, this is what executes for N number of frames
with rep.trigger.on_frame(num_frames=10, rt_subframes=50):
rep.randomizer.randomize_texture(pbr_shader)
# Preview the scene by executing the graph once, without writing.
rep.orchestrator.preview()
1 Like
Wonderful, thank you so much @pcallender! This workaround worked for me.
@pcallender Can we have this example script using omniverse:// links instead of the C:/ reference, so we can run it and see the results without having to setup the local environment? Would be nice to play around with this.
@pcallender This is great! thank you!
But how do we actually randomly choose between the different possible settings? for example, this is my code:
import omni.replicator.core as rep
shader_path = "/Environment/sky/Looks/SkyMaterial"
sunstudy_hours_dict = {
'5': {'TimeOfDay': 5, 'SHA': -103.6, 'Azimuth': -74.39, 'Elevation': -2.089, 'SunColor': (0.25, 0.013, 0.0)},
'6': {'TimeOfDay': 6, 'SHA': -90.302, 'Azimuth': -84.7575, 'Elevation': 6.05861, 'SunColor': (0.40997, 0.19664, 0.064)},
'8': {'TimeOfDay': 8, 'SHA': -59.9, 'Azimuth': -109.3115, 'Elevation': 24.6868, 'SunColor': (1, 0.98, 0.95)},
'10': {'TimeOfDay': 10, 'SHA': -30.14442, 'Azimuth': -139.52245, 'Elevation': 39.99774, 'SunColor': (1, 0.98, 0.95)},
'12': {'TimeOfDay': 12, 'SHA': -0.30232, 'Azimuth': -179.56448, 'Elevation': 46.57312, 'SunColor': (1, .98, .95)},
'14': {'TimeOfDay': 14, 'SHA': 29.5393, 'Azimuth': 140.4355, 'Elevation': 39.9977, 'SunColor': (1, .98, .95)},
'16': {'TimeOfDay': 16, 'SHA': 59.9, 'Azimuth': 109.426, 'Elevation': 24.6868, 'SunColor': (1, .98, .95)},
'18': {'TimeOfDay': 18, 'SHA': 89.67, 'Azimuth': 85.2266, 'Elevation': 6.43418, 'SunColor': (0.41913, 0.20716, 0.06765)},
'18.75': {'TimeOfDay': 18.75, 'SHA': 101.066, 'Azimuth': 76.83555, 'Elevation': -0.55645, 'SunColor': (0.25, 0.013, 0.0)},
'19': {'TimeOfDay': 19, 'SHA': 104.85558, 'Azimuth': 73.40426, 'Elevation': -2.83741, 'SunColor': (0.25, 0.013, 0.0)},
'21': {'TimeOfDay': 21, 'SHA': 135.17, 'Azimuth': 47.67297, 'Elevation': -19.21466, 'SunColor': (0.25, 0.013, 0.0)},
'23': {'TimeOfDay': 23, 'SHA': 165.48, 'Azimuth': 16.528, 'Elevation': -29.27166, 'SunColor': (0.25, 0.013, 0.0)}
}
def randomize_sunstudy_time():
hour = random.choice(list(sunstudy_hours_dict.keys()))
hour_attributes = sunstudy_hours_dict[hour]
with rep.get.prims(shader_path):
rep.modify.attribute("inputs:TimeOfDay", hour_attributes['TimeOfDay'])
rep.modify.attribute("inputs:SHA", hour_attributes['SHA'])
rep.modify.attribute("inputs:Azimuth", hour_attributes['Azimuth'])
rep.modify.attribute("inputs:Elevation", hour_attributes['Elevation'])
rep.modify.attribute("inputs:SunColor", hour_attributes['SunColor'])
how can I randomly choose an item from the dictionary and use it? random.choice will only run once since it does not belong to replicator and rep.core.distribution.choice returns a replicatoritem which I dont think I can use?
any input on this would be super helpful. Thank you.
1 Like
You can also set the sun position based on time of day:
# Load the sky
rep.create.from_usd(usd="https://omniverse-content-production.s3.us-west-2.amazonaws.com/Environments/2023_1/DomeLights/Dynamic/CumulusLight.usd")
# Select the objects in the stage
sky = rep.get.prim_at_path(path="/Replicator/Ref_Xform/Ref/Looks/SkyMaterial/Shader")
# Set sun position and time of day
with sky:
rep.modify.attribute(name="inputs:SunPositionFromTOD",value=True)
rep.modify.attribute(name="inputs:TimeOfDay",value=rep.distribution.uniform(lower=6.0,upper=19.5))
1 Like