Hello NVIDIA Community,
During the development encountered a bug with omni.replicator.core create.projection_material method.
Version isaac_sim-2023.1.1
Exception log:
2024-05-20 18:12:43 [264,651ms] [Error] [omni.graph.core.plugin] /Replicator/SDGPipeline/OgnCreateProjectionMaterial: [/Replicator/SDGPipeline] Assertion raised in compute - Index out of range.
File "\documents\ov\pkg\isaac_sim-2023.1.1\extscache\omni.replicator.core-1.10.20+105.1.wx64.r.cp310\omni\replicator\core\ogn\python\_impl\nodes\OgnCreateProjectionMaterial.py", line 323, in compute
_apply_vertex_offsets(projection_prim, offset_scale)
File "\documents\ov\pkg\isaac_sim-2023.1.1\extscache\omni.replicator.core-1.10.20+105.1.wx64.r.cp310\omni\replicator\core\ogn\python\_impl\nodes\OgnCreateProjectionMaterial.py", line 190, in _apply_vertex_offsets
vertex_normals[face_vertex_indices[x + count]] += normals[x + count]
By investigating further into code I have found that iterations in method ‘_apply_vertex_offsets’ doesn’t count for complex meshes. The bug arises due to an assumption in the _apply_vertex_offsets function that all vertices, normals, face vertex counts, and face vertex indices will align perfectly across all meshes. This assumption fails for complex meshes, leading to an “index out of range” error.
Some experiments showed that ‘_apply_vertex_offsets’ works for some meshes indeed, thus like replicator tutorials provided in documentation are working perfectly (replicator tutorials). Primitives like cylinders or basic shapes often have vertices and face data that align perfectly. However more intricate meshes, may have more complicated vertex and face arrangements. They might have non-manifold edges, more vertices, or varying face structures that can cause misalignment in the indices.
In order to demonstrate this I have included an example script for isaacsim, where you can uncomment VERSION 1 OR VERSION 2 before running the script and see that assertion error. Version 2 uses SM_Armchair.usd from omniverse-content-production default assets pack included with omniverse.
from pathlib import Path
import omni
import omni.replicator.core as rep
import omni.usd
stage = omni.usd.get_context().get_stage()
# Uncomment VERSION 1 OR VERSION 2 before RUN
# VERSION 1
# path = "/Cylinder"
# result, path = omni.kit.commands.execute("CreateMeshPrimCommand", prim_type="Cylinder")
# VERSION 2
# itemPrim = stage.DefinePrim("/object", "Xform")
# itemPrim.GetReferences().AddReference(
# assetPath="http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Office/Props/SM_Armchair.usd",
# primPath="",
# )
# path = "/object/SM_Armchair"
# Create Light
rep.create.plane(scale=(10, 10, 1))
rep.create.light(rotation=(0, 45, 0), intensity=3000, light_type="distant")
rep.create.light(position=(1, 0, 1), intensity=10000, scale=0.1, light_type="sphere")
cube = rep.create.cube(
visible=True,
semantics=[("class", "cube")],
position=(2, 0, 0.3),
rotation=(0, 0, 0),
scale=(0.2, 0.2, 0.2),
)
sem = [("class", "shape")]
target = rep.get.prims(path_pattern=path)
with target:
proj1 = rep.create.projection_material(cube, sem, offset_scale=0.0001)
# Modify the cube position, and update the projection
with proj1:
rep.modify.projection_material(
diffuse=Path(rep.example.TEXTURES_DIR).joinpath("smiley_albedo.png").as_posix()
)
rep.orchestrator.run(1)
Output with simple shape:
Output with bug:
Output with fixed ‘_apply_vertex_offsets’
The algorithm should be robust enough to handle varying complexities in mesh structures. This includes additional checks to ensure indices are valid before accessing elements.
And fix for replicator.core code
~/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.replicator.core-1.10.20+105.1.lx64.r.cp310/omni/replicator/core/ogn/python/_impl/nodes/OgnCreateProjectionMaterial.py
# original iteration
# for faceVertCount in face_vertex_counts:
# for x in range(faceVertCount):
# vertex_normals[face_vertex_indices[x + count]] += normals[x + count]
# vertex_normals_divcount[face_vertex_indices[x + count]] += 1
# count += faceVertCount
# FIX - Iterate through the faces, accumulating normals checking for valid ranges
for faceVertCount in face_vertex_counts:
for x in range(faceVertCount):
if (x + count) < len(face_vertex_indices):
vertex_index = face_vertex_indices[x + count]
if vertex_index < len(vertex_normals) and (x + count) < len(normals):
vertex_normals[vertex_index] += normals[x + count]
vertex_normals_divcount[vertex_index] += 1
count += faceVertCount
# original
# for i in range(len(vertex_normals)):
# vertex_normals[i] = vertex_normals[i] / vertex_normals_divcount[i]
# FIX - Normalize the accumulated vertex normals, check for 0 div
for i in range(len(vertex_normals)):
if vertex_normals_divcount[i] > 0:
vertex_normals[i] = vertex_normals[i] / vertex_normals_divcount[i]
Please check this solution and maybe include it in the next omniverse release.
