Defect Detection Extension: Projection Mapping Issue with Target Assets

I tried projection mapping on the target assets based on the link below. However, the result was unsuccessful.
Generate Synthetic Data — Omniverse Workflows latest documentation (nvidia.com)

The following error occurred.

2024-01-19 05:42:50  [Error] [omni.graph.core.plugin] /Replicator/SDGPipeline/OgnCreateProjectionMaterial: Assertion raised in compute - 
2024-01-19 05:42:50  [Error] [omni.graph.core.plugin] 	Error in 'pxrInternal_v0_22__pxrReserved__::UsdStage::_SetValueImpl' at line 6174 in file /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp : 'Empty typeName for </Projection/table_low_Cube_Xform.xformOp:translate>'
2024-01-19 05:42:50  [Error] [omni.graph.core.plugin]   File "/home/nvidia/.local/share/ov/pkg/deps/8021e278fce7781f5bb307a5b4b24ed4/extscache/omni.replicator.core-1.9.8+105.0.lx64.r.cp310/omni/replicator/core/ogn/python/_impl/nodes/OgnCreateProjectionMaterial.py", line 264, in compute
2024-01-19 05:42:50  [Error] [omni.graph.core.plugin]     projection_prim = _duplicate_target(target_prim, projection_path)
2024-01-19 05:42:50  [Error] [omni.graph.core.plugin]   File "/home/nvidia/.local/share/ov/pkg/deps/8021e278fce7781f5bb307a5b4b24ed4/extscache/omni.replicator.core-1.9.8+105.0.lx64.r.cp310/omni/replicator/core/ogn/python/_impl/nodes/OgnCreateProjectionMaterial.py", line 176, in _duplicate_target
2024-01-19 05:42:50  [Error] [omni.graph.core.plugin]     projection_prim.GetAttribute("xformOp:translate").Set(Gf.Vec3d(0, 0, 0))
2024-01-19 05:42:50  [Error] [omni.graph.core.plugin] 

The Hierarchy of ‘Projection’ is Not Configured in the Correct Position.

What can be done to achieve projection mapping correctly?

1 Like

I have conducted further investigation.

It is likely that the mesh for projection mapping is created by copying parameters from the USD property of the target mesh. The issue appears to be caused by the absence of necessary parameters during the copying process. What USD parameters should an asset have to be capable of projection mapping?


  • The mesh for projection copying the shape and position of the target mesh.
    → Parameters of the mesh set as the target are obtained through the get_attribution process and inherited by the projection mesh.
    → If essential parameters for creating the projection mesh are missing during its creation, it may not be copied correctly, or the hierarchy may not be established properly.

  • Errors are outputted for missing parameters such as xformOP:translate or xformOP:rotateXYZ.

  • Errors are outputted for parameters related to face vertices.

  • Parameters of assets capable of projection mapping:
    A plane created using rep.create.plane().

Car panel (acquired from demo samples).

Hi,experts.

I’d appreciate it if you could give me an update on the situation.

Hi,experts.

I’d appreciate it if you could give me an update on the situation.

Hi,experts.

I’d appreciate it if you could give me an update on the situation.

Have you ever solved this problem? I also encounter this problem when using this extension, I think maybe the usd files created by cad files have some error

I think your problem is related to my topic https://forums.developer.nvidia.com/t/bug-report-fix-ogncreateprojectionmaterial-fails-to-create-a-shell-around-non-simple-mesh/293594

After loading isaac->environments->Simple Room and running this script:

from pathlib import Path

import omni
import omni.replicator.core as rep
import omni.usd

stage = omni.usd.get_context().get_stage()

path = "/SimpleRoom/table_low_327/table_low"

cube = rep.create.cube(
    visible=True,
    semantics=[("class", "cube")],
    position=(0, 0, 0.3),
    rotation=(0, -90, 0),
    scale=(0.2, 0.2, 0.2),
)
sem = [("class", "shape")]


target = rep.get.prims(path_pattern=path)
with target:
    proj1 = rep.create.projection_material(cube, sem, offset_scale=0.0001)

# Modify the cube position, and update the projection
with proj1:
    rep.modify.projection_material(
        diffuse=Path(rep.example.TEXTURES_DIR).joinpath("smiley_albedo.png").as_posix()
    )

rep.orchestrator.run(1)

I see this in console and no projection on table asset:

2024-05-21 16:15:33 [507,922ms] [Error] [omni.graph.core.plugin] /Replicator/SDGPipeline/OgnCreateProjectionMaterial: [/Replicator/SDGPipeline] Assertion raised in compute - Index out of range.
  File "\documents\ov\pkg\isaac_sim-2023.1.1\extscache\omni.replicator.core-1.10.20+105.1.wx64.r.cp310\omni\replicator\core\ogn\python\_impl\nodes\OgnCreateProjectionMaterial.py", line 385, in compute
    _apply_vertex_offsets(projection_prim, offset_scale)
  File "\documents\ov\pkg\isaac_sim-2023.1.1\extscache\omni.replicator.core-1.10.20+105.1.wx64.r.cp310\omni\replicator\core\ogn\python\_impl\nodes\OgnCreateProjectionMaterial.py", line 218, in _apply_vertex_offsets
    vertex_normals[face_vertex_indices[x + count]] += normals[x + count]

But after applying patch to OgnCreateProjectionMaterial.py from my post and running script I see correct projection

Screenshot_1

Maybe this helps you too.

1 Like

@asek.s

Thank you for sharing the information!
By trying out the code in your post, I was able to project images onto the table asset in Simpleroom.
Thank you so much!

@matsumoto-kai
I am glad to hear this.

Could you please mark it as solution and upvote my original post.

I am hoping that someone from nvidia team members will react so they can fix OgnCreateProjectionMaterial.py globally. I could not find a way on how to post a bug report other than to this forum.

1 Like

Jumping in from asek.s’s other forum post with the issue and fix. I’ve submitted this to be looked at by the devs.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.