Replicator's "Projection Material" no longer works in Isaac Sim 4.0.0 (Replicator 1.11.8)

Hello.
@pcallender
@dennis.lynch

I have noticed that in Isaac Sim 4.0.0 (Replicator 1.11.8) that the Replicator Materials, such as “Project PBR Material”, “Project Mask”, “Project Normals” are no longer shown under the “Replicator” section when I open the Material Graph tab.

Actually, the “Replicator” section that used to exist in the previous Replicator version (or Isaac Sim 2023.1.1) no longer exists when I open the Material Graph tab.

However, in the previous Replicator version(1.10.10) in Isaac Sim 2023.1.1, and according to the Replicator documentation, there should be a “Replicator” section when I open the Material Graphs tab as shown below:
https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator/replicator_materials.html#replicator-materials

Also, when I run the following “rep.modify.projection_material” sample code in the Replicator API document, “ProjectPBRMaterial” is not created as it should.

import omni.replicator.core as rep
import os
torus = rep.create.torus()
cube = rep.create.cube(position=(50, 100, 0), rotation=(0, 0, 90), scale=(0.2, 0.2, 0.2))
sem = [('class', 'shape')]
with torus:
    projection = rep.create.projection_material(cube, sem)
with projection:
    rep.modify.projection_material(diffuse=os.path.join(rep.example.TEXTURES_DIR, "smiley_albedo.png"))

Instead of the proper ProjectPBRMaterial, I am given this weird shader with just the diffuse_texture parameter as shown in the Material Graph and the error message, which I copy pasted below.

2024-07-18 06:18:06  [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
2024-07-18 06:18:06  [Error] [carb] [Plugin: omni.sensors.nv.lidar.ext.plugin] Dependency: [omni::sensors::lidar::IGenericModelOutputIOFactory v0.1] failed to be resolved.
2024-07-18 06:18:07  [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
2024-07-18 06:18:25  [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated key:'ALT + S (On Press)'
2024-07-18 06:18:25  [Warning] [omni.kit.hotkeys.core.registry]   -- Global [omni.kit.window.script_editor] omni.kit.window.script_editor::Save.ALT + S
2024-07-18 06:18:25  [Warning] [omni.kit.hotkeys.core.registry]   -- Global [omni.anim.curve.core-1.1.13] omni.anim.curve.core-1.1.13::add_key.ALT + S
2024-07-18 06:18:25  [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.window.script_editor] omni.kit.window.script_editor::Save.ALT + S, error code: Duplicated key definition
2024-07-18 06:18:45  [Warning] [omni.fabric.plugin] No source has valid data array=0000020AB4E8D638 usdValid=0 cpuValid=0
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(38,10): C120 could not find module '::replicator_definitions' in module path
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(146,22): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(147,19): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(148,24): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(151,17): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(169,45): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(169,126): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ab2e99ed0(173,55): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::Z73file_3A::Z16C_3A::users::admin::appdata::local::ov::pkg::Z34isaac_2Dsim_2D4_2E0_2E0::extscache::ZB3omni_2Ereplicator_2Ecore_2D1_2E11_2E8_2B106_2E0_2E0_2Ewx64_2Er_2Ecp310::mdl::project_pbr_material
2024-07-18 06:18:45  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::Z73file_3A::Z16C_3A::users::admin::appdata::local::ov::pkg::Z34isaac_2Dsim_2D4_2E0_2E0::extscache::ZB3omni_2Ereplicator_2Ecore_2D1_2E11_2E8_2B106_2E0_2E0_2Ewx64_2Er_2Ecp310::mdl::project_pbr_material
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(38,10): C120 could not find module '::replicator_definitions' in module path
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(146,22): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(147,19): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(148,24): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(151,17): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(169,45): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(169,126): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad32ad430(173,55): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:18:45  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::Z73file_3A::Z16C_3A::users::admin::appdata::local::ov::pkg::Z34isaac_2Dsim_2D4_2E0_2E0::extscache::ZB3omni_2Ereplicator_2Ecore_2D1_2E11_2E8_2B106_2E0_2E0_2Ewx64_2Er_2Ecp310::mdl::project_pbr_material
2024-07-18 06:18:45  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::Z73file_3A::Z16C_3A::users::admin::appdata::local::ov::pkg::Z34isaac_2Dsim_2D4_2E0_2E0::extscache::ZB3omni_2Ereplicator_2Ecore_2D1_2E11_2E8_2B106_2E0_2E0_2Ewx64_2Er_2Ecp310::mdl::project_pbr_material
2024-07-18 06:18:45  [Error] [omni.hydra] Failed to create MDL shade node for prim '/World/Looks/ProjectPBRMaterial/Shader'. createMdlModule failed.
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(38,10): C120 could not find module '::replicator_definitions' in module path
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(146,22): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(147,19): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(148,24): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(151,17): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(169,45): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(169,126): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: file:/C:/users/admin/appdata/local/ov/pkg/isaac-sim-4.0.0/extscache/omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310/mdl/project_pbr_material.mdl?watch=0000020ad34769c0(173,55): C108 'replicator_definitions' is not a package or module name
2024-07-18 06:19:52  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::Z73file_3A::Z16C_3A::users::admin::appdata::local::ov::pkg::Z34isaac_2Dsim_2D4_2E0_2E0::extscache::ZB3omni_2Ereplicator_2Ecore_2D1_2E11_2E8_2B106_2E0_2E0_2Ewx64_2Er_2Ecp310::mdl::project_pbr_material
2024-07-18 06:19:52  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::Z73file_3A::Z16C_3A::users::admin::appdata::local::ov::pkg::Z34isaac_2Dsim_2D4_2E0_2E0::extscache::ZB3omni_2Ereplicator_2Ecore_2D1_2E11_2E8_2B106_2E0_2E0_2Ewx64_2Er_2Ecp310::mdl::project_pbr_material
2024-07-18 06:19:52  [Warning] [omni.kit.material.library.material_library] process_ov_neuraylib: failed to get module from C:\users\admin\appdata\local\ov\pkg\isaac-sim-4.0.0\extscache\omni.replicator.core-1.11.8+106.0.0.wx64.r.cp310\mdl\project_pbr_material.mdl
2024-07-18 06:19:56  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: C120 could not find module '::infinite_pbr_material' in module path
2024-07-18 06:19:56  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::infinite_pbr_material
2024-07-18 06:19:56  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::infinite_pbr_material
2024-07-18 06:19:56  [Warning] [omni.kit.window.material_graph.shader_registry.mdl_registry] Pymdl could not fetch module: infinite_pbr_material.mdl from database
2024-07-18 06:19:56  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: C120 could not find module '::project_pbr_material' in module path
2024-07-18 06:19:56  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::project_pbr_material
2024-07-18 06:19:56  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::project_pbr_material
2024-07-18 06:19:56  [Warning] [omni.kit.window.material_graph.shader_registry.mdl_registry] Pymdl could not fetch module: project_pbr_material.mdl from database
2024-07-18 06:19:56  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: C120 could not find module '::replicator_definitions' in module path
2024-07-18 06:19:56  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::replicator_definitions
2024-07-18 06:19:56  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::replicator_definitions
2024-07-18 06:19:56  [Warning] [omni.kit.window.material_graph.shader_registry.mdl_registry] Pymdl could not fetch module: replicator_definitions.mdl from database

Lastly, I have also noticed that the replicator script and the “custom projection material” that used to work just fine in the previous Isaac Sim (2023.1.1) does not work in the newer version of Isaac Sim (4.0.0)(Replicator 1.11.8).

When I load a USD file with a “custom projection material” that was made with Replicator Materials (by connecting theProject Normal, Project Mask helper nodes in Material Graph) in the previous version of Isaac Sim (2023.1.1), and run the replicator script, I get the following errors on the console tab instead of the defects being projected on to the target mesh.

(I have blurred out the viewport for confidentiality)

I have also attached the log file for your reference.
kit_20240717_175303.log (1.1 MB)

So my questions are as follows:

As shown in this post, Replicator’s “projection material” feature no longer works in the latest updates of Isaac Sim and Replicator.

If it has been modified, could you direct me to any resources or documentation that explain the new changes regarding Replicator’s “Projection Materials”?

Or, will this feature be no longer available in future updates of Isaac Sim/Replicator?

I particularly found the Replicator’s “projection material” method extremely useful for generating defect images and really hope that is not the case.

Your expertise on this would be greatly appreciated.

Thanks!

i think the problem has got to do with the replicator’s mdl files (infinite_pbr_material.mdl, project_pbr_material.mdl, replicator_definitions.mdl).

When I open the Material Graph tab on Isaac Sim 4.0.0, I get the following warning/error msgs.

Any help on how to fix this would be much appreciated!

Thanks!

log file: kit_20240722_141959.log (1.1 MB)

2024-07-22 05:20:20  [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
2024-07-22 05:20:20  [Error] [carb] [Plugin: omni.sensors.nv.lidar.ext.plugin] Dependency: [omni::sensors::lidar::IGenericModelOutputIOFactory v0.1] failed to be resolved.
2024-07-22 05:20:21  [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
2024-07-22 05:40:53  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: C120 could not find module '::infinite_pbr_material' in module path
2024-07-22 05:40:53  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::infinite_pbr_material
2024-07-22 05:40:53  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::infinite_pbr_material
2024-07-22 05:40:53  [Warning] [omni.kit.window.material_graph.shader_registry.mdl_registry] Pymdl could not fetch module: infinite_pbr_material.mdl from database
2024-07-22 05:40:53  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: C120 could not find module '::project_pbr_material' in module path
2024-07-22 05:40:53  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::project_pbr_material
2024-07-22 05:40:53  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::project_pbr_material
2024-07-22 05:40:53  [Warning] [omni.kit.window.material_graph.shader_registry.mdl_registry] Pymdl could not fetch module: project_pbr_material.mdl from database
2024-07-22 05:40:53  [Error] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp error: C120 could not find module '::replicator_definitions' in module path
2024-07-22 05:40:53  [Warning] [rtx.neuraylib.plugin] Loading MdlModule to DB (rtx_scope0) failed: ::replicator_definitions
2024-07-22 05:40:53  [Warning] [rtx.neuraylib.plugin] Loading transaction committed (this thread). MdlModule is NOT in the DB (rtx_scope0): ::replicator_definitions
2024-07-22 05:40:53  [Warning] [omni.kit.window.material_graph.shader_registry.mdl_registry] Pymdl could not fetch module: replicator_definitions.mdl from database

There seems to be an issue with the Material Graph editor at the moment. It’s not searching the “Additional User Paths” and searching the “Custom Paths”. The Replicator extension is attempting to write the mdl search path to “Custom Paths”, but it’s being overwritten by the value in "C:\Users\<User>\Documents\Kit\shared\material.config.toml". So, as a workaround for now, the path to the mdl folder can be added as a path to that file.

To easily get the path to the Replicator mdl folder, open the extension window, select the “omni.replicator.core” extension and click the folder icon in the replicator extension description:

Copy the path to the “mdl” folder in this location and paste it into the “local” array of the “searchPaths” section of the “material.config.toml” file. It should look something like this:

[searchPaths]
local = [
  "C:/Users/<User>/AppData/Local/ov/pkg/<IsaacSim version>/extscache/omni.replicator.core-1.11.11+106.0.1.wx64.r.cp310/mdl",
]
custom = []

(Replace <User> and <IsaacSim version> with appropriate values)

Save the file, close IsaacSim and reopen it and the Material Graph window and it should populate the Replicator materials.

You can also do this via the User Interface

Edit->Preferences->Material->Custom Paths

1 Like

@dennis.lynch @ecameracci
problem solved.
thanks for the help!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.