Hello all,
I am currently using NVIDIA Isaac Sim 5.1.0 to generate synthetic data for defect detection on aerospace blades.
I have a blade .stl model and a set of defect maps (albedo, normal, roughness, mask). My initial approach has been to use a plane (mesh) with the same material as the blade and place it on top of the surface to simulate defects. However, due to the complex curved geometry of the blade, it is very difficult to align the plane properly. This also makes it hard to scale or randomize defect placement efficiently.
In several tutorials, I’ve seen defect generation done using decal projection. Unfortunately, I cannot find any decal functionality (e.g., Create → Rendering → Decal) in my current Isaac Sim version.
My goal is to:
-
Apply different types of defects (scratches, dents, etc.)
-
Randomize their position, orientation, and size
-
Generate synthetic datasets at scale
Given this, what would be the recommended workflow in Isaac Sim 5.1?
Specifically:
-
Is there a supported way to use decals or projection-based defects?
-
If not, is material-based blending (e.g., OmniSurface with masks) the preferred approach?
-
Are there any Replicator-based examples or extensions that handle this cleanly?
Any guidance or best practices would be greatly appreciated.
Hello guys,
I would appreciate any help on this issue. Thanks in advance.
Best Regards,
Nikhil Masinete
Hi Nikhil, The Create → Rendering → Decal menu is an Omniverse Create feature and isn’t exposed in Isaac Sim’s UI. However, Isaac Sim 5.1.0 ships with omni.replicator.core which has two built-in approaches for exactly this - projecting defect textures onto curved mesh surfaces with randomization support. Both are available via Replicator’s Python API. Approach 1: Projection Material (rep.create.projection_material + rep.modify.projection_material ) Uses a custom MDL shader (ProjectPBRMaterial) to project your defect textures onto the blade surface. You place a “proxy” prim that acts like a projector — its position, rotation, and scale control where and how the defect appears. Replicator creates a thin shell mesh around your blade and the projection is handled entirely in the shader, so it works well on curved geometry without alignment issues.
- Supports diffuse, normal, roughness, metallic textures - Texture group randomization via rep.utils.get_files_group and rep.modify.projection_material - Position/rotation/scale randomization via rep.modify.pose on the proxy prim - Semantic labels for segmentation annotation
Approach 2: Mesh Decal (rep.create.mesh_decal) Uses GPU-accelerated geometry clipping (Warp) to extract an actual mesh patch from your blade within a clip volume, then applies your defect textures to the extracted geometry. The decal mesh conforms exactly to the blade’s surface curvature. - Supports diffuse, normal, roughness, metallic, and opacity textures - Opacity/mask texture is important here to define the defect shape (without it you get a solid rectangle)
- Position, rotation, scale, and texture all accept Replicator distributions for randomization - Semantic labels for segmentation annotation - Added in Replicator 1.11.0 Which to start with: For your use case - randomized scratches/dents on curved aerospace blades at scale - I’d recommend starting with Projection Material. It’s shader-based so it handles curved surfaces more naturally and is faster for large-scale generation. The Mesh Decal approach is better suited when you need each defect as a distinct geometry prim (e.g., for instance-level segmentation) or need multiple overlapping defects on the same surface.
STL note: STL files have no UV or material data. After importing (via File → Import or the omni.kit.converter.cad extension), you’ll need to assign a base material to the blade before applying either approach.
Relevant API docs: - omni.replicator.core API reference (https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator/api.html) - rep.create.projection_material and rep.modify.projection_material for Approach 1 - rep.create.mesh_decal for Approach 2 - rep.utils.get_files_group for organizing texture sets with suffix conventions (e.g., _D, _N, _R, _O)
Hello,
I really appreciate your time for replying to my question!
I have both of your approaches and was not able to succeed with my goal.
Issue:
I am using a cube as a proxy_prim. While I can see the cube intersecting the mesh in the viewport, the textures (diffuse and opacity) never appear on the target mesh. When I try to use rep.modify.attribute, I get errors stating that inputs:diffuse_texture cannot be found on the Cube_Xform.
So my questions are the following.
Questions:
-
In this version of Replicator, what are the exact internal attribute names for the Projection node’s diffuse and opacity maps?
-
Why would Replicator attempt to write attributes to the proxy_prim (Cube_Xform) instead of the Projection prim itself?
-
Is there a specific way to force the “Hydra” texture plugin to stay active during an orchestrator.run() loop?
This is how my stage does look today. I kindly request to also let me know if there is a possibility to enable decal funtionality in UI. alternatively, if there is an another way to solve this problem. ?
Hi Nikhil, Thanks for confirming. The issue is that rep.modify.attribute(“inputs:di ffuse_texture”) targets the proxy prim (your Cube_Xform), which has no texture attributes. The texture inputs actually live on the ProjectPBRMaterial shader that’s bound to an internal shell mesh Replicator creates under your blade. You should never need to touch those attributes directly - use rep.modify.projection_material( ) instead.
To answer your three questions:
1. The internal attribute names are inputs:diffuse_texture, inputs:normalmap_texture, inputs:reflectionroughness_text ure, and inputs:metallic_texture - but they live on the Shader prim under /Looks/ProjectPBRMaterial/Shader, not on the proxy. You don’t need to access them manually. 2. Replicator isn’t writing to Cube_Xform - the error is telling you those attributes don’t exist there. The proxy only holds primvars:projection_prim (a link to the projection shell mesh). rep.modify.projection_material( ) handles the indirection internally. 3. You don’t need to force the Hydra texture plugin - rep.orchestrator.run() (or step_async()) triggers full render frames and the shader updates are picked up automatically through USD.
Important STL note: After importing your .stl, you must assign a base material (e.g., OmniPBR) to the blade mesh before creating the projection. STL has no UV/material data, but the projection system doesn’t need UVs - the ProjectPBRMaterial MDL shader projects textures mathematically using the proxy prim’s transform. Re: decals in the UI - the Create → Rendering → Decal menu is an Omniverse Create feature, not exposed in Isaac Sim. But rep.create.projection_material achieves the same result programmatically and is better for automated SDG pipelines anyway.
Closing the topic due to inactivity.
To summarize,
1. The proxy prim is just a transform controller - it has no material/texture attributes 2. The texture inputs live on an internal Shader prim (/Looks/ProjectPBRMaterial/Shader) that Replicator creates automatically
3. The correct API is rep.modify.projection_material() which handles the routing internally
If you have any additional questions, please feel free to submit them!
