"I want to implement decal effects in Isaac Sim—is this achievable?
And there isn’t much explanation, only a short section. I also executed Python code, which doesn’t seem to be suitable for Isaac Sim.
from pxr import Usd, UsdGeom
from pathlib import Path
import omni.usd
import omni.replicator.core as rep
# Axis and unitscale can also be set optionally
#rep.settings.set_stage_up_axis("Z")
#rep.settings.set_stage_meters_per_unit(1.0)
#Create Light
distance_light = rep.create.light(rotation=(400,-23,-94), intensity=10000, light_type="distant")
stage = omni.usd.get_context().get_stage()
plane = rep.create.plane(position=(50, 100, 0), rotation=(0, 45, 10))
cube = rep.create.cube(visible=False, semantics=[('class', 'cube')], position=(50, 100, 0), rotation=(0, 0, 90), scale=(0.2, 0.2, 0.2))
sem = [('class', 'shape')]
# Randomizer for scattering
def get_shapes():
shapes = rep.get.prims(semantics=[('class', 'cube')])
with shapes:
rep.randomizer.scatter_2d(plane)
return shapes.node
rep.randomizer.register(get_shapes)
# Create the projection with the plane as the target
with plane:
proj1 = rep.create.projection_material(cube, sem)
# Modify the cube position, and update the projection
with rep.trigger.on_frame(num_frames=30):
rep.randomizer.get_shapes()
with proj1:
rep.modify.projection_material(diffuse=Path(rep.example.TEXTURES_DIR).joinpath("smiley_albedo.png").as_posix())
I’ve noticed some official documentation methods, but they don’t seem to apply to Isaac Sim. I’m wondering: Can Isaac Sim actually support this kind of effect?"
Considering this is such a fundamental feature in any engine…
Let me reach out to the internal team for your question
@zhengwang
Thank you for your reply. I tried using “USD Composer” and followed the same method to create decals as per the tutorial. I also attempted using code, but unfortunately, it still failed. The shader started throwing errors.
This official documentation seems outdated—it’s from a very early version and is no longer applicable to the current Isaac Sim or USD Composer. It should be deprecated.
However, I’d like to know if there are alternative methods to achieve decal projection effects. I look forward to your assistance.
@zhengwang
I’m very sorry to disturb you, and I hope it won’t cause any inconvenience to you.
I would like to know if there are any suggestions or progress regarding Decal?
Hi @Julie_A No worries. It is part of my job. Let me follow up with the team again. Thanks for your patience.
There shouldn’t be anything preventing this from being used within Isaac Sim. Its what we recommend by default as the Kit app for use with Replicator, and what we mostly use ourselves.
There’s a few caveats to note specifically for Isaac Sim workflows. Replicator scripts generate Omnigraphs that actually do the execution, where Isaac Sim workflows are mostly pythonic.
Take a quick look at this example where replicator is used in conjunction.
There’s also a new “functional” feature in replicator that’s purely pythonic. It might be simpler, but I need to check a few things. I’ll take a look at the decal example and see if there’s any issues and circle back.
@pcallender
Thank you very much for your response.
Regarding Replicator, I’m very familiar with it as I’ve been using it in “Standalone” mode for data generation. I also have some understanding of Omnigraph.
According to the documentation at Replicator Materials — Omniverse Extensions, I’ve tried creating Decals using Omnigraph, and also attempted the Python approach, but neither seems to work properly and both return errors. Furthermore, I haven’t found any other documentation about Decal creation.
It appears there isn’t a well-documented method for creating Decals, and the official Isaac Sim documentation doesn’t provide any guidance on Decal implementation.
I would really like to know if there exists a specific, reliable method to properly create and generate Decals in Isaac Sim.
I greatly appreciate your assistance and look forward to your reply.
We have a specific MDL for decals already built into Kit. Add “simple_sticker”
@Richard3D
Thank you for your help. Does Isaac Sim 4.5 have this feature? Or is there any documentation available to learn and create it?
Try this modified script (I tested with isaacsim 5.0, but it ought to work in 4.5 also). There were some incorrect rotations in there that resulted in the decal being rotated away from the plane.
As a note. This particular feature in replicator is intended for situations where you need a specific surface feature where it can also be segmented for classification. Its not intended as a general purpose decal. Under the hood it duplicates the mesh applies a special projection material onto it, and does some projection math to match the orientation of the local negative X of the cube.
- Open a new scene in Isaac Sim
- Press the “Run” button at the bottom of the script editor
- In the menu under Tools>Replicator>Step select this a few times to see the smiley face move around on the surface.
from pxr import Usd, UsdGeom
from pathlib import Path
import omni.usd
import omni.replicator.core as rep
# Axis and unitscale can also be set optionally
#rep.settings.set_stage_up_axis("Z")
#rep.settings.set_stage_meters_per_unit(1.0)
#Create Light
distance_light = rep.create.light(rotation=(400,-23,-94), intensity=10000, light_type="distant")
stage = omni.usd.get_context().get_stage()
plane = rep.create.plane(position=(0, 0, 0), rotation=(0, 0, 0))
cube = rep.create.cube(visible=False, semantics=[('class', 'cube')], position=(0, 0, 0), rotation=(0, -90, 90), scale=(0.2, 0.2, 0.2))
sem = [('class', 'shape')]
# Randomizer for scattering
def get_shapes():
shapes = rep.get.prims(semantics=[('class', 'cube')])
with shapes:
rep.randomizer.scatter_2d(plane)
return shapes.node
rep.randomizer.register(get_shapes)
# Create the projection with the plane as the target
with plane:
proj1 = rep.create.projection_material(cube, sem)
# Modify the cube position, and update the projection
with rep.trigger.on_frame(num_frames=30):
rep.randomizer.get_shapes()
with proj1:
rep.modify.projection_material(diffuse=Path(rep.example.TEXTURES_DIR).joinpath("smiley_albedo.png").as_posix())
rep.orchestrator.preview()
1 Like
@pcallender
Thank you very much for your help. I copied the code and gave it a try. I’m using Isaac Sim 4.50.
It was created successfully without any errors, but I feel there are several issues.
It doesn’t resemble a projection (like a movie being played or a projector’s projection). Instead, it looks more like an image pasted onto a cube, failing to achieve the actual projection effect. I’ve attached a screenshot of the result. It seems more like a transparent texture applied to a “plane” rather than a true projection effect.
It seems to only project onto itself, while other objects remain completely unaffected—which defeats the purpose of using a “Decal.”
“Decal” is a relatively basic yet highly frequently used feature. It can simulate various sensors and accomplish many tasks. I hope to get some suggestions and assistance on this.
“add_simple_sticker” is a layered material node in the graph editor. It takes a material as it’s base input. And has controls for a sticker with IOR, reflectivity, roughness and normals.
@Richard3D @pcallender
Very sorry, I tried this method. I attempted to use “add_simple_sticker,” but it doesn’t seem to have any effect. There’s still no projection displayed. Could I be doing something wrong, or are there any other solutions? Because I found some related documentation, but it’s extremely minimal—almost no documentation on how to do projection decals.
Can you provide me with an “MDL” or “USD” for my research and study? I really look forward to your assistance.
No, that is not correct. First of all, do not use Displacement. Second, the Simple sticker requires a “base” material and a “mask” material, and then a “color”.
@Richard3D
Thank you for your reply. I tried it immediately.
-
The blue part is the “base material,” and it has a black-and-white checkerboard texture connected.
-
The red part is the “mask,” which is connected to a.png.
3.The “sticker color” is adjusted to red.
I’ve uploaded my USD file.
sticker.zip (50.8 KB)
Result: First, it didn’t project (like a projector). Instead, it seemed to be projected onto the Cube, rather than being emitted from the Cube. This is very different from what I intended. I want to know if there’s a way to achieve the effect I want, like the projection shown in the image below.
Thank you once again for your assistance. I look forward to your reply.
I am also a user looking for decal effect like this.
A workaround that may work for you is to to use projector mode in the rect light in isaac sim. There is a setting which allows to link the projection texture and turn on the projection mode.
But that approach doesn’t work well for me. I am still eager to find solutions in MDL or customize shaders.
Looking forward to the reply. Thank you.
Ok lets take a step back and get your intent right.
“Instead, it seemed to be projected onto the Cube,” - ? Nothing is getting “projected” here. It is called a “sticker”. Like a sticker on your laptop.
A “projector” refers to a light source. Projecting a movie.
Are you looking for :
- A LIGHT SOURCE to project a pattern onto a scene (Like you are now showing in the above clip)
Example would be Batman’s Giant Spot Light in the sky
OR
- A TEXTURE SOURCE to be part of a textured material, which shows a decal
Example would be a Sticker on a bottle with a color logo, on a transparent background.
???
They are very different things.
Both are easy to do. But they require completely different approaches. You started with one thing, and then switched to another.
If you want to “emit” something from the cube, you just make it a light source. Just go to “emissive” in the material, and checkbox it on and, you have a light source. Then if you want to “mask” that light source, you just add a mask. So to create Batman’s search light, it is just a simple disc, made emissive, with a batman logo as the mask. That is it. Very simple.
