Getting semantic information on "invisible" assets in Isaac Sim 4.5

Omniverse Isaac Sim
,
1

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: Geforce RTX 4090
  • Driver Version: 570.133.20

Topic Description

Detailed Description

In my scenario I have a custom asset in Isaac Sim to which I added a little defect. E.g. I added a hole on a metal profile, but I did this by editing the file itself on Blender. Now I want to generate some synthetic data with this part, but it is important to me to have the semantics or a tight 2d bounding box around the defect I have. The way I thought of doing it is by creating a small Cube asset in Isaac and shaping it around my defect. This should give me the bbox that I need. However, from the camera that I am using, I do not want the small Cube to be visible. I only want its semantics.

TL;DR: Can I get the replicator to give me semantic segm/tight bbox of an “Invisible” asset in Isaac Sim. I tried everything, even assigning a material to my asset with opacity 0, but it doesn’t work. I also tried doing it through scripting and not the GUI.

Steps to Reproduce

  1. Add asset on stage.
  2. Make it invisible.
  3. Export (colorized) semantic segmentation or bbox 2D tight from the replicator.
  4. Asset does not show up.

Error Messages

Screenshots or Videos

Additional Information

What I’ve Tried

  1. Setting it to invisible.
  2. Changing Purpose to proxy or guide.
  3. Assigning a material to the asset and setting its opacity to 0.

Related Issues

Additional Context

3

We will consult with our team and get back to you.

4

This code is from Getting Started Scripts — Isaac Sim Documentation

            # Capture again with the cube hidden
            UsdGeom.Imageable(cube).MakeInvisible()
            await rep.orchestrator.step_async(delta_time=0.0)
            UsdGeom.Imageable(cube).MakeVisible()

Could you try a similar sequence of calls, but instead make the cube visible before capturing?

5

I am not sure what you mean by this. What should I try here?

6

You can try using MakeVisible() before capturing your data and then call MakeVisible() again after your capture if you’ve hidden it elsewhere, or use MakeInvisible() after capturing only if you want to keep the asset hidden for other purposes.