Bounding boxes are not rendered in generated images

Omniverse Synthetic Data Generation (SDG)
1

Hello,

I am using replicator for synthetic data generation using Isaac Sim v4.2. I am having issues in generating bounding boxes for the labelled prims.

In the writer, I have enabled rgb, semantic segmentation, bounding box 3d, bounding box 2d loose and bounding box 2d tight. My assumption is the bounding boxes are automatically generated if enabled in writer. For example, when I enabled semantic segmentation then respective images are automatically generated. Below is the code snippet I have used:

    # Initialize and attach writer
    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize(
        output_dir="_output",
        rgb=True,
        semantic_segmentation=True,
        bounding_box_3d=True,
        bounding_box_2d_loose=True,
        bounding_box_2d_tight=True)
    writer.attach([render_product])
    writer.attach([render_product_static])

In addition to this, I have also added annotator registry as below:

camera_static = rep.create.camera(name="static_camera", position=(16, -27.5, 6.2), rotation=(60, 0, 90))
render_product_static = rep.create.render_product(camera_static, (1024, 1024))

bbox_2d_tight = rep.AnnotatorRegistry.get_annotator("bounding_box_2d_tight")
bbox_2d_tight.attach(render_product_static)

bbox_2d_loose = rep.AnnotatorRegistry.get_annotator("bounding_box_2d_loose")
bbox_2d_loose.attach(render_product_static)

bbox_3d = rep.AnnotatorRegistry.get_annotator("bounding_box_3d")
bbox_3d.attach(render_product_static)

But this too didn’t worked.

Being said, if I use Visualizer tool I can see the bounding boxes as shown below

image
image1879×715 90.5 KB

While looking for similar issues, I came across a thread that has bounding box issues. I tried the code snippet of that thread which is confirmed as working by other commentators. However, that snippet is also not generating any bounding boxes.

This appears to be an issue with Isaac Sim v4.2.

How can I solve this issue?
Thanks in advance.

2

@ahaidu Can you please guide me with the bounding box issue?

I would like to know whether bounding boxes are automatically generated in BasicWriter? Do we need to write any additional scripts for the same?

3

Hi there,

you can use either the annotators and access then using get_data, or direclty with the basic writer by setting, as you did bounding_box_2d_tight=True. You should however attach both render products at once: writer.attach([render_product, render_product_static])

Here is an example of using the writer and the annotators (get_data()):