About the Synthetic Data Generation (SDG) category - Omniverse Replicator

Omniverse Synthetic Data Generation (SDG)
1

Omniverse replicator is in beta and ready for experimentation. To use, download the Omniverse launcher and install the Code App.

Important links:

For background: What Is Synthetic Data? | NVIDIA Blogs

2

Hi there,
Thank you for opening this forum.

We are interested in building a synthetic data set for 3D deep learning.

Last year we built a digital twin using Omniverse Create, however, we got a bit stuck trying to replicate the specific sensors we use in real life.

In particular, we are using the RealSense L515 lidar sensor, and wanted to replicate it in Isaac Sim so that the synthetic point clouds generated match the resolution of our real data.

We want to use this data to train a 3D deep learning model for object detection.

Is there a library of off-the-shelve sensors we can use in Isaac sim? Perhaps documentation about them? Or a how-to guide to create your own sensors which can output a KITTI data set we can use for training?

Thanks!

1 Like
3

Hello,
We recommend you check out:
Replicator — Omniverse Extensions documentation (nvidia.com).
Omniverse Replicator is now accessible through Code, but in the next release of isaac sim, it will be accessible that way too.

We don’t provide off the shelf specific cameras but we give you parameters that you can tweak to match the camera you want. Omni Replicator — omni_replicator 101.1.0 documentation (nvidia.com)

With regards to specific depth data, Replicator comes with a depth annotator, which lets you write depth.
These are all the kinds data you can write.

writer.initialize(
    output_dir="out_dir_test_visualization",
    rgb=True,
    bounding_box_2d_tight=True,
    bounding_box_2d_loose=True,
    semantic_segmentation=True,
    instance_segmentation=True,
    distance_to_camera=True,
    distance_to_image_plane=True,
    bounding_box_3d=True,
    occlusion=True,
    normals=True,
)
1 Like