Closing the Sim2Real Gap with NVIDIA Isaac Sim and NVIDIA Isaac Replicator

Originally published at: https://developer.nvidia.com/blog/closing-the-sim2real-gap-with-nvidia-isaac-sim-and-nvidia-isaac-replicator/

NVIDIA Isaac Replicator, built on the Omniverse Replicator SDK, can help you develop a cost-effective and reliable workflow to train computer vision models using synthetic data.

Thank you for the very interesting article. I am amazed what can be done with Omniverse and Isaac Sim.

Being new to the two as well as USD, I wondered how these things can be accomplished. Would be happy if you can guide me in the right direction.

My thoughts are concerning the following:

  • How did you manage to place ceiling lights to the scene? Can they be somehow “attached” to the ceiling (so they would stay at the ceiling even when the ceiling is moved, i.e. changing the room height)?
  • How are the other props placed into the scene? Manually or with some randomizer/replicator? Can you check for collisions while placing these objects?
  • you seemed to solve the qr-code problem by randomizing the wall texture. What if I’d be interested in detecting objects mounted to the wall, like shelves or the like. Can props be attached and place on the wall easily as well?

Hope you can answer those questions and guide me towards more ressources. Looking forward what can already done with omniverse and Isaac Sim