Starting synthetic data recorder through script

Omniverse Isaac Sim
1

Hello, I am trying to start the synthetic data recorder at certain events, such as when a vehicle reaches a certain location. I am trying to do this within a python Omniverse object behavior script. Essentially, I would like to hit the start button for the synthetic data recorder through python scripting. What modules and commands do I need to make this work?

3

Hi there,

you can take a look at this example, it does something similar by accessing annotator data:

The next Isaac SIm release (2023.1.0) will support custom trigger events on writers as well.

4

Hello, thank you for the response. I have tried to implement this code in the on_update() portion of the object behavior script, but it appears to write empty data sets. The cam object is created as an Xform, but when I try to print the rgb_annot.get_data(), it prints “[ ]”. Any help would be appreciated, the code is provided below.

def sdrecorder(self):

# Create a camera and render product to collect the data from
cam = rep.create.camera(position=(-3739, 277, -2097), look_at=(-2739, 138, -1637))
rp = rep.create.render_product(cam, (512, 512))

# Accesing the data directly from annotators
rgb_annot = rep.AnnotatorRegistry.get_annotator("rgb")
rgb_annot.attach([rp])

print(rgb_annot.get_data())
5

You will need to either put rep.orchestrator.step() or world.step(render=True) to feed data to the annotator.

6

I have been trying to do that, but there seems to be an issue. I have a USD file open when I am running the code and would like to use the world in that USD scene (if thats how ‘World’ type works). Thank you for your continued support. I have attached the error messages and code. I have tried using world.play(), world.render(), and tried with and without the step_sim=True condition to no avail (and have been referencing Beginner - Get world handle [Python]).

def sdrecorder(self):

world = World()
world.play()

# Create a camera and render product to collect the data from
#cam = get_existing_camera(existing_camera_path)
cam = rep.create.camera(position=(-3739, 277, -2097), look_at=(-2739, 138, -1637))
rp = rep.create.render_product(cam, (512, 512))

# Accesing the data directly from annotators
rgb_annot = rep.AnnotatorRegistry.get_annotator("rgb")
rgb_annot.attach([rp])

world.step(render=True, step_sim=True)
print(rgb_annot.get_data())

error_msgs
error_msgs3357×810 379 KB

7

I am also trying to do it with the replicator which seems easier. I have attached my efforts. At one point it created the correct directory but nothing was written into that folder.

def sdrecorder(self):
with rep.new_layer():
# Create a camera and render product to collect the data from
#cam = get_existing_camera(existing_camera_path)
cam = rep.create.camera(position=(-3739, 277, -2097), look_at=(-2739, 138, -1637))
rp = rep.create.render_product(cam, (512, 512))

# Accesing the data directly from annotators
    #rgb_annot = rep.AnnotatorRegistry.get_annotator("rgb")
    #rgb_annot.attach([rp])
    rep.orchestrator.step()

    writer = rep.WriterRegistry.get("BasicWriter")
    writer.initialize(output_dir=".../_output", rgb=True, bounding_box_2d_tight=True)
    writer.attach([rp])
8

Hi there,

for the first part you might need to call world.reset() after creating the world, to set up the simulation context.

For the second part, you should call orchestrator.step() after you create, initialize and attach a render product to your writer. Every step() call will trigger the data to be written by the writer.

Best,
Andrei