Offline data generation semantic labels not working for bounding boxes

[ian.oneill]

I’m trying to generate randomized visual and bounding box data. The usd files that I’m loading in have semantic labels, and when viewing the bounding boxes through the websocket gui, they are visible. However, when generating data and using the BasicWriter, the .json files for the bounding boxes are empty, even with multiple labeled objects visible in the rgb output.

[ian.oneill]

The attached images show the output, how the models that are being loaded in are labeled, and the bounding box working in gui.

[ian.oneill]

UPDATE: This problem was fixed by changing this code

1    #this function chooses the random objects to be instantiated in the frame and modifies their position, material, and rotation
2    def get_random_object(size):
3        usd_files = glob.glob("/path/to/**/*.usd")
4        print(usd_files)
5        get_objects = rep.randomizer.instantiate(usd_files, size=size, mode='point_instance')
6        with get_objects:
7            rep.randomizer.materials(mat_list)
8            rep.modify.pose(
9                position=rep.distribution.uniform((-40, -40, 0), (40, 40, 80)),
10                rotation=rep.distribution.uniform((-90, -180, 0), (90, 180, 0))
11            )
12        return get_objects.node
13    rep.randomizer.register(get_random_object)
14    
15    #call the randomization functions and make the frames
16    with rep.trigger.on_frame(num_frames=CONFIG["num_frames"]):
17        #calling random object function to get desired objects
18        rep.randomizer.get_random_object(10)
19        #randomizing background and lighting by calling dome light function
20        rep.randomizer.dome_lights()

to this

1#this function chooses the random objects to be instantiated in the frame and modifies their position, material, and rotation
2    def get_random_object(size):
3        usd_files = glob.glob("/path/to/**/*.usd")
4        random_objects = random.sample(usd_files, size)
5        print(random_objects)
6        objects_list = []
7        for each in random_objects:
8            objects_list.append(rep.create.from_usd(each))
9        print(objects_list)
11        
13        get_objects = rep.randomizer.instantiate(objects_list, size=size, mode='point_instance')
14        with rep.create.group(objects_list):
15            rep.randomizer.materials(mat_list)
16            rep.modify.pose(
17                position=rep.distribution.uniform((-20, -20, 0), (20, 20, 40)),
18                rotation=rep.distribution.uniform((-90, -180, 0), (90, 180, 0))
19            )
20        return get_objects.node
21    rep.randomizer.register(get_random_object)
22    
23    #call the randomization functions and make the frames
24    with rep.trigger.on_frame(num_frames=CONFIG["num_frames"]):
25        #calling random object function to get desired objects
26        rep.randomizer.get_random_object(3)
27        #randomizing background and lighting by calling dome light function
28        rep.randomizer.dome_lights()

It was necessary to run the rep.create.from_usd function in order to bring in the semantic information.

However, this has led to a new problem where it is not resampling the usd files on every frame. It will pick size usd files from the path, and just use those in every frame rather randomly repicking new ones, which is the desired behavior.

Is there a way to choose new .usd files every frame and bring them in with semantic information?

[nworkerxz9q8]

Hello, sorry for the delay. I am checking with a workaround to this. Will get back to you soon.

[dennis.lynch]

Hello, there are two different ways to achieve what you are looking for, but each have their own trade-offs.

Instantiate is the easiest, but won’t allow you to modify the chosen objects beyond their pose and their semantics will all be the same

with rep.new_layer():
    with rep.trigger.on_frame(num_frames=NUM_FRAMES):
        with rep.randomizer.instantiate(
                rep.utils.get_usd_files(PATH_TO_CONTENT_DIR, recursive=True, path_filter="SM_"), # path_filter is a regex to match file names
                size=NUM_OBJECTS,
                mode="scene_instance", # or 'point_instance'
                with_replacements=False,):
            rep.modify.pose(position=rep.distribution.uniform((-200,0,-200),(200,0,200)))
            rep.modify.semantics([("class", "test")])