[Replicator Interactive SDG] fruit box tutorial on local machine

I am trying to run the fruitbox demo in the Script Editor of Omniverse Code. However, the fruits are not shown in the Nucleus. Further, I see this message regarding collision.

Do you know why fruit props are not visible?

  FRUIT_PROPS = {
        'apple': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Apple.usd',
        'avocado': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Avocado01.usd',
        'kiwi': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Kiwi01.usd',
        'lime': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Lime01.usd',
        'lychee': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Lychee01.usd',
        'pomegranate': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Pomegranate01.usd',
        'onion': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Vegetables/RedOnion.usd',
        'lemon': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Decor/Tchotchkes/Lemon_01.usd',
        'orange': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Decor/Tchotchkes/Orange_01.usd'    }
    

Screenshot from 2023-07-04 20-11-16

The full code is:

# run using path to ov/pkg/code-2022-3.1 && ./omni.code.sh --no-window --/omni/replicator/script=path to this script

import datetime
now = datetime.datetime.now()
import omni.replicator.core as rep

with rep.new_layer():
    CRATE = 'omniverse://localhost/NVIDIA/Samples/Marbles/assets/standalone/SM_room_crate_3/SM_room_crate_3.usd'    
    SURFACE = 'omniverse://localhost/NVIDIA/Assets/Scenes/Templates/Basic/display_riser.usd'    
    ENVS = 'omniverse://localhost/NVIDIA/Assets/Scenes/Templates/Interior/ZetCG_ExhibitionHall.usd'
    FRUIT_PROPS = {
        'apple': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Apple.usd',
        'avocado': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Avocado01.usd',
        'kiwi': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Kiwi01.usd',
        'lime': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Lime01.usd',
        'lychee': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Lychee01.usd',
        'pomegranate': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Fruit/Pomegranate01.usd',
        'onion': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Food/Vegetables/RedOnion.usd',
        'lemon': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Decor/Tchotchkes/Lemon_01.usd',
        'orange': 'omniverse://localhost/NVIDIA/Assets/ArchVis/Residential/Decor/Tchotchkes/Orange_01.usd'    }
    
    def random_props(file_name, class_name, max_number=1, one_in_n_chance=3):
        instances = rep.randomizer.instantiate(file_name, size=max_number, mode='scene_instance')
        print(file_name)
        with instances:
            rep.modify.semantics([('class', class_name)])
            rep.modify.pose(
                position=rep.distribution.uniform((-8, 5, -25), (8, 30, 25)),
                rotation=rep.distribution.uniform((-180,-180, -180), (180, 180, 180)),
                scale = rep.distribution.uniform((0.8), (1.2)), 
            )

            rep.modify.visibility(rep.distribution.choice([True],[False]*(one_in_n_chance)))
        return instances.node
    
    def sphere_lights(num):
        lights = rep.create.light(
            light_type="Sphere",
            temperature=rep.distribution.normal(6500, 500),
            intensity=rep.distribution.normal(30000, 5000),
            position=rep.distribution.uniform((-300, -300, -300), (300, 300, 300)),
            scale=rep.distribution.uniform(50, 100),
            count=num        )
        return lights.node    
    rep.randomizer.register(sphere_lights)
    
    env = rep.create.from_usd(ENVS)
    surface = rep.create.from_usd(SURFACE)
    with surface:
        rep.physics.collider()
    crate = rep.create.from_usd(CRATE)
    with crate:
        rep.physics.collider()
        rep.physics.mass(mass=10000)
        rep.modify.pose(
                position=(0, 20, 0),
                rotation=(0, 0, 90)
            )
        
    
    camera = rep.randomizer.register.camera()
    render_product = rep.create.render_product(camera, resolution=(1024, 1024))
    
    rep.randomizer.register(random_props)
    rep.randomizer.register(sphere_lights)
    
    # trigger on frame for an interval
    with rep.trigger.on_frame(num_frames=100):
        rep.modify.timeline(5, "frame")
        for n, f in FRUIT_PROPS.items():
            random_props(f, n)
        rep.randomizer.sphere_lights(5)
        with camera:
            rep.modify.pose(position=rep.distribution.uniform((-20, 90, -17), (10, 140, -15)), look_at=(0, 20, 20))
            
    # initialize and attach writer
    writer = rep.WriterRegistry.get("BasicWriter")
    now = now.strftime("%Y-%m-%d_%H:%M:%S")
    output_dir = "/hdd/SDG_out/fruit_box/" + now
    writer.initialize(output_dir=output_dir, rgb=True, bounding_box_2d_tight=True)
    writer.attach([render_product])
            
    

this is really strange, search doesn’t work (previously it worked for other things like forklift)

However, it is somehow stuck in running phase and no image is produced.
I wonder if it may have anything to do with the collision warning?

Here’s output of $ tiptop:

Are there any errors or warnings in the log?

I tried running and saw the warning: 'function' object has no attribute 'camera'
So I had to change camera = rep.randomizer.register.camera() to camera = rep.create.camera() and the script ran, but generation had other errors.

1 Like

Ok, after a bit more debugging, I found that there was an issue with the way visibility was being set.

I changed

rep.modify.visibility(rep.distribution.choice([True],[False]*(one_in_n_chance)))

to

a = [True]
a.extend([False]*(one_in_n_chance))
rep.modify.visibility(rep.distribution.choice(a))

Another issue on my end at least was running on Windows with ‘:’ characters in the output path is not allowed, so I removed now from the output_dir and was able to run and get output.

eg.

1 Like

@ecameracci thanks a lot for trying this. With using both your changes, it works now.
However, the collisions are not realistic. Do you know how it can get fixed? For example the pomegranate and avocado are spawn in middle of the fruitbox which is not realistic. Thanks for any feedback.

Hi, Mona :)

Which version of replicator are you using?

Unfortunately, I do not believe that these collider commands on the crate (or even adding colliders to the fruit in the box) will have any affect on how they are spawned. I believe the root position of each is just sampled randomly within the x,y,z coordinates you specified in rep.modify.pose and it is placed there. So it’s possible the fruits could initially overlap each other too.

One way you could get the fruits to not initially overlap each other is to use the scatter3d function, which has a flag where you can set collision checking to True. For this, you would want to create a cuboid representing the inside of the box, and use that as an arg to scatter3d (as well as the fruit). However, replicator 1.6.3 (which you are likely using) does not have a way of checking for collisions with the hull of the box. So as a workaround you could specify the cuboid for the inside of the box to be slightly smaller, so it is a distance “r” from the inside of the box, where “r” is the radius of the largest piece of fruit.

Let me know if this makes sense and is a feasible solution.

Replicator 1.9 makes all of this easier, and does collision checking with as many other shapes as you want – and it also can prune the sampling space to make sampling more efficient. I think I made some hacked code for doing this with 1.6.3 but it’s kind of messy and it currently is just for scatter2d.

Sneek peek at 1.9 scatter3d function attached … where cones are prevented from initially colliding with the inner blue cylinder.

-Henry


1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.