Collision Detection for Scatter not working well

it’s not loading anything from the nucleus properly, i’m in contact with an expert on this though

Run this command in the container:

    /isaac-sim/runheadless.native.sh  --no-window \
                                      --/app/viewport/grid/enabled=false \
                                      --/app/viewport/defaults/guide/grid/visible=false \
                                      --/omni/replicator/replicatorYaml/yamlPath=$FILE

Full scenario with cloud links:

# Default scene settings. This script default is 1 (IsaacSim), so if you change
# the meters per unit or up axis, you need to alter the coordinates and rotations.

# IsaacSim default is 1, Code is 0.01
stage_unit_setting:
  settings.set_stage_meters_per_unit:
    meters_per_unit: 1

# IsaacSim default is "Z"", Code is "Y"
stage_up_axis_setting:
  settings.set_stage_up_axis:
    up_axis: Z

# Enable Path Tracing
path_tracing:
  settings.set_render_pathtraced:
    samples_per_pixel: 32

path_tracing_clamp:
  settings.carb_settings:
    setting: /rtx/pathtracing/clampSpp
    value: 0

# Load the stage
load_stage:
  create.from_usd:
    usd: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/full_warehouse.usd

# Select the objects in the stage
forklift:
  get.prim_at_path:
    path:  /Replicator/Ref_Xform/Ref/forklift

floor:
  get.prims:
    semantics: [[class, floor]]

racks_and_walls:
  get.prims:
    semantics: [[class, rack], [class, wall]]

camera:
  create.camera:
    position: [0, 0, 0]
    rotation: [0, 0, 0]

# Create output
render_product:
  create.render_product:
    camera: camera
    resolution: [1280, 720]

writer:
  writers.get:
    name: BasicWriter
    init_params:
      rgb: True
      bounding_box_2d_tight: True
      semantic_filter_predicate: class:sign|cone|person
      output_dir: /workspaces/Surveily.Datasets.Generator.Omniverse/.out/generation/test

writer_attach:
  writer.attach:
    render_products: render_product

# Load the animations
actor_load_animations:
  create.from_dir:
    recursive: True
    semantics: [[animation, actor]]
    dir_path: omniverse://localhost/NVIDIA/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Motion/

actor_animations:
  get.prims:
    semantics: [[animation, actor]]

# Set the trigger as on_frame
trigger:
  trigger.on_frame:
    interval: 2
    max_execs: 20
    #rt_subframes: 10

# When the trigger executes, run the following logic
with_trigger:
  with.trigger:
    # Create and scatter actor
    actor_instance:
      randomizer.instantiate:
        paths:
          - omniverse://localhost/NVIDIA/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Actor/business-f-0002/business-f-0002.usd
        size: #15
          distribution.uniform:
            lower: [1]
            upper: [10]
        use_cache: False

    with.actor_instance:
      modify.pose:
        scale: 0.01
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      modify.semantics:
        semantics: [[class, person]]
      modify.animation:
        values: 
          distribution.sequence:
            items: actor_animations
      randomizer.scatter_2d:
        seed: 1
        surface_prims: floor
        check_for_collisions: True

    # create and scatter cones
    cone_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_TrafficCone.usd
        size: #50
          distribution.uniform:
            lower: [20]
            upper: [50]
        mode: scene_instance

    with.cone_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 1
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # create and scatter wet floor signs
    wet_floor_sign_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_WetFloorSign.usd
        size: #50
          distribution.uniform:
            lower: [20]
            upper: [50]
        mode: scene_instance

    with.wet_floor_sign_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 3
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # move the camera around and rotate
    with.camera:
      modify.pose:
        position:
          distribution.uniform:
            lower: [-23.6, -3, 2.5]
            upper: [3.03, 3, 7.5]
        rotation:
          distribution.uniform:
            lower: [0, -30, -120]
            upper: [0, -10, -60]

    # move the forklift around and rotate
    with.forklift:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      # BELOW BREAKS THE SCRIPT:
      # randomizer.scatter_2d:
      #   seed: 0
      #   surface_prims: floor
      #   no_coll_prims: racks_and_walls
      #   check_for_collisions: True

I just got it going in the container using this yaml file - synthetic-data-examples/omni.replicator_yaml/warehouse.yaml at main · Surveily/synthetic-data-examples · GitHub

and I have the same result as I posted here. I stepped replciator and I don’t see any collisions. Is there a certain amount of times I should step it to see the bug?

@hclever a couple of points:

The script below 100% reproduces the issue, please try it and look for collisions between forks and racks.

# Default scene settings. This script default is 1 (IsaacSim), so if you change
# the meters per unit or up axis, you need to alter the coordinates and rotations.

# IsaacSim default is 1, Code is 0.01
stage_unit_setting:
  settings.set_stage_meters_per_unit:
    meters_per_unit: 1

# IsaacSim default is "Z"", Code is "Y"
stage_up_axis_setting:
  settings.set_stage_up_axis:
    up_axis: Z

# Enable Path Tracing
path_tracing:
  settings.set_render_pathtraced:
    samples_per_pixel: 32

path_tracing_clamp:
  settings.carb_settings:
    setting: /rtx/pathtracing/clampSpp
    value: 0

# Load the stage
load_stage:
  create.from_usd:
    usd: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/full_warehouse.usd

# Select the objects in the stage
forklift:
  get.prim_at_path:
    path:  /Replicator/Ref_Xform/Ref/forklift

floor:
  get.prims:
    semantics: [[class, floor]]

racks_and_walls:
  get.prims:
    semantics: [[class, rack], [class, wall]]

camera:
  create.camera:
    position: [0, 0, 0]
    rotation: [0, 0, 0]

# Create output
render_product:
  create.render_product:
    camera: camera
    resolution: [1280, 720]

writer:
  writers.get:
    name: BasicWriter
    init_params:
      rgb: True
      bounding_box_2d_tight: True
      semantic_filter_predicate: class:sign|cone|person
      output_dir: /workspaces/Surveily.Datasets.Generator.Omniverse/.out/Issues/ghosting

writer_attach:
  writer.attach:
    render_products: render_product

# Load the animations
actor_load_animations:
  create.from_dir:
    recursive: True
    semantics: [[animation, actor]]
    dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Motion/

actor_animations:
  get.prims:
    semantics: [[animation, actor]]

# Set the trigger as on_frame
trigger:
  trigger.on_frame:
    interval: 2
    max_execs: 20
    #rt_subframes: 10

# When the trigger executes, run the following logic
with_trigger:
  with.trigger:
    # Create and scatter actor
    actor_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Actor/business-f-0002/business-f-0002.usd
        size: #15
          distribution.uniform:
            lower: 1
            upper: 10
        use_cache: False

    with.actor_instance:
      modify.pose:
        scale: 0.01
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      modify.semantics:
        semantics: [[class, person]]
      modify.animation:
        values: 
          distribution.sequence:
            items: actor_animations
      randomizer.scatter_2d:
        seed: 1
        surface_prims: floor
        check_for_collisions: True

    # create and scatter forklifts
    fork_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Props/Forklift/forklift.usd
        size: 40
        mode: scene_instance

    with.fork_instance:
      modify.pose:
        scale: 100
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 100
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # create and scatter cones
    cone_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_TrafficCone.usd
        size: #50
          distribution.uniform:
            lower: 20
            upper: 50
        mode: scene_instance

    with.cone_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 1
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # create and scatter wet floor signs
    wet_floor_sign_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_WetFloorSign.usd
        size: #50
          distribution.uniform:
            lower: 20
            upper: 50
        mode: scene_instance

    with.wet_floor_sign_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 3
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # move the camera around and rotate
    with.camera:
      modify.pose:
        position:
          distribution.uniform:
            lower: [-23.6, -3, 2.5]
            upper: [3.03, 3, 7.5]
        rotation:
          distribution.uniform:
            lower: [0, -30, -120]
            upper: [0, -10, -60]

    # move the forklift around and rotate
    with.forklift:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      # randomizer.scatter_2d:
      #   seed: 0
      #   surface_prims: floor
      #   no_coll_prims: racks_and_walls
      #   check_for_collisions: True

I tried with this script and i only see output frames with the person in various places and same camera view? The scatter throws an error but I don’t see the forklift in any of the frames and the error seems unrelated to the forklift. see below -

Hey @hclever

  • Did you use the updated scenario yaml that I posted on 28th of March?
  • Did you update the output_dir in the scenario?

Please try the following instead, there was a bug with forklift scale. Make sure to update output_dir

# Default scene settings. This script default is 1 (IsaacSim), so if you change
# the meters per unit or up axis, you need to alter the coordinates and rotations.

# IsaacSim default is 1, Code is 0.01
stage_unit_setting:
  settings.set_stage_meters_per_unit:
    meters_per_unit: 1

# IsaacSim default is "Z"", Code is "Y"
stage_up_axis_setting:
  settings.set_stage_up_axis:
    up_axis: Z

# Enable Path Tracing
path_tracing:
  settings.set_render_pathtraced:
    samples_per_pixel: 32

path_tracing_clamp:
  settings.carb_settings:
    setting: /rtx/pathtracing/clampSpp
    value: 0

# Load the stage
load_stage:
  create.from_usd:
    usd: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/full_warehouse.usd

# Select the objects in the stage
forklift:
  get.prim_at_path:
    path:  /Replicator/Ref_Xform/Ref/forklift

floor:
  get.prims:
    semantics: [[class, floor]]

racks_and_walls:
  get.prims:
    semantics: [[class, rack], [class, wall]]

camera:
  create.camera:
    position: [0, 0, 0]
    rotation: [0, 0, 0]

# Create output
render_product:
  create.render_product:
    camera: camera
    resolution: [1280, 720]

writer:
  writers.get:
    name: BasicWriter
    init_params:
      rgb: True
      bounding_box_2d_tight: True
      semantic_filter_predicate: class:sign|cone|person
      output_dir: /workspaces/Surveily.Datasets.Generator.Omniverse/.out/Issues/collisions

writer_attach:
  writer.attach:
    render_products: render_product

# Load the animations
actor_load_animations:
  create.from_dir:
    recursive: True
    semantics: [[animation, actor]]
    dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Motion/

actor_animations:
  get.prims:
    semantics: [[animation, actor]]

# Set the trigger as on_frame
trigger:
  trigger.on_frame:
    interval: 2
    max_execs: 20
    #rt_subframes: 10

# When the trigger executes, run the following logic
with_trigger:
  with.trigger:
    # Create and scatter actor
    actor_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Actor/business-f-0002/business-f-0002.usd
        size: #15
          distribution.uniform:
            lower: 1
            upper: 10
        use_cache: False

    with.actor_instance:
      modify.pose:
        scale: 0.01
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      modify.semantics:
        semantics: [[class, person]]
      modify.animation:
        values: 
          distribution.sequence:
            items: actor_animations
      randomizer.scatter_2d:
        seed: 1
        surface_prims: floor
        check_for_collisions: True

    # create and scatter forklifts
    fork_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Props/Forklift/forklift.usd
        size: 40
        mode: scene_instance

    with.fork_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 100
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # create and scatter cones
    cone_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_TrafficCone.usd
        size: #50
          distribution.uniform:
            lower: 20
            upper: 50
        mode: scene_instance

    with.cone_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 1
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # create and scatter wet floor signs
    wet_floor_sign_instance:
      randomizer.instantiate:
        paths:
          - http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_WetFloorSign.usd
        size: #50
          distribution.uniform:
            lower: 20
            upper: 50
        mode: scene_instance

    with.wet_floor_sign_instance:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      randomizer.scatter_2d:
        seed: 3
        surface_prims: floor
        no_coll_prims: racks_and_walls
        check_for_collisions: True

    # move the camera around and rotate
    with.camera:
      modify.pose:
        position:
          distribution.uniform:
            lower: [-23.6, -3, 2.5]
            upper: [3.03, 3, 7.5]
        rotation:
          distribution.uniform:
            lower: [0, -30, -120]
            upper: [0, -10, -60]

    # move the forklift around and rotate
    with.forklift:
      modify.pose:
        rotation:
          distribution.uniform:
            lower: [0, 0, -180]
            upper: [0, 0, 180]
      # randomizer.scatter_2d:
      #   seed: 0
      #   surface_prims: floor
      #   no_coll_prims: racks_and_walls
      #   check_for_collisions: True

Here is what I am getting in the output folder (rgb files). Collisions and ghosts.

Yes to both -

OK, I will give this a try.

-H

Hi,

The ghosting is a different issue.

In your scene you have 41 forklifts. With the first 40 you modify them using with.fork_instance: the remaining one you modify using with.forklift and no scatter node. The first 40 all seem to be fine - i.e. non colliding. I only see collisions with the last one, which randomizes without using scatter node for checking collisions. This is consistent with the first of the last set of images you showed.

Good spot. Unfortunately I get collisions in the scattered ones. Can you run it multiple times / steps and check again?

do you get any errors in your terminal where it is run? can you check the log file and search it for “Error”?