TypeError: randomize_movement_in_view() missing 1 required positional argument: 'prim'

Omniverse Synthetic Data Generation (SDG)
, ,
1

I am trying to randomize the pose and rotation of objects spawn in the scene relative to the camera, hence, I used this function from offline_pose_estimation.py from Isaac Sim Standalone/Replicator examples so that I could apply it on traffic cone.

offline_pose_generation.py from 6. Offline Pose Estimation Synthetic Data Generation — Omniverse Robotics documentation

and offline_generation.py from: 3. Offline Dataset Generation — Omniverse Robotics documentation
Here’s variable initialization inside offline_generation.py:

world = World()
world.reset()

F_X = 768.1605834960938
F_Y = 768.1605834960938

width = 1024
height = 1024


fov_x = 2 * math.atan(width / (2 * F_X))
fov_y = 2 * math.atan(height / (2 * F_Y))

test_translations = [0.15, 0.15, -0.5]
test_rotations = [0.4619398, 0.1913417, 0.4619398, 0.7325378]

object_scale = [1, 1, 1]


train_parts = []

Here’s a sample code:

def randomize_movement_in_view(self, prim):
    """Randomly move and rotate prim such that it stays in view of camera.

    Args:
        prim (DynamicObject): prim to randomly move and rotate.
    """

    if not self.test:
        camera_prim = world.stage.GetPrimAtPath(self.camera_path)
        rig_prim = world.stage.GetPrimAtPath(self.rig.prim_path)
        translation, orientation = get_random_world_pose_in_view(
            camera_prim,
            0.4, #MIN_DISTANCE
            1.4, #MAX_DISTANCE
            fov_x,
            fov_y,
            0.9, #FRACTION_TO_SCREEN_EDGE
            rig_prim,
            np.array([-180, -90, -180]), #MIN_ROTATION_RANGE
            np.array([180, 90, 180]), #MAX_ROTATION_RANGE 
        )
    else:
        translation = np.array(test_translations)
        orientation = np.array(test_rotations)

    prim.set_world_pose(translation, orientation)

and here’s how it’s used:

    # # Spawn a new cone at a random pose
    cone_prim = prims.create_prim(
        prim_path=f"{SCOPE_NAME}/Cone",
        position=(random.uniform(-50, -4), random.uniform(-1, 3), 0),
        orientation=euler_angles_to_quat([0, 0, random.uniform(0, math.pi)]),
        usd_path=prefix_with_isaac_asset_server(CONE_URL),
        semantic_label="Cone",
    )

    # Add the part to train the network on
    part_name = "Cone"
    ref_path = get_assets_root_path() + CONE_URL
    prim_type = f"{part_name}"
    # path = "/World/" + prim_type
    path = f"{SCOPE_NAME}/Cone"
    name = "train_part"

    for child_prim in cone_prim.GetChildren():
        if child_prim.IsA(UsdGeom.Mesh):
            mesh_path = child_prim.GetPath()
            print('Cone mesh_path: ', mesh_path)


    train_part = DynamicObject(
        usd_path=ref_path,
        prim_path=path,
        mesh_path=mesh_path,
        name=name,
        position=np.array([0.0, 0.0, 0.0]),
        scale=np.array(object_scale),
        mass=1.0,
    )

    train_part.prim.GetAttribute("physics:rigidBodyEnabled").Set(True)

Here’s the error I receive:

(isaac-sim) mona@ard-gpu-01:~/.local/share/ov/pkg/isaac_sim-2022.2.0$ ./python.sh ~/omniverse-sdg/offline_generation.py 

7.706s] app ready
[7.794s] RTX ready
[7.794s] RTX ready
[7.877s] Simulation App Startup Complete
Loading Stage /Isaac/Environments/Simple_Warehouse/full_warehouse.usd
2023-02-06 19:48:08 [10,800ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1f91860240(61,8): C181 unused variable 'Local15'
2023-02-06 19:48:08 [10,800ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1f91860240(61,8): C181 unused variable 'Local15'
2023-02-06 19:48:08 [10,800ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1f91860240(61,8): C181 unused variable 'Local15'
2023-02-06 19:48:08 [10,800ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1f91860240(61,8): C181 unused variable 'Local15'
2023-02-06 19:48:08 [10,800ms] [Warning] [rtx.neuraylib.plugin] Compiler Core: omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1f91860240(61,8): C181 unused variable 'Local15'
2023-02-06 19:48:08 [10,800ms] [Warning] [rtx.neuraylib.plugin] Compiler Core: omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1f91860240(61,8): C181 unused variable 'Local15'
2023-02-06 19:48:09 [11,648ms] [Warning] [omni.isaac.core.simulation_context.simulation_context] A new stage was opened, World or Simulation Object are invalidated and you would need to initialize them again before using them.
Cone mesh_path:  /MyScope/Cone/S_TrafficCone
-------------------------------------------------
mesh path:  /MyScope/Cone/S_TrafficCone
prim:  Usd.Prim(</MyScope/Cone/S_TrafficCone>)
assets root path:  omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0
self.usd_path:  omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Props/S_TrafficCone.usd
type of train_part:  <class 'flying_distractors.dynamic_object.DynamicObject'>
2023-02-06 19:48:09 [11,670ms] [Error] [__main__] Exception: randomize_movement_in_view() missing 1 required positional argument: 'prim'
Traceback (most recent call last):
  File "/home/mona/omniverse-sdg/offline_generation.py", line 635, in <module>
    main()
  File "/home/mona/omniverse-sdg/offline_generation.py", line 515, in main
    randomize_movement_in_view(train_part)
TypeError: randomize_movement_in_view() missing 1 required positional argument: 'prim'
[11.765s] Simulation App Shutting Down

The randomize_movement_in_view function is coming from offline_pose_generation/offline_pose_generation.py example in Replicator. I also copied its flying_distractor folder to my code folder:

(base) mona@ard-gpu-01:~/omniverse-sdg$ tree .
.
├── flying_distractors
│   ├── collision_box.py
│   ├── dynamic_asset_set.py
│   ├── dynamic_object.py
│   ├── dynamic_object_set.py
│   ├── dynamic_shape_set.py
│   ├── flying_distractors.py
│   ├── __init__.py
│   └── __pycache__
│       ├── collision_box.cpython-37.pyc
│       ├── dynamic_asset_set.cpython-37.pyc
│       ├── dynamic_object.cpython-37.pyc
│       ├── dynamic_object_set.cpython-37.pyc
│       ├── dynamic_shape_set.cpython-37.pyc
│       ├── flying_distractors.cpython-37.pyc
│       └── __init__.cpython-37.pyc
├── offline_generation.py
├── README.md
└── test_USD.py

2 directories, 17 files

The traffic Cone is from here:

Screenshot from 2023-02-06 14-26-13
Screenshot from 2023-02-06 14-26-131917×996 139 KB

2

Hello @mona.jalal! I’ve shared your post with the dev team for further assistance.

1 Like
3

I’m not seeing in your code where you call randomize_movement_in_view()

The error message is saying that a prim object needs to be passed-in to the function call.

1 Like
4

I commented it below because it was throwing the error as above

    part_name = "Cone"
    ref_path = get_assets_root_path() + CONE_URL
    prim_type = f"{part_name}"
    # path = "/World/" + prim_type
    path = f"{SCOPE_NAME}/Cone"
    name = "train_part"

    for child_prim in cone_prim.GetChildren():
        if child_prim.IsA(UsdGeom.Mesh):
            mesh_path = child_prim.GetPath()
            print('Cone mesh_path: ', mesh_path)


    train_part = DynamicObject(
        usd_path=ref_path,
        prim_path=path,
        mesh_path=mesh_path,
        name=name,
        position=np.array([0.0, 0.0, 0.0]),
        scale=np.array(object_scale),
        mass=1.0,
    )

    train_part.prim.GetAttribute("physics:rigidBodyEnabled").Set(True)


    # train_parts.append(train_part)

    # for train_part in train_parts:
    #     randomize_movement_in_view(train_part)
5

It doesn’t appear as if your code is part of a Python class like the example code is?

If it is, you would need to call the function with self.randomize_movement_in_view(train_part)

If it is not part of a class, you would need to change the function to remove references to self

def randomize_movement_in_view(prim):
    """Randomly move and rotate prim such that it stays in view of camera.

    Args:
        prim (DynamicObject): prim to randomly move and rotate.
    """

        translation = np.array(test_translations)
        orientation = np.array(test_rotations)

    prim.set_world_pose(translation, orientation)
6

Thanks for noting that.

so I have these in my code:

world = World()

world.reset()

F_X = 768.1605834960938

F_Y = 768.1605834960938

width = 1024

height = 1024

fov_x = 2 * math.atan(width / (2 * F_X))

fov_y = 2 * math.atan(height / (2 * F_Y))

test_translations = [0.15, 0.15, -0.5]

test_rotations = [0.4619398, 0.1913417, 0.4619398, 0.7325378]

camera_rotation = [180, 0, 0]

object_scale = [1, 1, 1]

CLASS_NAME_TO_INDEX = {'Cardbox': 1, "Forklift": 2, 'Pallet': 3} # map format ## What is th exact relation of this to DOPEWriter?

test = False

use_s3 = False # you can use this for DOPE writer

bucket="" # you can use thing for s3 bucket name

#endpoint="" # you can use thing for s3 endpoint URL

endpoint=None

train_parts = []

writer_helper = DOPEWriter # or 'basic'

# Top View Camera 
# Camera looking at the forklift from a top view with large min clipping to see the scene through the ceiling
top_view_cam = rep.create.camera(clipping_range=(6.0, 1000000.0))

print('dir top cam: ', dir(top_view_cam))

top_view_cam_node = top_view_cam.node
print('top view cam node: ', top_view_cam_node)
top_view_cam_rig_path = rep.utils.get_node_targets(top_view_cam_node, "inputs:prims")[0]  # not sure if this is the correct way to do this
print('top view cam rig path: ', top_view_cam_rig_path)
top_view_cam_path = str(top_view_cam_rig_path) + '/Camera'
print('top view cam path: ', top_view_cam_path)
top_view_cam_rig = XFormPrim(prim_path=top_view_cam_rig_path)

def randomize_movement_in_view(prim):
    """Randomly move and rotate prim such that it stays in view of camera.

    Args:
        prim (DynamicObject): prim to randomly move and rotate.
    """

    if not test:
        #camera_prim = world.stage.GetPrimAtPath(camera_path)
        camera_prim = world.stage.GetPrimAtPath(top_view_cam_path)
        #rig_prim = world.stage.GetPrimAtPath(rig.prim_path)
        rig_prim = world.stage.GetPrimAtPath(top_view_cam_rig.prim_path)
        translation, orientation = get_random_world_pose_in_view(
            camera_prim,
            0.4, #MIN_DISTANCE
            1.4, #MAX_DISTANCE
            fov_x,
            fov_y,
            0.9, #FRACTION_TO_SCREEN_EDGE
            rig_prim,
            np.array([-180, -90, -180]), #MIN_ROTATION_RANGE
            np.array([180, 90, 180]), #MAX_ROTATION_RANGE 
        )
    else:
        translation = np.array(test_translations)
        orientation = np.array(test_rotations)

    prim.set_world_pose(translation, orientation)

   # # Spawn a new cone at a random pose
    cone_prim = prims.create_prim(
        prim_path=f"{SCOPE_NAME}/Cone",
        position=(random.uniform(-50, -4), random.uniform(-1, 3), 0),
        orientation=euler_angles_to_quat([0, 0, random.uniform(0, math.pi)]),
        usd_path=prefix_with_isaac_asset_server(CONE_URL),
        semantic_label="Cone",
    )

    # print("-------------------------------------------------------------")
    # print("cone prim type: ", type(cone_prim))
    # print("methods inside prim: ", dir(cone_prim))
    # print("cone prim: ", cone_prim)
    # print("cone prim path: ", cone_prim.GetPrimPath())
    # print("cone prim get path: ", cone_prim.GetPath())


    # Add the part to train the network on
    part_name = "Cone"
    ref_path = get_assets_root_path() + CONE_URL
    prim_type = f"{part_name}"
    # path = "/World/" + prim_type
    path = f"{SCOPE_NAME}/Cone"
    name = "train_part"

    for child_prim in cone_prim.GetChildren():
        if child_prim.IsA(UsdGeom.Mesh):
            mesh_path = child_prim.GetPath()
            print('Cone mesh_path: ', mesh_path)


    train_part = DynamicObject(
        usd_path=ref_path,
        prim_path=path,
        mesh_path=mesh_path,
        name=name,
        position=np.array([0.0, 0.0, 0.0]),
        scale=np.array(object_scale),
        mass=1.0,
    )

    train_part.prim.GetAttribute("physics:rigidBodyEnabled").Set(True)
    

    train_parts.append(train_part)

    for train_part in train_parts:
        randomize_movement_in_view(train_part)

Output is:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
dir top cam:  ['_ReplicatorItem__fn', '__annotations__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '__wrapped__', '_add_context', '_clear_prim_stacks', '_exec_stack', '_get_dependent_execs', '_pop_context', '_pop_exec_context', '_prim_groups', '_prim_stack', '_sequential', '_stack', 'get_context', 'get_exec_context', 'get_prim_context', 'node', 'reset', 'set_prim_context', 'set_sequential']
top view cam node:  Node("/Replicator/SDGPipeline/OgnGroup")
top view cam rig path:  /Replicator/Camera_Xform
top view cam path:  /Replicator/Camera_Xform/Camera
top view cam rig:  <omni.isaac.core.prims.xform_prim.XFormPrim object at 0x7f2b40d84210>
Loading Stage /Isaac/Environments/Simple_Warehouse/full_warehouse.usd
2023-02-08 15:12:59 [9,542ms] [Warning] [omni.client.plugin]  Tick: authentication: Discovery(wss://localhost/omni/discovery): Error creating Api/Connection search: Not connected
2023-02-08 15:13:02 [12,433ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1c7c2b26a0(61,8): C181 unused variable 'Local15'
2023-02-08 15:13:02 [12,433ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1c7c2b26a0(61,8): C181 unused variable 'Local15'
2023-02-08 15:13:02 [12,433ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1c7c2b26a0(61,8): C181 unused variable 'Local15'
2023-02-08 15:13:02 [12,433ms] [Warning] [rtx.neuraylib.plugin] [MDLC:COMPILER]   1.0   MDLC   comp warn : omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1c7c2b26a0(61,8): C181 unused variable 'Local15'
2023-02-08 15:13:02 [12,433ms] [Warning] [rtx.neuraylib.plugin] Compiler Core: omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1c7c2b26a0(61,8): C181 unused variable 'Local15'
2023-02-08 15:13:02 [12,433ms] [Warning] [rtx.neuraylib.plugin] Compiler Core: omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Materials/MaterialInstanceDynamic_1220.mdl?watch=00007f1c7c2b26a0(61,8): C181 unused variable 'Local15'
2023-02-08 15:13:03 [13,386ms] [Warning] [omni.isaac.core.simulation_context.simulation_context] A new stage was opened, World or Simulation Object are invalidated and you would need to initialize them again before using them.
Cone mesh_path:  /MyScope/Cone/S_TrafficCone
-------------------------------------------------
mesh path:  /MyScope/Cone/S_TrafficCone
prim:  Usd.Prim(</MyScope/Cone/S_TrafficCone>)
assets root path:  omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0
self.usd_path:  omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Environments/Simple_Warehouse/Props/S_TrafficCone.usd
2023-02-08 15:13:03 [13,410ms] [Error] [__main__] Exception: Accessed schema on invalid prim
Traceback (most recent call last):
  File "/home/mona/omniverse-sdg/offline_generation.py", line 761, in <module>
    main()
  File "/home/mona/omniverse-sdg/offline_generation.py", line 550, in main
    randomize_movement_in_view(train_part)
  File "/home/mona/omniverse-sdg/offline_generation.py", line 147, in randomize_movement_in_view
    np.array([180, 90, 180]), #MAX_ROTATION_RANGE
  File "/home/mona/.local/share/ov/pkg/isaac_sim-2022.2.0/exts/omni.isaac.core/omni/isaac/core/utils/random.py", line 123, in get_random_world_pose_in_view
    random_translation_from_prim = get_translation_from_target(random_translation_from_camera, camera_prim, coord_prim)
  File "/home/mona/.local/share/ov/pkg/isaac_sim-2022.2.0/exts/omni.isaac.core/omni/isaac/core/utils/transformations.py", line 137, in get_translation_from_target
    source_to_target = get_relative_transform(source_prim, target_prim)
  File "/home/mona/.local/share/ov/pkg/isaac_sim-2022.2.0/exts/omni.isaac.core/omni/isaac/core/utils/transformations.py", line 106, in get_relative_transform
    source_to_world_row_major_tf = UsdGeom.Xformable(source_prim).ComputeLocalToWorldTransform(Usd.TimeCode.Default())
RuntimeError: Accessed schema on invalid prim
[13.550s] Simulation App Shutting Down

Here I am trying to spawn a traffic cone and keep it in the view of top_view_camera in offline_generation.py code.

Also, this part of original offline_generation.py code changes the pose of camera.

        with top_view_cam:
            rep.modify.pose(
                position=rep.distribution.uniform(
                    (foklift_pos_gf[0], foklift_pos_gf[1], 9), (foklift_pos_gf[0], foklift_pos_gf[1], 11)
                ),
                rotation=rep.distribution.uniform((0, -90, 0), (0, -90, 180)),
            )