ValueError: Unable to set target prim of type <class 'pxr.Usd.Prim'> in Isaac Sim Script Editor

mona.jalal · September 7, 2023, 4:27pm

How should I fix this error? I want to throw the pallets using scatter_2d onto the floor_prims

Error:


import omni.usd

import omni.replicator.core as rep
from pxr import Usd, UsdGeom, Gf
from datetime import datetime



import numpy as np


PALLET_URL = "omniverse://localhost/NVIDIA/Assets/ArchVis/Industrial/Pallets/Pallet_B1.usd"
ENV_URL ="omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Samples/Replicator/Stage/full_warehouse_worker_and_anim_cameras.usd"


#open_stage(ENV_URL)
#result, error = omni.usd.get_context().open_stage(ENV_URL, load_set=omni.usd.UsdContextInitialLoadSet.LOAD_NONE)
omni.usd.get_context().open_stage(ENV_URL)

stage = omni.usd.get_context().get_stage()

#stage = get_current_stage()

#floor_prims = [x.GetPath() for x in stage.Traverse() if "SM_floor" in x.GetName() and prims_utils.get_prim_type_name(x.GetPath()) == "Xform"]
floor_prims_paths = [x.GetPath() for x in stage.Traverse() if "SM_floor" in x.GetName() ]




floor_prims = []
for floor_prim_path in floor_prims_paths:
    floor_prims.append(omni.usd.get_prim_at_path(floor_prim_path))
    

#bb_cache = bounds_utils.create_bbox_cache()

#combined_range_arr = bounds_utils.compute_combined_aabb(bb_cache, prim_paths=floor_prims)

#min_x, min_y, min_z, max_x, max_y, max_z = combined_range_arr

# bbox = omni.usd.get_context().compute_path_world_bounding_box(floor_prims[0])
# min_x, min_y, min_z, max_x, max_y, max_z = bbox

# print("bbox is: ", bbox)


def compute_bbox_with_cache(cache: UsdGeom.BBoxCache, prim: Usd.Prim) -> Gf.Range3d:
    """
    Compute Bounding Box using ComputeWorldBound at UsdGeom.BBoxCache. More efficient if used multiple times.
    See https://graphics.pixar.com/usd/dev/api/class_usd_geom_b_box_cache.html

    Args:
        cache: A cached, i.e. `UsdGeom.BBoxCache(Usd.TimeCode.Default(), ['default', 'render'])`
        prim: A prim to compute the bounding box.
    Returns:
        A range (i.e. bounding box), see more at: https://graphics.pixar.com/usd/release/api/class_gf_range3d.html

    """
    bound = cache.ComputeWorldBound(prim)
    bound_range = bound.ComputeAlignedBox()
    return bound_range


bbcache = UsdGeom.BBoxCache(Usd.TimeCode.Default(), ["default", "render"])

bb = compute_bbox_with_cache(bbcache, floor_prims[0])

max_x, max_y, max_z = bb.GetMax()

min_x_arr = []
min_y_arr = []
min_z_arr = []
max_x_arr = []
max_y_arr = []
max_z_arr = []

for floor_prim in floor_prims:
    tmp_bb = compute_bbox_with_cache(bbcache, floor_prim)

    tmp_min_x, tmp_min_y, tmp_min_z = tmp_bb.GetMin()
    tmp_max_x, tmp_max_y, tmp_max_z = tmp_bb.GetMax()
    min_x_arr.append(tmp_min_x)
    min_y_arr.append(tmp_min_y)
    min_z_arr.append(tmp_min_z)
    max_x_arr.append(tmp_max_x)
    max_y_arr.append(tmp_max_y)
    max_z_arr.append(tmp_max_z)
    

min_x = min(min_x_arr)
min_y = min(min_y_arr)
min_z = min(min_z_arr)

max_x = max(max_x_arr)
max_y = max(max_y_arr)
max_z = max(max_z_arr)




margin = 3



def scatter_pallets(prim):
    
    
    pallets = rep.create.from_usd(PALLET_URL, semantics=[('class', 'pallet')], count=50)
    with pallets:
        rep.physics.collider()
        rep.modify.pose(scale=0.001)
        rep.randomizer.scatter_2d(surface_prims=prim, check_for_collisions=True),
        
    return pallets

# Register randomization

rep.randomizer.register(scatter_pallets)


# Setup camera and attach it to render product
camera = rep.create.camera(
    focus_distance=800,
    focal_length=40
    # f_stop=0.5,
    # f_stop=500,
    #focal_length=613.634
)
render_product = rep.create.render_product(camera, resolution=(1024, 1024))


timestamp = datetime.now().strftime("%Y_%m_%d-%I_%M_%S_%p")
# Initialize and attach writer
writer = rep.WriterRegistry.get("BasicWriter")
writer.initialize(output_dir="/home/mona/Desktop/Isaac_Sim_Dummy_Out/scattered_pallets_warehouse/" + timestamp , rgb=True, bounding_box_2d_tight=True)
writer.attach([render_product])

with rep.trigger.on_frame(num_frames=50):
    pallets = rep.randomizer.scatter_pallets(floor_prims)
    print('pallets is: ', pallets)
    print('pallets type is: ', type(pallets))
    # pallets = rep.randomizer.randomize_pallet()
    with camera:
        rep.modify.pose(position=rep.distribution.uniform((min_x + margin, min_y + margin, 4), (max_x - margin, max_y - margin, 7)),
                        look_at=pallets)

ahaidu · September 13, 2023, 6:48am

Hi there,

It seems I have issues running your script, can you provide a more succinct version that contains the error you are having?
Are you using the latest isaac sim version, 2022.2.1?

Best,
Andrei

mona.jalal · September 14, 2023, 8:57pm

Here’s the code (removed comments – but the code is interleaved/needed)

# problem with scaling of pallets (pallets encompass the warehouse as of 09.07.2023)
import omni.usd

import omni.replicator.core as rep
from pxr import Usd, UsdGeom, Gf
from datetime import datetime



import numpy as np


PALLET_URL = "omniverse://localhost/NVIDIA/Assets/ArchVis/Industrial/Pallets/Pallet_B1.usd"
ENV_URL ="omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Samples/Replicator/Stage/full_warehouse_worker_and_anim_cameras.usd"

omni.usd.get_context().open_stage(ENV_URL)

stage = omni.usd.get_context().get_stage()

floor_prims_paths = [x.GetPath() for x in stage.Traverse() if "SM_floor" in x.GetName() ]




floor_prims = []
for floor_prim_path in floor_prims_paths:
    floor_prims.append(omni.usd.get_prim_at_path(floor_prim_path))



def compute_bbox_with_cache(cache: UsdGeom.BBoxCache, prim: Usd.Prim) -> Gf.Range3d:

    bound = cache.ComputeWorldBound(prim)
    bound_range = bound.ComputeAlignedBox()
    return bound_range


bbcache = UsdGeom.BBoxCache(Usd.TimeCode.Default(), ["default", "render"])

bb = compute_bbox_with_cache(bbcache, floor_prims[0])

max_x, max_y, max_z = bb.GetMax()

min_x_arr = []
min_y_arr = []
min_z_arr = []
max_x_arr = []
max_y_arr = []
max_z_arr = []

for floor_prim in floor_prims:
    tmp_bb = compute_bbox_with_cache(bbcache, floor_prim)
    # tmp_corners = tmp_bb.GetCorner
    tmp_min_x, tmp_min_y, tmp_min_z = tmp_bb.GetMin()
    tmp_max_x, tmp_max_y, tmp_max_z = tmp_bb.GetMax()
    min_x_arr.append(tmp_min_x)
    min_y_arr.append(tmp_min_y)
    min_z_arr.append(tmp_min_z)
    max_x_arr.append(tmp_max_x)
    max_y_arr.append(tmp_max_y)
    max_z_arr.append(tmp_max_z)
    

min_x = min(min_x_arr)
min_y = min(min_y_arr)
min_z = min(min_z_arr)

max_x = max(max_x_arr)
max_y = max(max_y_arr)
max_z = max(max_z_arr)




margin = 3



def scatter_pallets(prim):

    
    pallets = rep.create.from_usd(PALLET_URL, semantics=[('class', 'pallet')], count=50)
    with pallets:
        rep.physics.collider()
        # rep.modify.pose(scale=0.001)
        rep.modify.pose(scale=(0.01, 0.01, 0.01))
        rep.randomizer.scatter_2d(surface_prims=prim, check_for_collisions=True),
        
    return pallets

# Register randomization

rep.randomizer.register(scatter_pallets)




# Setup camera and attach it to render product
camera = rep.create.camera(
    focus_distance=800,
    focal_length=40
)
render_product = rep.create.render_product(camera, resolution=(1024, 1024))


timestamp = datetime.now().strftime("%Y_%m_%d-%I_%M_%S_%p")
# Initialize and attach writer
writer = rep.WriterRegistry.get("BasicWriter")
writer.initialize(output_dir="/home/mona/Desktop/Isaac_Sim_Dummy_Out/scattered_pallets_warehouse/" + timestamp , rgb=True, bounding_box_2d_tight=True)
writer.attach([render_product])

with rep.trigger.on_frame(num_frames=50):
    pallets = rep.randomizer.scatter_pallets(floor_prims)
    with camera:
        rep.modify.pose(position=rep.distribution.uniform((min_x + margin, min_y + margin, 4), (max_x - margin, max_y - margin, 7)),
                        look_at=pallets)

Version is: Isaac Sim Release 2022.2.1

Here’s the error:

The goal is:

find the bounding box of the warehouse
consider the result from #1 as floor prims
use the floor_prim as the scatter plane for scatter_2d for the pallets
look at the scattered pallets with the camera and take photos

ahaidu · September 18, 2023, 8:56am

Hi there,

the look_at arguments requires a list of paths in case of multiple prims. In the newer replicator version you can use get_output_prims on the node to access these:

    pallets = rep.create.from_usd(PALLET_URL, semantics=[("class", "pallet")], count=10)
    pallets_paths = [prim.GetPath() for prim in pallets.get_output_prims()["prims"]]

Here is a fixed script which should work with the shipping of version isaac sim version 2023.1.0:

import os

import omni
import omni.replicator.core as rep
from omni.isaac.core.utils.nucleus import get_assets_root_path
from pxr import Gf, Usd, UsdGeom

PALLET_URL = "/Isaac/Environments/Simple_Warehouse/Props/SM_PaletteA_01.usd"
ENV_URL = "/Isaac/Environments/Simple_Warehouse/full_warehouse.usd"

assets_root_path = get_assets_root_path()
omni.usd.get_context().open_stage(assets_root_path + ENV_URL)
stage = omni.usd.get_context().get_stage()
floor_prims_paths = [prim.GetPath() for prim in stage.Traverse() if "SM_floor" in prim.GetName()]

floor_prims = []
for floor_prim_path in floor_prims_paths:
    floor_prims.append(omni.usd.get_prim_at_path(floor_prim_path))

def compute_bbox_with_cache(cache: UsdGeom.BBoxCache, prim: Usd.Prim) -> Gf.Range3d:
    bound = cache.ComputeWorldBound(prim)
    bound_range = bound.ComputeAlignedBox()
    return bound_range


bbcache = UsdGeom.BBoxCache(Usd.TimeCode.Default(), ["default", "render"])
bb = compute_bbox_with_cache(bbcache, floor_prims[0])
max_x, max_y, max_z = bb.GetMax()
min_x_arr = []
min_y_arr = []
min_z_arr = []
max_x_arr = []
max_y_arr = []
max_z_arr = []

for floor_prim in floor_prims:
    tmp_bb = compute_bbox_with_cache(bbcache, floor_prim)
    tmp_min_x, tmp_min_y, tmp_min_z = tmp_bb.GetMin()
    tmp_max_x, tmp_max_y, tmp_max_z = tmp_bb.GetMax()
    min_x_arr.append(tmp_min_x)
    min_y_arr.append(tmp_min_y)
    min_z_arr.append(tmp_min_z)
    max_x_arr.append(tmp_max_x)
    max_y_arr.append(tmp_max_y)
    max_z_arr.append(tmp_max_z)


min_x = min(min_x_arr)
min_y = min(min_y_arr)
min_z = min(min_z_arr)
max_x = max(max_x_arr)
max_y = max(max_y_arr)
max_z = max(max_z_arr)
margin = 3


camera = rep.create.camera()
render_product = rep.create.render_product(camera, resolution=(1024, 1024))

writer = rep.WriterRegistry.get("BasicWriter")
out_dir = os.path.join(os.getcwd(), "_out_scattered_pallets_warehouse")
writer.initialize(output_dir=out_dir, rgb=True, bounding_box_2d_tight=True)
writer.attach([render_product])

with rep.trigger.on_frame(num_frames=50):
    pallets = rep.create.from_usd(PALLET_URL, semantics=[("class", "pallet")], count=10)
    pallets_paths = [prim.GetPath() for prim in pallets.get_output_prims()["prims"]]
    print(f"pallets_paths: {pallets_paths}")
    with pallets:
        rep.physics.collider()
        floor_paths_as_str = [str(path) for path in floor_prims_paths]
        rep.randomizer.scatter_2d(surface_prims=floor_paths_as_str, check_for_collisions=True),
    with camera:
        rep.modify.pose(
            position=rep.distribution.uniform((min_x + margin, min_y + margin, 4), (max_x - margin, max_y - margin, 7)),
            look_at=pallets_paths,
        )

mona.jalal · September 18, 2023, 12:44pm

Thanks a lot for your response. I don’t have access to the version you mentioned and the latest I have access to is 2022.2.1
Is there a way to get early access to your version?

running the code I got this error:

ahaidu · September 18, 2023, 12:48pm

2023.1.0 should be out in a couple of weeks, until then you could spawn the pallets using usd/isaac sim api before creating the graph and use their paths as inputs.

fayezsalka · September 19, 2023, 10:24am

sorry to hijack this thread, but will 2023.1.0 improve multi camera rendering gpu memory footprint? https://forums.developer.nvidia.com/t/2022-2-1-rendering-20-cameras-without-running-out-of-memory/266877

rthaker · September 19, 2023, 9:39pm

You will get answered by different dev in your thread. I would request to avoid adding different topic in different thread.

mona.jalal · October 30, 2023, 7:55pm

so I was finally able to try the code you shared in Isaac Sim 2023.1.0 today. If I use the PALLET_URL = "/Isaac/Environments/Simple_Warehouse/Props/SM_PaletteA_01.usd" it will show no pallet in created data. If I use PALLET_URL = "omniverse://localhost/NVIDIA/Assets/ArchVis/Industrial/Pallets/Pallet_B1.usd" it still encompasses a pallet around the warehouse. Could you please share the results you see? I don’t see any pallets created in the output images. Since there is none, the look_at param is also taking random photos that there is no pallet.

## running code in Isaac Sim 2023.1.0

import os

import omni
import omni.replicator.core as rep
from omni.isaac.core.utils.nucleus import get_assets_root_path
from pxr import Gf, Usd, UsdGeom

PALLET_URL = "/Isaac/Environments/Simple_Warehouse/Props/SM_PaletteA_01.usd"
ENV_URL = "/Isaac/Environments/Simple_Warehouse/full_warehouse.usd"

# PALLET_URL = "omniverse://localhost/NVIDIA/Assets/ArchVis/Industrial/Pallets/Pallet_B1.usd"
# ENV_URL ="omniverse://localhost/NVIDIA/Assets/Isaac/2023.1.0/Isaac/Samples/Replicator/Stage/full_warehouse_worker_and_anim_cameras.usd"


assets_root_path = get_assets_root_path()
omni.usd.get_context().open_stage(assets_root_path + ENV_URL)
stage = omni.usd.get_context().get_stage()
floor_prims_paths = [prim.GetPath() for prim in stage.Traverse() if "SM_floor" in prim.GetName()]

floor_prims = []
for floor_prim_path in floor_prims_paths:
    floor_prims.append(omni.usd.get_prim_at_path(floor_prim_path))

def compute_bbox_with_cache(cache: UsdGeom.BBoxCache, prim: Usd.Prim) -> Gf.Range3d:
    bound = cache.ComputeWorldBound(prim)
    bound_range = bound.ComputeAlignedBox()
    return bound_range


bbcache = UsdGeom.BBoxCache(Usd.TimeCode.Default(), ["default", "render"])
bb = compute_bbox_with_cache(bbcache, floor_prims[0])
max_x, max_y, max_z = bb.GetMax()
min_x_arr = []
min_y_arr = []
min_z_arr = []
max_x_arr = []
max_y_arr = []
max_z_arr = []

for floor_prim in floor_prims:
    tmp_bb = compute_bbox_with_cache(bbcache, floor_prim)
    tmp_min_x, tmp_min_y, tmp_min_z = tmp_bb.GetMin()
    tmp_max_x, tmp_max_y, tmp_max_z = tmp_bb.GetMax()
    min_x_arr.append(tmp_min_x)
    min_y_arr.append(tmp_min_y)
    min_z_arr.append(tmp_min_z)
    max_x_arr.append(tmp_max_x)
    max_y_arr.append(tmp_max_y)
    max_z_arr.append(tmp_max_z)


min_x = min(min_x_arr)
min_y = min(min_y_arr)
min_z = min(min_z_arr)
max_x = max(max_x_arr)
max_y = max(max_y_arr)
max_z = max(max_z_arr)
margin = 3


camera = rep.create.camera()
render_product = rep.create.render_product(camera, resolution=(1024, 1024))

writer = rep.WriterRegistry.get("BasicWriter")
#out_dir = os.path.join(os.getcwd(), "_out_scattered_pallets_warehouse")
out_dir = os.path.join('/home/mona/Desktop/Omniverse_Out/Scatter_Pallet_Isaac/', "_out_scattered_pallets_warehouse")
writer.initialize(output_dir=out_dir, rgb=True, bounding_box_2d_tight=True)
writer.attach([render_product])

with rep.trigger.on_frame(num_frames=50):
    pallets = rep.create.from_usd(PALLET_URL, semantics=[("class", "pallet")], count=10)
    pallets_paths = [prim.GetPath() for prim in pallets.get_output_prims()["prims"]]
    print(f"pallets_paths: {pallets_paths}")
    with pallets:
        rep.physics.collider()
        floor_paths_as_str = [str(path) for path in floor_prims_paths]
        rep.randomizer.scatter_2d(surface_prims=floor_paths_as_str, check_for_collisions=True),
    with camera:
        rep.modify.pose(
            position=rep.distribution.uniform((min_x + margin, min_y + margin, 4), (max_x - margin, max_y - margin, 7)),
            look_at=pallets_paths,
        )

ahaidu · October 30, 2023, 8:54pm

I quickly checked the script and it seems the pallets are outside of the warehouse. You can go to the /Replicator scope and press F on the generated Xforms to find the assets in the stage.

mona.jalal · November 1, 2023, 12:49pm

Thanks for your response. I am not sure how to go to /Replicator scope/ Is there a tutorial link you can share? I know pressing F focuses. But when I click on ‘Replicator’ in ‘Stage’ in right handside and then press F, it shows black screen.

Also, my goal was to scatter pallets inside the warehouse and have the camera look at them. Do you know why the code scatters them outside and if there is a fix?

ahaidu · November 1, 2023, 2:18pm

You can expand the /Replicator scope using the [+] sign, there you have the replicator generated prims. As you can see the pallets are scattered outside the warehouse.

I would suggest starting small, with a simple scatter function and build on top of that, this makes it easier to debug.

system · November 15, 2023, 2:19pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Scatter_2d doesn't have a scale parameter for objects? Isaac Sim	5	555	March 28, 2023
Using scatter_2d in Isaac Sim Script Editor Isaac Sim	1	432	September 11, 2023
Isaac Sim Script Editor: did not match C++ signature for get_world_transform_matrix Isaac Sim	3	436	March 22, 2023
Omni.replicator.core.scripts.randomizer has no attribute scatter_boxes Isaac Sim	2	1002	January 31, 2023
Running a script in Script Editor in Isaac Sim Isaac Sim python	2	988	March 9, 2023
Scaling of pallet randomization in warehouse stage doesn't work in rep.modify.pose in Isaac Sim Script Editor interactive Isaac Sim	3	346	September 28, 2023
Randomizing the color of floor in a stage usd inside offline_generation.py Synthetic Data Generation (SDG)	13	1414	March 7, 2023
Retrieving replicator value Isaac Sim	13	709	April 17, 2024
Doubts on publishing sim_camera data via python standalone Isaac Sim	8	1361	December 15, 2023
Objects Sinking into one another (replicator python API) Isaac Sim python , isaac-sim-v4-2-0	2	29	October 20, 2024

ValueError: Unable to set target prim of type <class 'pxr.Usd.Prim'> in Isaac Sim Script Editor

Related topics