[error][omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator

danielle.sisserman · March 20, 2024, 1:33pm

I have a simulation with an autonomous robot and some randomization. The point of this simulation is to run it for hours and collects large amount of data. sometimes, during the run of the scene, i get a none-ending flow of error messages:

from the log file:


2024-03-20 13:24:49 [1,281,226ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-20 13:24:49 [1,281,228ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-20 13:24:49 [1,281,240ms] [Info] [omni.kit.app._impl] [py stdout]: This instance is being destroyed

2024-03-20 13:24:49 [1,281,265ms] [Warning] [omni.anim.skelJoint.plugin] Prim /Render/RenderProduct_Replicator/PostRender/SDGPipeline/RenderProduct_Replicator_LdrColorSDPostCopyToHost is invalid when skeljoint processing prim resync
2024-03-20 13:24:49 [1,281,266ms] [Warning] [omni.anim.skelJoint.plugin] Prim /Render/RenderProduct_Replicator/PostRender/SDGPipeline/RenderProduct_Replicator_InstanceMappingPost is invalid when skeljoint processing prim resync
2024-03-20 13:24:49 [1,281,266ms] [Warning] [omni.anim.skelJoint.plugin] Prim /Render/RenderProduct_Replicator/PostRender/SDGPipeline/RenderProduct_Replicator_DistanceToCameraSDPostCopyToHost is invalid when skeljoint processing prim resync
2024-03-20 13:24:49 [1,281,274ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:49 [1,281,285ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,312ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,320ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,337ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,366ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,374ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,391ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,421ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,429ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,446ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,493ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,501ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,520ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,536ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,552ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,568ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,585ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,601ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-20 13:24:50 [1,281,619ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator

Its seems to be related to the viewport in the editor. The entire simulation freezes and the error messages are flooding the consule. What is it and how can i solve it?

update: logs from another run:

2024-03-21 08:12:07 [772,535ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,535ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,536ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,537ms] [Warning] [omni.fabric.plugin] addAttributeInternal: invalid attribute type.
2024-03-21 08:12:07 [772,537ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,538ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,541ms] [Warning] [omni.fabric.plugin] addAttributeInternal: invalid attribute type.
2024-03-21 08:12:07 [772,541ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,542ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,542ms] [Warning] [omni.fabric.plugin] addAttributeInternal: invalid attribute type.
2024-03-21 08:12:07 [772,543ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,543ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,543ms] [Warning] [omni.fabric.plugin] addAttributeInternal: invalid attribute type.
2024-03-21 08:12:07 [772,544ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,544ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,545ms] [Warning] [omni.fabric.plugin] addAttributeInternal: invalid attribute type.
2024-03-21 08:12:07 [772,545ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,546ms] [Warning] [omni.fabric.plugin] addAttributeInternal 0: invalid attribute type.
2024-03-21 08:12:07 [772,551ms] [Info] [omni.kit.app._impl] [py stdout]: This instance is being destroyed

2024-03-21 08:12:07 [772,588ms] [Warning] [omni.anim.skelJoint.plugin] Prim /Render/RenderProduct_Replicator/PostRender/SDGPipeline/RenderProduct_Replicator_LdrColorSDPostCopyToHost is invalid when skeljoint processing prim resync
2024-03-21 08:12:07 [772,588ms] [Warning] [omni.anim.skelJoint.plugin] Prim /Render/RenderProduct_Replicator/PostRender/SDGPipeline/RenderProduct_Replicator_InstanceMappingPost is invalid when skeljoint processing prim resync
2024-03-21 08:12:07 [772,588ms] [Warning] [omni.anim.skelJoint.plugin] Prim /Render/RenderProduct_Replicator/PostRender/SDGPipeline/RenderProduct_Replicator_DistanceToCameraSDPostCopyToHost is invalid when skeljoint processing prim resync
2024-03-21 08:12:07 [772,600ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:07 [772,610ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:07 [772,627ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:07 [772,654ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:08 [772,662ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:08 [772,678ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:08 [772,704ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator
2024-03-21 08:12:08 [772,712ms] [Error] [omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator

Thank you

ahaidu · March 21, 2024, 10:15am

Hi there, can you provide the isaac sim version you are using?

Does this error happen every run? Or only when it reaches some particual situation in the randomizations?

Is it possible to provide a repro snippet with the error?

Or snippets on how you set up your scenario and randomizations?

Thanks!

danielle.sisserman · March 21, 2024, 5:09pm

Hi @ahaidu,

I am using isaac sim 2023.1.1

I have a simulator extension with a menu and a start button. When i press start a render product is created and attached to my custom writer.

def init_writer(self, cam_path, resolution, model_yaml_path, colorizedSegmentation):


        rep.WriterRegistry.register(MowerWriter)


        self.rp = rep.create.render_product(cam_path, resolution)


        self.writer = rep.WriterRegistry.get(WRITER_NAME)


        self.writer.initialize(rgb_output_path = self.paths.rgb_directory_path,
                               seg_output_path = self.paths.segmented_directory_path,
                               depth_output_path = self.paths.depth_directory_path,
                               metadata_path = self.paths.data_directory_path,
                               model_yaml_path = model_yaml_path,
                               colorizedSegmented = colorizedSegmentation
                               )


        self.writer.attach(render_products = [self.rp])

To re-run a usd with my extension I reload the usd and re-enable my extension. When this error message floods the consule, I need to re-open the usd for it to stop. By in most cases, once i press Play (my extension button) the error re appears. only shuttting down and re starting isaac sim will give me a few runs before it happens again.

Can you explain please what this error is about? Where is the path /Render/RenderProduct_Replicator?

is there usually a USD RenderProduct? Where can I find it in cases everything works as expected?

Thank you

ahaidu · March 21, 2024, 7:00pm

Regarding the render product path, you can find them in the session layer (Layer → Burger Menu → Show Session Layer):

Here is a script you can run in the script editor on how the render products are named:

import omni.replicator.core as rep


cam_1 = rep.create.camera(name="cam_1")
cam_default_name = rep.create.camera()

rp = rep.create.render_product("/OmniverseKit_Persp", (512, 512))
rp_named = rep.create.render_product("/OmniverseKit_Persp", (512, 512), name="rp_persp_2")
rp_cam_1 = rep.create.render_product("cam_1", (512, 512), name="rp_cam_1")
rp_default_name = rep.create.render_product("cam_default_name", (512, 512), name="rp_cam_default_name")

ahaidu · March 21, 2024, 7:50pm

Could you provide the replicator version you are using?

danielle.sisserman · March 24, 2024, 12:35pm

hi @ahaidu , can you please instruct mewhere I can find the version of replicator. Thank you

danielle.sisserman · March 24, 2024, 1:04pm

@ahaidu where can i find this package? [omni.kit.app._impl]

I can only find kit.property and kit.loop in /AppData/Local/ov/pkg/isaac_sim-2023.1.1/ext

for some reason it is destroying RenderProduct_Replicator

Thank you

ahaidu · March 24, 2024, 3:47pm

You can take a look in Window-->Extensions by searching for omni.replicator.core

Or in the terminal after starting isaac sim. The extension is located in isaac_sim/extscache it can however contain multiple versions of extensions, you will know for sure which is loaded by going over the above ways.

ahaidu · March 24, 2024, 3:47pm

Could it be that your extension is doing more that that, and it is somehow deleting the render product?

danielle.sisserman · March 24, 2024, 4:41pm

@ahaidu It turns out that [Info] [omni.kit.app._impl] [py stdout] is from the print function. I have a deconstructor in my DataGenerator class:

 def __del__(self):
        print("This instance is being destroyed")

and that class holds the refrence to the render product. But i dont understand why the garbage collector will collect the instance of DataGenerator in the middle of the simulation. The Simulator class holds reference to it (self.data_generator)

danielle.sisserman · March 24, 2024, 7:06pm

@ahaidu ,

after debugging my code, I see that the instance of Simulator() is being collected by garbage collection (entering it’s own del()).

My Simulator() class has a Start function and a Stop functions which are called from UIdelegate when I press the buttons in the UI window.
It also holds all the components of the simulator such as an instance to the DataCollector and all the different parameters of the Simulations.

How can I ensure that the Simulator instance is collected after the simulation ends (I press stop). Is there in depth documentation of the lifespan of variables in an omniverse extension?

Thank you

Topic		Replies	Views
Isaac Sim 4.0: Invalid USD RenderProduct Prim Isaac Sim	0	105	July 2, 2024
Replicator sometimes seems to break the existing Camera Isaac Sim	3	1000	August 29, 2023
Isaac Sim 2022.2.0 freez at startup after first time Isaac Sim	13	1700	March 24, 2023
Randomizer based Replicator code gets slower every frame Synthetic Data Generation (SDG) python	15	1509	July 13, 2023
Sequentially creating and distroying render products/ writers Isaac Sim	0	125	July 12, 2024
What is HydraEngine? How does it relate to Replicator? Why would it fail to end compute graph? Isaac Sim isaac-sim-v4-0-0	11	339	January 8, 2025
RuntimeError: Failed to acquire interface: omni::kit::IViewport (pluginName: nullptr) Isaac Sim	10	2635	March 15, 2023
Degradation of rendering capabilities of Isaac-sim on 4.2.0 and 4.1.0 Isaac Sim isaac-sim-v4-2-0	7	356	October 21, 2024
It turns into a white asset without being colored in the assets Synthetic Data Generation (SDG)	7	580	December 5, 2023
Trying to run Replicator demo from documentation, kinda works but lots of "[omni.hydra] Failed to create MDL shade node for prim" type errors Synthetic Data Generation (SDG)	3	1009	April 4, 2023

[error][omni.hydra] Invalid USD RenderProduct Prim: /Render/RenderProduct_Replicator

Related topics