How to connect Omnigraph variables (read/write nodes) using python API?

Hello.

I cannot find any documentation on the correct way to connect/use Omnigraph variables in an omnigraph using the python omni.graph.core.Controller api.

I am creating variables as follows:

og.Controller.edit(
    { ... },
    {
        og.Controller.Keys.CREATE_VARIABLES: [
            ("has_inited", "bool", False),
            ("init_inv_tf", "matrixd[4]")
        ],
        ...
    }
)

And have tried to create read/write nodes to access them as follows:

og.Controller.edit(
    { ... },
    {
        og.Controller.Keys.CREATE_NODES: [
            ("ReadHasInited",       "omni.graph.core.ReadVariable"),
            ("WriteInitBool",       "omni.graph.core.WriteVariable"),
            ("ReadInitMatrix",      "omni.graph.core.ReadVariable"),
            ("WriteInitMatrix",     "omni.graph.core.WriteVariable"),
        ],
        og.Controller.Keys.CONNECT: [
            ("ReadHasInited.inputs:variableName",       "has_inited"),
            ("WriteInitBool.inputs:variableName",       "has_inited"),
            ("ReadInitMatrix.inputs:variableName",      "init_inv_tf"),
            ("WriteInitMatrix.inputs:variableName",     "init_inv_tf"),
        ],
        ...
    }
)

When inspecting the resultant omnigraph it first appears that everything is connected properly, but when running the sim the variables do not appear to do anything. Additionally, attempting to click on the read/write nodes in the gui makes them turn gray and resets their variableName attribute, rendering them useless:

Is this a bug or is there a different way that I can set up my omnigraph using python such that I can read/write to variables?

I am using Isaac Sim 4.2.0 on Ubuntu 22.04.5

Hi @s1ink ,

I had a lot of trouble with this too. The documentation on omni graphs is not practicle.
What I ended up doing is writing replicator code and inspected the omnigraph that was created. That way I knew exacly how to re-create it with the omni.graph.core library.

I can try to help you if youd like.

@danielle.sisserman Yes, please elaborate!

what are you trying to implement? we can write replicator code and then re-construct the graph that is generated with that code.

If you are less familiar with replicator, it is an API built on top of Isaac Sim. The way it works is that the replicator code is used to create an omnigraph and then at every time step the graph is triggered.

for example:
the replicator code which instantiates prims from a list of usd paths and scatters them on a plane is:

    instances = rep.randomizer.instantiate(paths = usd_paths, with_replacements = False, size = rep.distribution.choice(random_list), use_cache = False)

    with rep.utils.sequential():
        with instances:
            rep.randomizer.scatter_2d(traversable_plane, check_for_collisions=True)

Once I run that code, I can inspect the Graph that replicator generated from it:

So now, after looking at the graphs, I know what are the nodes and their connections. and I can now code just the graph by myself:

def init_graph(self, surface_prims_path, rand_usd_paths):
        self.graph = og.Controller.create_graph(
            graph_id = {
                "graph_path" : f"/Root/ScatterGraphs/{self.assets_name}Graph",
                "fc_backing_type" : og.GraphBackingType.GRAPH_BACKING_TYPE_FABRIC_WITHOUT_HISTORY, #TODO: MIGHT NEED TO REMOVE THIS
                "pipeline_stage" : og.GraphPipelineStage.GRAPH_PIPELINE_STAGE_ONDEMAND
            }
            )


        keys = og.Controller.Keys
        _, _, _, nodes = og.Controller.edit(
            self.graph,
            {
                keys.CREATE_NODES: [
                    ("OgnScatter2D", "omni.replicator.core.OgnScatter2D"),
                    ("OgnSamplePopulation", "omni.replicator.core.OgnSamplePopulation")
                    ],
                keys.CONNECT: [
                    ("OgnSamplePopulation.outputs:prims", "OgnScatter2D.inputs:prims")
                ],
                keys.SET_VALUES: [
                    ("OgnSamplePopulation.inputs:paths", rand_usd_paths),
                    ("OgnSamplePopulation.inputs:mode", 'scene_instance'),
                    ("OgnSamplePopulation.inputs:useCache", False),
                    ("OgnSamplePopulation.inputs:withReplacements", False),
                    ("OgnSamplePopulation.inputs:populationName", f"{self.assets_name}_ScatterPopulation"),
                    #("OgnScatter2D.inputs:surfacePrims", surface_prims_path),  
                    ("OgnScatter2D.inputs:checkForCollisions", self.check_for_collision),
                ]
            },
        )

Sweet. I am trying to store the pose of a prim on simulation startup (via its global transform) and then use the inverse to calculate a relative pose during subsequent iterations. Previously I used a Boolean variable to keep track of the initialization state and a matrix4d to store the transform.

I hope that’s enough for now to get started. I don’t have access to my code for another couple of hours (and forgot to push to git) but can get you more context then if you need it.

@s1ink also, were there any error/warning messages in the console when you run the sim?

@Simplychenable None on startup but when I try to access the graph in question I get:

2025-01-13 01:40:41 [37,852ms] [Error] [omni.kit.window.property.templates.simple_property_widget] Exception when async '<function SimplePropertyWidget._delayed_rebuild at 0x70e26a599c60>'
2025-01-13 01:40:41 [37,852ms] [Error] [omni.kit.window.property.templates.simple_property_widget] OmniGraphVariableRuntimeModel.__init__() takes from 5 to 6 positional arguments but 7 were given
2025-01-13 01:40:41 [37,853ms] [Error] [omni.kit.window.property.templates.simple_property_widget] Traceback (most recent call last):
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.window.property-1.11.3+10a4b5c0/omni/kit/window/property/templates/simple_property_widget.py", line 54, in wrapper
    return await func(*args, **kwargs)
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.window.property-1.11.3+10a4b5c0/omni/kit/window/property/templates/simple_property_widget.py", line 286, in _delayed_rebuild
    self._build_frame()
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.window.property-1.11.3+10a4b5c0/omni/kit/window/property/templates/simple_property_widget.py", line 269, in _build_frame
    self.build_items()
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget.py", line 569, in build_items
    self.build_nested_group_frames(stage, grouped_props)
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget.py", line 516, in build_nested_group_frames
    self._build_nested_group_frame(stage, prim_paths, display_group, 0, self._title)
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget.py", line 491, in _build_nested_group_frame
    collapse = self._build_nested_group_props(stage, prim_paths, display_group.props)
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget.py", line 425, in _build_nested_group_props
    self.build_property_item(stage, prop, prim_paths)
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.graph.ui-1.70.2+106.1.0.lx64.r.cp310/omni/graph/ui/_impl/properties_widget.py", line 538, in build_property_item
    super().build_property_item(stage, ui_prop, prim_paths)
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget.py", line 403, in build_property_item
    models = build_fn(
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.graph.ui-1.70.2+106.1.0.lx64.r.cp310/omni/graph/ui/_impl/properties_widget.py", line 913, in build_variable_prop
    model = UsdPropertiesWidgetBuilder.build(
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget_builder.py", line 304, in build
    return build_func(
  File "/home/hoodi/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.kit.property.usd-4.2.8+10a4b5c0/omni/kit/property/usd/usd_property_widget_builder.py", line 970, in gf_matrix_builder
    model = model_cls(
TypeError: OmniGraphVariableRuntimeModel.__init__() takes from 5 to 6 positional arguments but 7 were given