Instancing and Material Lookup with Omniverse Connector

Hi, I’m working on an Omniverse connector for a modeling/simulation pipeline and there are two different capabilities that I could use some help implementing. I understand the algorithms and processes but don’t know how to easily implement them using the C++ API.

The first is the ability to instance referenced USDs in a ‘parent’ USD. I’ve already written the conversion capability (individual part files) from my proprietary file type to USD but now I want to automate ‘sudo-assembly’ conversions (really just multiple different meshes instanced, no mechanical joints or anything of that nature).

The connect sample doesn’t seem to provide an example of referencing multiple USDs and transforming them.

My goal for this is just a streamlined import & transform/rotation/scale and I understand how it’s handled in the .USDA ASCII files, but I’d like to implement/automate it in my connector so I need to understand how this is performed in the C++ API.

I see a vector of USD geometry transform operations was created and used in the live edit portion of the sample but I’m having a hard time following it line by line.

If you have some small example it would be really helpful to see how multiple USDs are referenced/imported and instanced into one stage.

The other capability’s implementation is more foreign to me as it isn’t represented anywhere in the sample.

In the GTC connector tutorial video made by Brian, he discussed the pipeline for the Rhino connector. Part of that dealt with materials and while I currently don’t need to be creating any MDL materials, I want to make use of the ones already available. I want to have some kind of material lookup (just using the vMaterials library for now) and was curious if you have an example of how this can be applied (i.e. having a .csv of my textures and some corresponding MDL locations and selecting an MDL based on that table)

My background is in modeling and Pixar’s documentation, while occasionally helpful, is quite dense for me. So any code example or explanation anyone can provide would be extremely helpful.

Thank you!

Here’s some sample code for referencing USD files and transforming them:

Code for referencing USD files into a shot stage in C++ would be something like this:
        // Reference the "Models/Sphere.usd"
        {
            pxr::SdfPath pathRef = pathWorld.AppendChild(pxr::TfToken("RefSphere"));
            pxr::UsdPrim primRef = stage->OverridePrim(pathRef); // Create an "over".
            primRef.GetReferences().AddReference("Models/Sphere.usd");
            primRef.SetInstanceable(true); // Let the renderer instance the referenced part geometry if it's used more than once.
            pxr::UsdGeomXformable xform(primRef);
            xform.ClearXformOpOrder();
            pxr::GfVec3f translation(-1.25f, 0.0f, 0.0f); // Move to the left.
            pxr::UsdGeomXformOp xformOp = xform.AddTranslateOp(pxr::UsdGeomXformOp::PrecisionFloat); // Default would be PrecisionDouble.
            xformOp.Set(translation);
        }

As for referencing MDL files, if you know their location you can use the Connect Sample’s MDL shader specification code in the static void createMaterial(UsdGeomMesh meshIn) function.

Hi,

Thank you for the example! The transform process seems quite straightforward, as does the assignment of materials.

I tried mimicking this process but I’m getting hung up on the actual import/reference of an external usd and how it is working here - the pixar usd syntax is rather opaque to me, particularly the right sides of those assignments - could you help me clarify what’s going on in those first three lines?

Here’s where my understanding is lacking in those lines and the assumptions I’ve made:


			(1) pxr::SdfPath pathRef = pathWorld.AppendChild(pxr::TfToken("RefSphere"));

As I understand it, an SdfPath is basically a filesystem path. Here it looks like it starts with an absolute path to the folder containing the usd geometry file (is that what pathWorld is? It’s not defined in the shown scope), but I don’t understand what the purpose of the AppendChild function is, and by extension I don’t know what this “refSphere” token is for


			(2) pxr::UsdPrim primRef = stage->OverridePrim(pathRef); // Create an "over".

As I understand it we’re creating a USD prim here that will soon have geometry reference data ‘pushed’ to it (the geometry data won’t actually be pushed but it will be referenced). I just don’t know what the right side of this statement is doing or what the comment means by “over”


			(3) primRef.GetReferences().AddReference("Models/Sphere.usd");

This looks like it’s just setting the reference of our new prim to wherever our geometry is located. It doesn’t include the full filepath though so I assume it’s appended onto the “pathWorld” variable above in parallel to that child token (hence why I’m confused on what that token is doing).


lastly, in that fifth line, the UsdGeomXformable is our instanced object right? i.e. creating another with a different name would yield a new transformable instance.

Thank you for taking time to help me understand how this works, I really appreciate it.

Ok, so it’s not only about the references which you asked for, it’s basically about the USD C++ API as a whole.
Please keep working through the USD API reference here: https://graphics.pixar.com/usd/docs/api/index.html

That example code is an excerpt of a small Omniverse connector I wrote while still learning USD and experimenting with references.

  	(1) pxr::SdfPath pathRef = pathWorld.AppendChild(pxr::TfToken("RefSphere"));

As I understand it, an SdfPath is basically a filesystem path. Here it looks like it starts with an absolute path to the folder containing the usd geometry file (is that what pathWorld is? It’s not defined in the shown scope), but I don’t understand what the purpose of the AppendChild function is, and by extension I don’t know what this “refSphere” token is for

Yes, all node locations in your USD stage are specified by a unique SdfPath which is similar to a file system hierarchy.
This section in Pixar’s documentation describes SdfPath: SdfPath Class Reference

The pathWorld is the SdfPath to the root prim of the stage. That’s usually a UsdGeomXform.
The code which happens before that looks like this:

// With stage created....

pxr::UsdGeomSetStageUpAxis(stage, pxr::UsdGeomTokens->y);

pxr::SdfPath pathStage = pxr::SdfPath::AbsoluteRootPath();

pxr::SdfPath pathWorld = pathStage.AppendChild(pxr::TfToken("World"));
pxr::UsdGeomXform xformWorld = pxr::UsdGeomXform::Define(stage, pathWorld);

if (xformWorld)
{
    stage->SetDefaultPrim(xformWorld.GetPrim());

    // Reference the Models/Sphere.usd
    {
        ...
    }
    // Reference the same or other models here.
    ...
}

The AppendChild() function is simply creating a new SdfPath by concatenating the parent path and the child token.
So if your parent path was “World” then path.AppendChild(TfToken("Child")) creates a path “World/Child”. Simple as that.
In this example it’s using the hardcoded name “RefSphere” for the child because that node is a reference of an external USD scene named “Sphere.usd”. So the pathRef in that example becomes “World/RefSphere”.
That’s just some path. That does nothing to your stage, yet, which is the next step.

  	(2) pxr::UsdPrim primRef = stage->OverridePrim(pathRef); // Create an "over".

As I understand it we’re creating a USD prim here that will soon have geometry reference data ‘pushed’ to it (the geometry data won’t actually be pushed but it will be referenced). I just don’t know what the right side of this statement is doing or what the comment means by “over”

That command is making sure there is an UsdPrim at that path inside the stage which can be used to access and change parameters inside the later referenced *.usd. It doesn’t define what it is, yet. That’s resolved by USD internally later.

If you look into an USD ASCII file (*.usda) which you can create from any binary *.usd file by using the usdcat helper tool inside the USD repository, then you’ll see that referenced layers are represented by an “over” keyword.
That comes from the C++ API function UsdStage::OverridePrim().
You need that to be able to override individual parameters like the transform or material assignments inside the referenced UsdPrim hierarchy.

Please read the whole USD Introduction and esp. this chapter on Referencing Layers: https://graphics.pixar.com/usd/docs/Referencing-Layers.html
and this chapter inside the USD API reference: https://graphics.pixar.com/usd/docs/api/_usd__page__scenegraph_instancing.html
which go into much more detail about references and instancing.
That’s all I used to get that example code working. Well, I needed to translate the introductory Python example code back to C++ by doing a lot of searches in the USD API reference.

  	(3) primRef.GetReferences().AddReference("Models/Sphere.usd");

This looks like it’s just setting the reference of our new prim to wherever our geometry is located. It doesn’t include the full filepath though so I assume it’s appended onto the “pathWorld” variable above in parallel to that child token (hence why I’m confused on what that token is doing).

Yes, the primRef which has been added to the stage gets a reference attached at the back of the prepend list.
Explained here: https://graphics.pixar.com/usd/docs/api/class_usd_references.html

In this case that is a relative path to the location of the current stage.
So let’s say the current stage we are working on has been created in omniverse://localhost/Users/Shot.usd the referenced file is in omniverse://localhost/Users/Models/Sphere.usd

Lastly, in that fifth line, the UsdGeomXformable is our instanced object right? i.e. creating another with a different name would yield a new transformable instance.

No, the primRef is the handle of the referenced object. It’s a layer of the UsdGeomXform of the referenced stage Models/Sphere.usd.
To be able to change the transformations (translation, rotation, scale, or full matrix), you need to access the UsdGeomXformable of that prim. That xform variable is just the handle to that transformable.
Explained here: https://graphics.pixar.com/usd/docs/api/class_usd_geom_xformable.html#details

Note that UsdGeomMeshes themselves can be transformed in USD as well, but since USD doesn’t like referencing UsdGeomeMeshes itself, but always wants to reference a node above that to define a “scope” since referenced objects can only access things in their scope, matters get rather complicated in USD when you really only want to instance mesh geometry.
There is this sentence inside the “scenegraph_instancing” link above which explains the reasoning:
“Since instancing shares the scenegraph hierarchy beneath instance prims, instancing a single prim that has no descendants provides no benefits. In the case that instancing a single prim is desired (e.g., instancing a mesh), that prim should be made a descendant of another prim that is referenced into the scene”)

You’ll get all sorts of warnings along the lines of “cannot access objects outside the referenced scope” when not adhering to that instancing methodology.

If you reference one and the same *.usd multiple times, add the set the instanceable flag with SetInstanceable(true) on the referenced prim to let USD generate a single instance which is used multiple times.
In Omniverse you can see the effect of that when looking into the RTX renderer statistics and compare the number of instances and the number of meshes in geometry acceleration structures.
To reuse these meshes is the whole point of this referencing and instancing.

When reading the USD documentation about this, related topics are “internal references”, “payload”, variant set", and the whole layering concept in general.

1 Like

Hi @droettger!

Thank you for your response, your explanations were immensely helpful in aiding my understanding of relating the API back to the structure of a USD.

I’ve read through many parts of the API documentation, but I’ve skipped around a bit and occasionally find the lack of examples (specifically the lack of relation of c++ methods to their effect on the ASCII usda files) lead me to not able to apply the information with much confidence. Any tutorials on pixar’s other site are all for python and USDA so there’s much trial/error and guesswork on my end in trying to reconcile the two.

That aside,

To test the process we talked about above, I imported and instanced a mesh object from Pixar’s ‘Kitchen Scene’ and it seemed to work without any issue, it gave the following debug info in the RTX Rendering Statistics window you mentioned:

It shows that we have rendered two different meshes and six total instances, which makes perfect sense because this book object is made up of two meshes. Yay! :)



There is one other major issue I’m having with instancing, and I’m not sure exactly what’s going wrong - but I’ve narrowed it down a bit. Any time I try to reference a USD generated from my connector, no geometry shows up. Here is what it looks like in create with two successfully referenced and instanced books, but two failed imports of my illuminator geometry (I removed materials on my objects for now).

I realize the top level kitchen stage uses instancing for all of the subassemblies/parts so there must be some step in the process generating those usds that I’m missing

My connector mimics the stage creation process used in the NVIDIA connector example, so I figured I’d go back to that super basic cube example and see if I missed something in translation from there.

The result I ended up with was the same as when I use my connector, there is an imported/referenced object in the tree but there isn’t any geometry data inside.

I thought it might be because I just had all of the individual meshes underneath the root branch of the model tree, but resolving that by adding a branch for geometry and reorganizing elements in create didn’t seem to provide a different result.

I thought this might work because of what you said but also because of something I read in the USDReferences section of the API documentation:

Reasons why adding a reference may fail, why adding a reference may succeed but still generate errors, and what it all means

AddReference() and SetReferences() can each fail for a number of reasons. If one of the specified prim targets for one of the references is not a prim, we will generate an error, fail to author any scene description, and return false . If anything goes wrong in attempting to write the reference, we also return false, and the reference will also remain unauthored. Otherwise, if the reference was successfully authored, we will return true . A successful reference authoring operation may still generate composition errors! Just because the reference you specified was syntactically correct and therefore successfully authored, does not imply it was meaningful. If you wish to ensure that the reference you are about to author will be meaningfully consumable by your stage, you are strongly encouraged to ensure it will resolve to an actual file by using UsdStage::ResolveIdentifierToEditTarget() before authoring the reference.

When adding an internal reference, the given prim path is expected to be in the namespace of the owning prim’s stage. Sub-root prim paths will be translated from this namespace to the namespace of the current edit target, if necessary. If a path cannot be translated, a coding error will be issued and no changes will be made. Non-sub-root paths will not be translated.

Immediately upon successful authoring of the reference (before returning from AddReference(), RemoveReference(), ClearReferences(), or SetReferences()), the UsdStage on which the reference was authored will recompose the subtree rooted at the prim hosting the reference. If the provided identifier does not resolve to a layer that is already opened or that can be opened in the usd format, or if the provided primPath is not an actual prim in that layer, the stage’s recomposition will fail, and pass on composition errors to the client.

The function mentioned: UsdStage::ResolveIdentifierToEditTarget() is returning a blank string for my models, which is definitely not a promising sign according to the API document.

I figure there must be a silly mistake I’m making somewhere, could you please point me in the right direction?

One other note is that when I import the object using the create UI, it comes in as a transform with no geometry data, whereas if I do it using c++ it comes in without a type at all. To reiterate, the above image imported:
2 instances of the book that I got from the Pixar kitchen scene
1 illuminator imported from my connector
1 illuminator imported from create’s UI (same exact file as the one above)

If necessary I will append the shown attempted usd file below but to save your time, I’ll show the two .usda files here: I’ll show the one containing the book geometry and the one containing the illuminator geometry.

Illuminator.usda (7.4 MB)
Book.geom.usda (94.0 KB)

Is there a significant difference in structure that is causing my issue? Each file has a root level transform and one transform beneath it which is holding geometry data.

Thank you for your help,

-Matthew



These shouldn’t be necessary but in case there’s valuable information in them:






Here is the assembly file
Book_and_Illuminator_Import_Example.usd (2.0 KB)

Here is the book model and the satellite/illuminator model:
Book.usd (529 Bytes)
Illuminator.usd (2.2 MB)

I believe these are also dependents for the book file, as the geometry is actually contained in this second file:
Book_payload.usd (229 Bytes)
Book.geom.usd (24.9 KB)

I’ve read through many parts of the API documentation, but I’ve skipped around a bit and occasionally find the lack of examples (specifically the lack of relation of c++ methods to their effect on the ASCII usda files) lead me to not able to apply the information with much confidence. Any tutorials on pixar’s other site are all for python and USDA so there’s much trial/error and guesswork on my end in trying to reconcile the two.

That describes my experience exactly.

I used your Book_and_Illuminator_Import_Example.usd, the Illuminator.usda, and the Book.geom.usda.
I also converted all other *.usd files to *.usda with Pixar’s usdcat tool to see where these expect their files.

Then I build a Book_and_Illuminator.usda scene manually inside a notepad referencing the Book.geom.usda twice and the Illuminator.usda.

First of all, always clear the Console output in Create before loading a new scene.
If anything is not working, look at all errors.
The three icons in the top right of the console allow toggling the display of the information, warnings, and errors like a DIP switch.

Fix all errors in your scene and repeat until there are no more errors.
If you do not change the name of the scene every time you change it, you need to clear the loaded scene by menu File → New before loading it again.

1.) You wanted to reference the Book.usd file from the relative folder references/Book.
But the original Book.usd expects itself to be in the folder ./assets/Book.
That’s crucial since that Book.usd itself is referencing a payload Book_payload.usd which is also located inside the ./assets/Book.

You cannot simply copy around individual files from another scene without adhering to its asset structure!
Everybody working on a USD stage has to agree on a common file structure and typical scene hierarchy, which apparently has developed over the years at Pixar and also is no problem for generally static movie assets.
You need to copy everything in its required folder structure and reference from there.

I used the Book.geom.usda instead, which is not using a payload and I renamed the “references” folder back to “assets” to match what the original Book.usd requires, in case you want to use that, and copied the Illuminator.usda into that assets folder as well, and changed the references’ paths inside the Book_and_Illuminator.usda.

2.) I added a defaultPrim variable to all files because that is crucial when not referenceing individual nodes (like shown in the ParkingLot example inside the USD API reference I linked above). USD needs to know which prim to pick when referencing a *.usd file.

3.) I removed the light from the Illuminator.usda.
That was under the now defaultPrim “Root” node and was imported as well, which made the image brighter when adding that DistantLight via Create.

With these changes (all done by typing in an editor on USDA files) the attached Book_and_Illuminator.zip scene worked the way I think you wanted.
Now you just need to implement that same structure inside your connector and it should result in the same scene.
The books look so bright because the distant light shines directly from above.

Book_and_Illuminator.zip (1.0 MB)

1 Like

@droettger These points and the properly formatted USDA were tremendously helpful! Thank you so much!