Reducing/optimizing smaller rendered Projects to compile more of them into one single project

I have been working on a project with multiple , what I call as , cells.
Each cell has its own data like robots. workbenches, turn tables, and various other components.
and have completed rendering the components on each of the cell.

I’m now trying to bring all the cells together into one Project to show a full story.
each of the cell may be around 2 - 3 GB. and there are close to 15 - 20 cells.

I tried most of the manual optimization options to make it lighter or reduce the USD size, but failed to get the required result, as many a times the process would just end or the application would just crash.
also the manual method is time consuming, and sometimes runs overnight and still wouldn’t get complete.

now I’m left with multiple rendered cells but with no suitable way of combining them,
I’m using the RTX A6000 GPU.

Has anyone ever tried to recreate something on such large scale and managed to get it done.?
any suggestions would be much appreciated.

We regular test massive scenes of 100GB, so a 2-3 gb scene even ten times is not an issue for the raw software or hardware. It must require a good workflow to optimize. When you say you have multiple “rendered” cells, what do you mean ? You mean they are already rendered to a png output frame, or you have a complete setup usd file “ready to render” with materials applied ?

The entire point of usd and usd composer it to merge MASSIVE scenes together. It sounds like you are trying to “merge” the scenes, but this is not the way. Simply start a brand new blank scene and start dragging in each cell usd file. So “cell1.usd” and “cell2.usd” etc etc. You can literally drag and drop them in and it will just reference each usd file. The new scene will be tiny. Like 100kb. But it will refer to multiple 3GB usd files. You do not actually “merge” the files. Just reference the files. It should work perfectly. If not we need to look at your workflow.

Hello Richard.

I suppose what we might be lacking is a workflow, as you mentioned.

How I started out is I first export a simulation into Omniverse, saved it as a USD and started adding materials to it.
This becomes one rendered Cell.
Each cell has a simulation now running with rendered materials.
each cell now has a USD and its Sysroot folder.

Until now I have tried the following methods.

  • Open a Main Layout USD and start bringing the other Cells / USD by dragging and dropping the USD file into the graphics Window.
    This is probably what you might say is merging.
    but after bringing 5 to 6 cells into the Parent Layout USD, the composer either goes into "Not Responding " mode, Or just Crashes.

we tried another method as well

  • Open a new USD and save it with a new name.
    drag and drop the USDs / Rendered Cells one by one.
    ( as you mentioned, this way each USD would get referenced. )
    but when we try to bring the biggest USD as reference, the composer again goes into "Not Responding " mode, Or just Crashes.

when you state Just reference the file, there is an option called " add reference " within the File Tab, do you happen to mean this option ?

I guess we need to actually understand more about the workflow part.
whether we are following the right one.?
how does the referencing work ?
are we correctly using the referencing option ?

please provide your guidance on how do I go about this,
and complete the combining of the cells into one project.

Can you provide this data so we can test ?
On the other post you mentioned about stripping out the animation of each usd file to make them work. Did that work ? Maybe it’s not the usd files or the software but the simulation data. Have you tried starting over and actually doing the simulation inside of composer ? It has excellent simulation workflow.

Can you provide your system specs. Can you provide the scene files and / or a video of the crash.

One thing to try is to only do a few cells at a time and then save the file and quit composer and re-open. Then add 3 more. And repeat. Maybe for some reason it can add some cells but then composer hangs. Repeat in cycles of 3 and see how far you get. Make sure you have no other software open.

Hello Richard,

Apologies, couldn’t get back sooner.
we did try to strip us USD off the Simulation,
however that did not make much change to the data it was still lagging.

Please find below the System specs

Make: - DELL
Model: - Precision 7820 Tower
Processor: - Intel Xeon silver 4214 CPU @ 2.20GHz, 2195 Mhz, 12 Cores, 24 Logical Processors
RAM: - 128GB
Storage: - Min 500GB SSD
Network: - Minimum 1GbE and Max 10GbE.

unfortunately we cannot provide the Scene files as we are bound under confidentiality, also capturing the video is a challenge as well, since it usually shows processing for a long time, before it crashes

I am sorry to hear this. I am not sure we can help further without the files. It sounds like you have to work within the boundaries of the limitations. All you can do is look to reduce the total load by reducing the total complexity.

Hello Richard.

Well Yes, we are still trying our best to optimize the scene for each USD and combine them.
The Part about referencing is not work as it is supposed to, as the Composer goes into Non responsive mode, once we add the second USD for referencing.

hopefully Someone else has tried this as well and may have achieved the feat.
will look forward if any other solution comes our way.

thanks & Regards

Yes I am not sure without at least the files and a video. I have a 60gb scene open now with 231 parts and it’s fine. The software can easily handle it. It may be something with the workflow or the file. The entire point of USD and USD Composer it is can just go and go and that you can load unlimited geometry. If that is the not case it can only be your computer or your files.

Hello Richard.

I could not revert sooner due to other engagements.
In the Meanwhile, we were trying to convince the concerned customer to provide us the permission to share the USD scene or the PS file.
however, they are still reluctant on the same,
and the issue still persists.

If I get the customer convinced on it, I will definitely share it across.

but considering your understanding of the challenges we have been facing,
have there been any probable solution that we could look at, in the meanwhile?
from the PS team or the Omniverse team?

I think it is just too much animated data. As I said, the only way you can solve this is more GPUs or less geometry. For less geometry, you are going to have to simplify the geometry or reduce the amount. You were going to send me just one cell remember. Did you do that ? Send me one cell over.